Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatio-Anatomical Ontology
1 like669 views
We present an approach using medical expert knowledge represented in formal ontologies to check the results of automatic medical object recognition algorithms for spatial plausibility. Our system is based on the comprehensive Foundation Model of Anatomy ontology which we extend with spatial relations between a number of anatomical entities. These relations are learned inductively from an annotated corpus of 3D volume data sets. The induction process is split into two parts. First, we generate a quantitative anatomical atlas using fuzzy sets to represent inherent imprecision. From this atlas we then abstract the information further onto a purely symbolic level to generate a generic qualitative model of the spatial relations in human anatomy. In our evaluation we describe how this model can be used to check the results of a state-of-the-art medical object recognition system for 3D CT volume data sets for spatial plausibility. Our results show that the combination of medical domain knowledge in formal ontologies and sub symbolic object recognition yields improved overall recognition precision.
![15
1/30/2015
? Classical Logic:
leftFrom(left kidney, right kidney) ? {0,1}
? Fuzzy Logic:
leftFrom(left kidney, right kidney) ? [0,1]
? Representation of direction in 2D:
0
?¦Ð
+/-¦Ð
-?¦Ð
R
T
right
above
left
below
¦Á
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
unten rechts oben
cos^2(x)
-?¦Ð ?¦Ð0
Truthvalue
angle
below aboveright
Daniel Sonntag, Daniel.Sonntag@dfki.de](https://image.slidesharecdn.com/kdir2010-moeller-2-101107133904-phpapp02/85/Automatic-Spatial-Plausibility-Checks-for-Medical-Object-Recognition-Results-Using-a-Spatio-Anatomical-Ontology-15-320.jpg)
![18
1/30/2015
Spatial Relations in OWL Model
? Extension of the formalism in the
Foundational Model of Anatomy
[0..1]
left|right|
above|¡
term
truthValue
Instance of SimpleFuzzyRelation
Anatomical Entity B
Anatomical Entity A
location
related
Object
FuzzySpatialAssoc
iationRelation
type
[0..1]
left|right|
above|¡
directionalTerm
truthValue
Instance of SimpleFuzzyRelation
[0..1]
left|right|
above|¡
directionalTerm
truthValue
Instance of SimpleFuzzyRelation
[0..1]
left|right|
above|¡
term
truthValue
Instance of SimpleFuzzyRelation
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
Daniel Sonntag, Daniel.Sonntag@dfki.de](https://image.slidesharecdn.com/kdir2010-moeller-2-101107133904-phpapp02/85/Automatic-Spatial-Plausibility-Checks-for-Medical-Object-Recognition-Results-Using-a-Spatio-Anatomical-Ontology-18-320.jpg)
More Related Content
Similar to Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatio-Anatomical Ontology (20)
Recently uploaded (20)
![[Webinar] Scaling Made Simple: Getting Started with No-Code Web Apps](https://cdn.slidesharecdn.com/ss_thumbnails/webinarscalingmadesimplegettingstartedwithno-codewebapps-mar52025-250305183437-f03c78a3-thumbnail.jpg?width=560&fit=bounds)
Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatio-Anatomical Ontology
- 1. 1 1/30/2015 Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatial-Anatomical Ontology Manuel M?ller, Patrick Ernst, Andreas Dengel, Daniel Sonntag German Research Center for Artificial Intelligence University of Kaiserslautern
- 2. 2 1/30/2015 With the shift to the application of digital imaging techniques for medical diagnosis, such as CT, MRI, etc., the volume of digital images produced in modern clinics increased tremendously. Our clinical partner,the University Hospital Erlangen in Germany, has a total of about 50 TB of medical images. Currently, they have about 150,000 medical examinations producing 13 TB of data per year.
- 3. 3 1/30/2015 MEDICO, RadSem and RadSpeech: A Mashup
- 4. 4 1/30/2015 Automatic object recognition algorithms
- 5. 5 1/30/2015 Our approach is to augment medical domain ontologies and allow for an automatic detection of anatomically implausible constellations in the results of a state-of-the-art system for automatic object recognition in 3D CT scans. The output of our system also provides feedback which anatomical entities are most likely to have been located incorrectly.
- 7. 7 1/30/2015 Motivation ? Automatic object recognition algorithms available for several organs in 3D ? Perform reasonably well (in many cases) ? Integration with anatomical background knowledge: Foundational Model of Anatomy Daniel Sonntag, Daniel.Sonntag@dfki.de
- 8. 8 1/30/2015 Proposed Solution: Integration of automatic object recognition algorithms with high-level knowledge about human anatomy. Motivation In some cases the automatic object recognition is grossly wrong. Daniel Sonntag, Daniel.Sonntag@dfki.de
- 9. 9 1/30/2015 Outline 1. Goals and Prerequisites 2. Hierarchical Algorithm for Learning Spatial Relations 3. Application to Automatic Object Recognition 4. Conclusion Daniel Sonntag, Daniel.Sonntag@dfki.de
- 10. 10 1/30/2015 Goals and Prerequisites ? Goals: ¨C bridge semantic gap between low-level and high-level information ¨C develop system integrating information from both sources ¨C perform reasoning to check plausibility of object recognition results ? Prerequisites: Automatic object recognition algorithms Structural anatomical knowledge Spatial relations of human anatomy Integration of low-level and high-level information Daniel Sonntag, Daniel.Sonntag@dfki.de
- 11. 11 1/30/2015 Inductive Approach ? Qualitative representation: left/right,¡ ? Human Anatomy inherently variable ? Fuzzy ? Data-driven Patient 1 Patient 2 Patient 3 Patient 4 Patient 5 Canonical Anatomy Daniel Sonntag, Daniel.Sonntag@dfki.de
- 12. 12 1/30/2015 Hierarchical Algorithm for Learning Spatial Relations Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
- 13. 13 1/30/2015 Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus ? Collected at the University Hospital Erlangen ? from 2002 to 2008 ? Cancer patients with lymphoma ? 3D volume data sets from computer tomography scanners ? Available image data Daniel Sonntag, Daniel.Sonntag@dfki.de
- 14. 14 1/30/2015 ? Statistical algorithms for the detection of various anatomical entities ¨C Constrained MSL ¨C Hierarchical Active Shape Models ¨C Patch-based Deformable Models ¨C Trainable Boundary Detector see: Seifert, Kelm, M?ller, Mukherjee, Cavallaro, Huber, Comaniciu: ¡°Semantic Annotation of Medical Images¡±, SPIE Medical Imaging 2010 Results: ? Meshes: 6 different organs left/right kidney, left/right lung, urinary bladder, prostate gland ? Landmarks: 22 exposed points: top point of the liver, ¡ ? Manually generated gold standard of automatically annotated volume data sets: 1 017 labeled volumes Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
- 15. 15 1/30/2015 ? Classical Logic: leftFrom(left kidney, right kidney) ? {0,1} ? Fuzzy Logic: leftFrom(left kidney, right kidney) ? [0,1] ? Representation of direction in 2D: 0 ?¦Ð +/-¦Ð -?¦Ð R T right above left below ¦Á Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 unten rechts oben cos^2(x) -?¦Ð ?¦Ð0 Truthvalue angle below aboveright Daniel Sonntag, Daniel.Sonntag@dfki.de
- 16. 16 1/30/2015 Patient 5 Patient 4 Patient 3 Patient 2 Patient 1 Natural variability in human anatomy R Z ¦Á=3¡ã ?right kidney right from left kidney¡°: Truth value Absolutefrequency 10 R Z ¦Á=0¡ã R Z¦Á=4¡ã R Z¦Á=4¡ã R Z ¦Á=0¡ã avg= 0,92¡ Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
- 17. 17 1/30/2015 A B (a) A B (b) (c) Right Left C B A Relation Types Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus ? Direction: ?left kidney left from right kidney¡° ? Adjacency: ?prostate adjacent to urinary bladder¡° ? Between: ?bronchial bifurcation between left and right lung¡° ? Evaluated with medical experts Daniel Sonntag, Daniel.Sonntag@dfki.de
- 18. 18 1/30/2015 Spatial Relations in OWL Model ? Extension of the formalism in the Foundational Model of Anatomy [0..1] left|right| above|¡ term truthValue Instance of SimpleFuzzyRelation Anatomical Entity B Anatomical Entity A location related Object FuzzySpatialAssoc iationRelation type [0..1] left|right| above|¡ directionalTerm truthValue Instance of SimpleFuzzyRelation [0..1] left|right| above|¡ directionalTerm truthValue Instance of SimpleFuzzyRelation [0..1] left|right| above|¡ term truthValue Instance of SimpleFuzzyRelation Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Daniel Sonntag, Daniel.Sonntag@dfki.de
- 19. 19 1/30/2015 ? Example: Learning spatial relations ? Comparison between learned model and new object recognition result ? Results: true positives 407 true negatives 431 false positives 67 false negatives 213 precision 85,7% recall 65,5% Patient 1Patient 2 Patient 3 Patient 4 Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Incorrectly located organ Spatial Consistency Check Daniel Sonntag, Daniel.Sonntag@dfki.de
- 20. 20 1/30/2015 Medico Server ?MEDICO Ontology ?Sesame Triplestore ?>2 Mio. Triples ?Semantic Annotation Store ?3D Volume Renderer ?Based on MITK State of the Art Organ and Landmark Detection Ontology-based Visual Navigation Application Central Java-based Data Exchange Application MEDICOServer Volume Parser MITK Semantic Navigation Semantic Search XMLRPC SPARQL CORBA CORBA CORBA Java API CTC-WP4 Triple Store Query Broker RadSpeech
- 21. 21 1/30/2015 Retrieval and examination of 2D/3D image series
- 22. 22 1/30/2015 Conclusion ? Hierarchical abstraction process to learn spatial relations from annotated volume data sets ? Method for the generation of a fuzzy anatomical atlas from different patients ? Spatial consistency check comparing automatic object recognition results with Daniel Sonntag, Daniel.Sonntag@dfki.de Pre-Processing Automatic Annotation Fuzzy Atlas Qualitative Representation Spatial Reasoning Corpus Patient 5 Patient 4 Patient 3 Patient 2 Patient 1 R Z ¦Á=3¡ã R Z ¦Á=0¡ã R Z¦Á=4¡ã R Z¦Á=4¡ã R Z ¦Á=0¡ã ?right kidney right from left kidney¡°: Truth value Absolutefrequency 10