Illumination Independent Marker Tracking using Cross-Ratio Invariance
0 likes1,147 views
The document presents a method for illumination independent marker tracking using cross-ratio invariance, addressing the challenges posed by varying light conditions on corner detection. It outlines the proposed correction strategy to detect and adjust corner shifts caused by non-linear mappings of scene radiance and measured brightness, providing validation results that demonstrate improved depth estimation accuracy. The discussion highlights future directions for refining the method, including analytical formulations for minimization.
![Motivation
Medical applications using markers
A low cost and accurate guidance system for
laparoscopic surgery : Validation on an
abdominal phantom [Nicolau VRST2005]
A low cost simulator to practice ultrasound
image interpretation and probe manipulation :
Design and rst evaluation [Nicolau ISVRI2011]
Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 4 / 18](https://image.slidesharecdn.com/vagslides-141111151837-conversion-gate01/85/Illumination-Independent-Marker-Tracking-using-Cross-Ratio-Invariance-8-320.jpg){kind=tracking}
![Motivation
Medical applications using markers
Error on tag position
) Larger position errors for :
Needle tip
UltraSound virtual plane
A low cost and accurate guidance system for
laparoscopic surgery : Validation on an
abdominal phantom [Nicolau VRST2005]
A low cost simulator to practice ultrasound
image interpretation and probe manipulation :
Design and rst evaluation [Nicolau ISVRI2011]
Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 4 / 18](https://image.slidesharecdn.com/vagslides-141111151837-conversion-gate01/85/Illumination-Independent-Marker-Tracking-using-Cross-Ratio-Invariance-9-320.jpg){kind=tracking}
Illumination Independent Marker Tracking using Cross-Ratio Invariance
- 1. Illumination Independent Marker Tracking using Cross-Ratio Invariance Vincent Agnus St辿phane Nicolau Luc Soler Institut de Recherche Contre les Cancers de l'Appareil Digestif ACM Symposium on Virtual Reality Software and Technology (VRST2014) November 11, 2014 Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 1 / 18
- 2. Outline Causes & Eects of light condition on corner detection How cross-ratio can be used to correct corner shifting Method validation Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 2 / 18
- 3. Motivation Corner Detection : Illumination Conditions Example Light conditions inuence the estimation of corner position Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 3 / 18
- 4. Motivation Corner Detection : Illumination Conditions Example Light conditions inuence the estimation of corner position Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 3 / 18
- 5. Motivation Corner Detection : Illumination Conditions Example Light conditions inuence the estimation of corner position Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 3 / 18
- 6. Motivation Corner Detection : Illumination Conditions Example Light conditions inuence the estimation of corner position Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 3 / 18
- 7. Motivation Corner Detection : Illumination Conditions Example Light conditions inuence the estimation of corner position Outcomes : Errors Image : 2 pixels ) World : 3 mm depth ) medical applications Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 3 / 18
- 8. Motivation Medical applications using markers A low cost and accurate guidance system for laparoscopic surgery : Validation on an abdominal phantom [Nicolau VRST2005] A low cost simulator to practice ultrasound image interpretation and probe manipulation : Design and rst evaluation [Nicolau ISVRI2011] Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 4 / 18
- 9. Motivation Medical applications using markers Error on tag position ) Larger position errors for : Needle tip UltraSound virtual plane A low cost and accurate guidance system for laparoscopic surgery : Validation on an abdominal phantom [Nicolau VRST2005] A low cost simulator to practice ultrasound image interpretation and probe manipulation : Design and rst evaluation [Nicolau ISVRI2011] Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 4 / 18
- 10. Motivation Corner Detection Shift under light conditions Shifting Causes : Mitsunaga and Nayar 1999 non-linear mapping between scene radiance and measured brightness measured brightness rasterisation (spatial and intensity) on CDD White Black Scene Radiance state of the art Studies : marker detection , shifting ) Biased images Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 5 / 18
- 11. Motivation Corner Detection Shift under light conditions Shifting Causes : Mitsunaga and Nayar 1999 non-linear mapping between scene radiance and measured brightness measured brightness rasterisation (spatial and intensity) on CDD White Black Radiometric Response Scene Radiance Measured Brightness state of the art Studies : marker detection , shifting ) Biased images Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 5 / 18
- 12. Motivation Corner Detection Shift under light conditions Shifting Causes : Mitsunaga and Nayar 1999 non-linear mapping between scene radiance and measured brightness measured brightness rasterisation (spatial and intensity) on CDD White Black Radiometric Response Scene Radiance Measured Brightness state of the art Studies : marker detection , shifting ) Biased images Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 5 / 18
- 13. Motivation Corner Detection Shift under light conditions Shifting Causes : Mitsunaga and Nayar 1999 non-linear mapping between scene radiance and measured brightness measured brightness rasterisation (spatial and intensity) on CDD White Black Rasterisation Radiometric Spatial Intensity Response Scene Radiance Measured Brightness state of the art Studies : marker detection , shifting ) Biased images Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 5 / 18
- 14. Proposed Method Key Idea Light Condition underexposure : black region %, white region overexposure : black region , white region % ) error on estimated position of the marker How we can detect and correct this phenomena ? Our method use the marker geometry : inner outer corners check that white/black region proportion are preserved Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 6 / 18
- 15. Proposed Method Corner Detection Shift Detection Correction Cross Ratio (CR) A projective invariant feature dened for collinear points sets A B C D A0 B0 C0 D0 CR = (AB=AC) (DB=DC) = (A0B0=A0C0) (D0B0=D0C0) Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 7 / 18
- 16. Proposed Method Corner Detection Shift Detection Correction Cross Ratio (CR) A projective invariant feature dened for collinear points sets A B C D A0 B0 C0 D0 CR = (AB=AC) (DB=DC) = (A0B0=A0C0) (D0B0=D0C0) Cross Ratio on markers A BC D B A C D CR should be constant : position light Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 7 / 18
- 17. Proposed Method Theoretical Cross Ratio vs Measured Cross Ratio Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 8 / 18
- 18. Proposed Method Theoretical Cross Ratio vs Measured Cross Ratio CRmodel Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 8 / 18
- 19. Proposed Method Theoretical Cross Ratio vs Measured Cross Ratio CRmodel Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 8 / 18
- 20. Proposed Method Theoretical Cross Ratio vs Measured Cross Ratio CRmodel Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 8 / 18
- 21. Proposed Method Cross Ratio : selected points Cross Ratio : Our Study A BC D Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 9 / 18
- 22. Proposed Method Cross Ratio : selected points Cross Ratio : Our Study A BC D 4+4+2 lines for CR Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 9 / 18
- 23. Proposed Method Cross Ratio : selected points Cross Ratio : Our Study A BC D 4+4+2 lines for CR Cross Ratio : inner corners of the marker (future) Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 9 / 18
- 24. Proposed Method Correction of corner shifts How cross ratio invariance can be used to x corners shifting ? Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 10 / 18
- 25. Proposed Method Correction of corner shifts Idea Displace the corners such that CR(displaced) = CRmodel Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 11 / 18
- 26. Proposed Method Correction of corner shifts Idea Displace the corners such that CR(displaced) = CRmodel Method H (+dx;+dy) (+dx;dy) p1 p2 p5 p6 p3 p7 p14 p15 p4 p9 p8 p10p13 p12 p11 p16 p0 1 H(p1) p0 i = H(pi) + (dx;dy) (dx; dy) = argmin dx;dy X (a;b;c;d)2I (CR(p0 a; p0 b; p0 c; p0 d) CRmodel)2 Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 11 / 18
- 27. Proposed Method Correction of corner shifts New Corner Positions H (+dx;+dy) (+dx;dy) p1 p2p5 p6 p3 p7 p14 p15 p8 p10p13 p4 p9 p12 p11p16 p0 1 H(p1) correct(pi) = H1 H(pi) + (dx;dy) | {z } p0 i corner in original image corrected corner Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 12 / 18
- 28. Proposed Method Optimization Process Minimization argmin dx;dy X (a;b;c;d)2I (CR(p0 a; p0 b; p0 c; p0 d) CRmodel)2 Levenberg-Marquardt algorithm not optimal : jacobian not provided, estimated from a forward-dierence approximation 5 ms for each image : SciPy , (Core i7@2.2GHz), 10 lines for CR Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 13 / 18
- 29. Validation Validation Protocol Metric Distance camera $ Marker (before/after correction) vs a reference distance (chessboard) Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 14 / 18
- 30. Validation Validation Protocol Metric Distance camera $ Marker (before/after correction) vs a reference distance (chessboard) Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 14 / 18
- 31. Validation Validation Protocol Metric Distance camera $ Marker (before/after correction) vs a reference distance (chessboard) Pi O check Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 14 / 18
- 32. Validation Validation Protocol Metric Distance camera $ Marker (before/after correction) vs a reference distance (chessboard) Pi Pproj i O check Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 14 / 18
- 33. Validation Validation Experiments Chessboard distance cross ratio Before correction Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 15 / 18
- 34. Validation Validation Experiments cross ratio After correction Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 15 / 18
- 35. Validation Validation Experiments distance After correction Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 15 / 18
- 36. Validation Validation Experiments Logitech c HD Pro Webcam C910 19201080 Microsoft c Lifecam 1280720 Logitech c Procam 640480 Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 15 / 18
- 37. Validation Validation Experiments 3 cameras 5 orientations (range 40-60cm) 8 light conditions 3 groups : illumination condition. over-exposure generate more errors. without correction with correction Under-exposure 1.15 0.22 mm 0.42 0.19 mm Normal exposure 0.90 0.32 mm 0.67 0.22 mm Over-exposure 2.60 0.70 mm 0.71 0.25 mm Table: average errors of the marker depth estimation Cross Ratio : Can detect extreme light condition Can correct depth estimation (1 mm) Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 16 / 18
- 38. Discussion Discussion Perspectives Discussion estimated homography H ! corner correction assumption detected corners are collinear assumption of isotropic inuence of light after homographic rectication Future Works use inner corners of markers minimization : analytical formulation ? Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 17 / 18
- 39. Q A Thank you for your attention vincent.agnus@ircad.fr Vincent Agnus (IRCAD) Illumination Independent Marker Tracking using Cross-Ratio Invariance 18 / 18