際際滷

際際滷Share a Scribd company logo

Bayesian Depth-from-Defocus with Shading Constraints

Aaron Karper
Aaron Karper

The document presents a method for estimating depth from a single image using both depth from defocus and depth from shading constraints. It estimates depth by maximizing the probability of depth given image intensities using both models. Depth from defocus models blur between images taken with different focal lengths, while depth from shading uses image gradients and estimated illumination. The paper shows combining the models yields better depth estimates than using smoothness priors, especially in untextured regions.

1 of 18
Bayesian Depth-from-Defocus with Shading Constraints Chen Lin Shuochen Su Aaron Karper paper by Yasuyuki Matsushita Kun Zhou Stephen Lin Dec 17th 2013 paper: 2013 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 1 / 16
1 Overview of Depth from Defocus MAP estimate 2 Overview of Depth from Shading 3 Optimizing both Models Depth-from-defocus 4 Results 5 Discussion Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 2 / 16
Overview of Depth from Defocus Overview of Depth from Defocus 1 0 = F 1 d 1 vd Rv 1 1 b= F d 1 vd 2 In focus spread out of focus Model Point-spread as Gaussian xT x blur h(p| 2 ) e 22 = 粒b calibrate 粒 2 2 I2 (p) = I1 (p) h(p| 2 1 ) basically d Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 3 / 16
Overview of Depth from Defocus MAP estimate Overview of Depth from Defocus MAP estimate d = arg max P(d|I1 , I2 ) d = arg max P(I1 , I2 |d) P(d) d N around deconv N (c, 1 ) 2了 = arg min L(I1 , I2 |d) + L(d) d (I1 (p) h(p|d) I2 (p))2 = ppixels (di dj )2 了 i,j竜 Standard Depth from Defocus Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 4 / 16
Overview of Depth from Defocus MAP estimate Overview of Depth from Defocus MAP estimate d = arg max P(d|I1 , I2 ) d = arg max P(I1 , I2 |d) P(d) d N around deconv N (c, 1 ) 2了 = arg min L(I1 , I2 |d) + L(d) d (I1 (p) h(p|d) I2 (p))2 = ppixels (di dj )2 了 i,j竜 Standard Depth from Defocus Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 4 / 16
Overview of Depth from Defocus MAP estimate Overview of Depth from Defocus MAP estimate d = arg max P(d|I1 , I2 ) d = arg max P(I1 , I2 |d) P(d) d N around deconv N (c, 1 ) 2了 = arg min L(I1 , I2 |d) + L(d) d (I1 (p) h(p|d) I2 (p))2 = ppixels (di dj )2 了 i,j竜 Standard Depth from Defocus Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 4 / 16
Overview of Depth from Defocus Overview of Depth from Defocus Advantages: Passive perception Single camera Disadvantages: Lense aperture necessary Needs texture Advances in lense technology Precise Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 5 / 16
Overview of Depth from Shading Overview of Depth from Shading s(n) = Vn, n M n: normals M: illumination Measure M with Lambertian sphere. Uniform albedo: (pj pi ), L(d) = 了 i,j竜 ni + nj ni + nj If albedo present, remove with depth estimate. Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 6 / 16
Overview of Depth from Shading Overview of Depth from Shading Advantages: Disadvantages: Passive perception Texture hinders perception Single camera Calibration necessary Very precise if applicable Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 7 / 16
Optimizing both Models Depth-from-defocus Optimizing both Models Depth-from-defocus Cyclic dependencies in L(I1 , I|d) Resolved by magic1 1 Markov random 鍖elds and loopy belief propagation. Explanation if requested: 15 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 8 / 16
Optimizing both Models Optimizing both Models Depth estimate depends on depth-from-defocus, depth-from-shading. Depth-from-shading pro鍖ts from depth estimate for albedo removal. therefore estimation-maximization Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 9 / 16
Results Results Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 10 / 16
Results Results Better results than smoothness prior. Especially on untextured regions. As good results even on textured regions. Calibration necessary. Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 11 / 16
Discussion Discussion Calibration hinders application. Not clear why magic2 wasnt extended to integrate depth from shading, depth from defocus. Not clear why sensor fusion wasnt done. 2 Markov random 鍖elds and loopy belief propagation. Explanation if requested: 15 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 12 / 16
Discussion Questions? Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 13 / 16
Discussion Sensor Fusion depth d S1 = N (d, 1 (I )) Estimate global for both models. Use to estimate d for each pixel. I S2 = N (d, 2 (I )) intensity Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 14 / 16
Discussion Markov random 鍖elds Each node represents a value (a proposition). Each node has a belief3 . Each node can depend on other nodes. Connections if dependency. Basically undirected Bayes Net. Solve with Belief Propagation. 3 Like a probability, but can take values in R0 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 15 / 16
Discussion Belief Propagation Nodes send own belief to nodes that depend on them. Update belief on message. Pray for convergence. Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 16 / 16

More Related Content

Bayesian Depth-from-Defocus with Shading Constraints

  • 1. Bayesian Depth-from-Defocus with Shading Constraints Chen Lin Shuochen Su Aaron Karper paper by Yasuyuki Matsushita Kun Zhou Stephen Lin Dec 17th 2013 paper: 2013 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 1 / 16
  • 2. 1 Overview of Depth from Defocus MAP estimate 2 Overview of Depth from Shading 3 Optimizing both Models Depth-from-defocus 4 Results 5 Discussion Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 2 / 16
  • 3. Overview of Depth from Defocus Overview of Depth from Defocus 1 0 = F 1 d 1 vd Rv 1 1 b= F d 1 vd 2 In focus spread out of focus Model Point-spread as Gaussian xT x blur h(p| 2 ) e 22 = 粒b calibrate 粒 2 2 I2 (p) = I1 (p) h(p| 2 1 ) basically d Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 3 / 16
  • 4. Overview of Depth from Defocus MAP estimate Overview of Depth from Defocus MAP estimate d = arg max P(d|I1 , I2 ) d = arg max P(I1 , I2 |d) P(d) d N around deconv N (c, 1 ) 2了 = arg min L(I1 , I2 |d) + L(d) d (I1 (p) h(p|d) I2 (p))2 = ppixels (di dj )2 了 i,j竜 Standard Depth from Defocus Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 4 / 16
  • 5. Overview of Depth from Defocus MAP estimate Overview of Depth from Defocus MAP estimate d = arg max P(d|I1 , I2 ) d = arg max P(I1 , I2 |d) P(d) d N around deconv N (c, 1 ) 2了 = arg min L(I1 , I2 |d) + L(d) d (I1 (p) h(p|d) I2 (p))2 = ppixels (di dj )2 了 i,j竜 Standard Depth from Defocus Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 4 / 16
  • 6. Overview of Depth from Defocus MAP estimate Overview of Depth from Defocus MAP estimate d = arg max P(d|I1 , I2 ) d = arg max P(I1 , I2 |d) P(d) d N around deconv N (c, 1 ) 2了 = arg min L(I1 , I2 |d) + L(d) d (I1 (p) h(p|d) I2 (p))2 = ppixels (di dj )2 了 i,j竜 Standard Depth from Defocus Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 4 / 16
  • 7. Overview of Depth from Defocus Overview of Depth from Defocus Advantages: Passive perception Single camera Disadvantages: Lense aperture necessary Needs texture Advances in lense technology Precise Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 5 / 16
  • 8. Overview of Depth from Shading Overview of Depth from Shading s(n) = Vn, n M n: normals M: illumination Measure M with Lambertian sphere. Uniform albedo: (pj pi ), L(d) = 了 i,j竜 ni + nj ni + nj If albedo present, remove with depth estimate. Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 6 / 16
  • 9. Overview of Depth from Shading Overview of Depth from Shading Advantages: Disadvantages: Passive perception Texture hinders perception Single camera Calibration necessary Very precise if applicable Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 7 / 16
  • 10. Optimizing both Models Depth-from-defocus Optimizing both Models Depth-from-defocus Cyclic dependencies in L(I1 , I|d) Resolved by magic1 1 Markov random 鍖elds and loopy belief propagation. Explanation if requested: 15 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 8 / 16
  • 11. Optimizing both Models Optimizing both Models Depth estimate depends on depth-from-defocus, depth-from-shading. Depth-from-shading pro鍖ts from depth estimate for albedo removal. therefore estimation-maximization Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 9 / 16
  • 12. Results Results Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 10 / 16
  • 13. Results Results Better results than smoothness prior. Especially on untextured regions. As good results even on textured regions. Calibration necessary. Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 11 / 16
  • 14. Discussion Discussion Calibration hinders application. Not clear why magic2 wasnt extended to integrate depth from shading, depth from defocus. Not clear why sensor fusion wasnt done. 2 Markov random 鍖elds and loopy belief propagation. Explanation if requested: 15 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 12 / 16
  • 15. Discussion Questions? Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 13 / 16
  • 16. Discussion Sensor Fusion depth d S1 = N (d, 1 (I )) Estimate global for both models. Use to estimate d for each pixel. I S2 = N (d, 2 (I )) intensity Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 14 / 16
  • 17. Discussion Markov random 鍖elds Each node represents a value (a proposition). Each node has a belief3 . Each node can depend on other nodes. Connections if dependency. Basically undirected Bayes Net. Solve with Belief Propagation. 3 Like a probability, but can take values in R0 Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 15 / 16
  • 18. Discussion Belief Propagation Nodes send own belief to nodes that depend on them. Update belief on message. Pray for convergence. Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Shading Constraints Bayesian Depth-from-Defocus with Lin Dec 17th 2013paper: 2013 16 / 16