Deep Learning and Texture Mapping
?
1 like?485 views
This document discusses using fully convolutional networks for image segmentation and texture mapping without a 3D model. It introduces deep learning and convolutional neural networks, then describes implementing a fully convolutional network for image segmentation. The network is trained on labeled images to perform pixel-wise segmentation. Corner detection and grid generation are used to create a texture mapping grid. Finally, a texture image is mapped to the gridded segmented image for texture mapping without a 3D model.
![TRAINING RESULT
? FCN initialized using pretrained VGG weights on ImageNet
? 16.000 iterations
87
88
89
90
91
92
93
94
0 5000 10000 15000 20000
ValidationScore[%]
Iteration
Max F1 (Raw)
87,5
88
88,5
89
89,5
90
90,5
0 5000 10000 15000 20000
ValidationScore[%]
Iteration
Avarage Precision (Raw)](https://image.slidesharecdn.com/fcnn-180228213650/85/Deep-Learning-and-Texture-Mapping-20-320.jpg)
Deep Learning and Texture Mapping
- 1. IMAGE SEGMENTATION AND TEXTURE MAPPING USING FULLY CONVOLUTIONAL NETWORKS EFE KAPTAN
- 2. INTRODUCTION ? Texture mapping techniques require a 3D Model ? Requires human interaction ? Hard to adopt on internet applications Sample 3d model Texture mapping on a 3D model
- 3. NEW TECHNIQUE ? Image segmentation with Deep Learning ? Grid generation ? Texture mapping without a 3D Model
- 4. DEEP LEARNING ? Deep learning is a machine learning technique that teaches computers to do what comes naturally to humans: learn by example. ? Utilizes learning algorithms that derive meaning out of data by using a hierarchy of multiple layers that mimic the neural networks of our brain. ? If you provide the system tons of information, it begins to understand it and respond in useful ways.
- 5. HISTORY
- 7. DEEP LEARNING POPULARITY Journal articles mentioning ＾deep learning￣ or ＾deep neural network￣, by nation https://www.nitrd.gov/PUBS/national_ai_rd_strategic_plan.pdf Data for this figure was obtained from a search of the Web of Science Core Collection for "deep learning" or "deep neural net*", for any publication, retrieved 30 August 2016
- 8. LARGE SCALE VISUAL RECOGNITION CHALLENGE Human error rate is %5 ImageNet Large Scale Visual Recognition Challenge (ILSVRC) - http://www.image-net.org/challenges/LSVRC/
- 9. CONVOLUTIONAL NEURAL NETWORK ? Convolution layer is a feature detector that automagically learns to filter out not needed information from an input by using convolution kernel.
- 10. VISUALIZATION OF A FILTER
- 11. WHAT¨S INSIDE HIDDEN LAYERS? Zeiler M. D. And Fergus R., ＾Visualizing and understanding convolutional networks,￣ Nov. 2013
- 12. WHAT¨S INSIDE HIDDEN LAYERS? Zeiler M. D. And Fergus R., ＾Visualizing and understanding convolutional networks,￣ Nov. 2013
- 13. WHAT¨S INSIDE HIDDEN LAYERS? Zeiler M. D. And Fergus R., ＾Visualizing and understanding convolutional networks,￣ Nov. 2013
- 14. FULLY CONVOLUTION NETWORK ? Pixelwise predictions of the same size as the input image Jonathan Long, Evan Shelhamer - 2014
- 15. TEXTURE MAPPING ? We desire to warp the rectangle into the quadrilateral. ? 2D texture mapped to 3D polygon
- 16. BILINEAR INTERPOLATION ? Bilinear warp using a grid
- 17. IMPLEMENTATION INPUT IMAGE SEGMENTATION GRID GENERATION TEXTURE MAPPING ? Main steps in architecture
- 18. TRAINING ? Ground-truth images prepared ? Black and white hand generated images Dataset Number of images Training 200 Testing 25 Validation 25 ? Data distribution
- 19. TRAINING ON GPU ? VGG-16 network ? Trained for 6 hours on TESLA K80 GPU!
- 20. TRAINING RESULT ? FCN initialized using pretrained VGG weights on ImageNet ? 16.000 iterations 87 88 89 90 91 92 93 94 0 5000 10000 15000 20000 ValidationScore[%] Iteration Max F1 (Raw) 87,5 88 88,5 89 89,5 90 90,5 0 5000 10000 15000 20000 ValidationScore[%] Iteration Avarage Precision (Raw)
- 22. CORNER DETECTION ? FAST corner detection algorithm implemented ? A candidate corner ? such that this pixel should be surrounded by n pixels that have a brighter or darker color
- 23. GRID GENERATION ? Grid generation using linear interpolation. ? Vertical and horizontal lines are divided into equal spaces. ? A sample of 10x10 grid
- 24. FINAL GRID ? Final grid with 50 rows and 50 columns
- 25. TEXTURE MAPPING ? Mapping a sample seamless texture image to calculated grid ? Combining final texture with original cropped image
- 28. SPECIAL THANKS TO ASST.PROF.DR TARKAN AYDIN