Capstone Design(1) 최종 발표
- 1. Calorie Application 김현우 서유정 박세환 오현주 김선주 교수님 김경민 조교님 based on Faster-RCNN Team HEPC
- 2. Quick Recap
- 3. Korean food image recognition
- 4. SOCIAL DIETthrough objective calorie intake assessment
- 5. 밥 생선구이 무생채 소세지 Our goal 생선구이: 190 kcal 소시지: 110 kcal 무생채: 20 kcal 밥: 120 kcal Total 440 kcal
- 8. Training Outline 300 training images per class 100 test images per class 40 classes • Batch size: 128 • Iteration: 140,000 • Pre trained CNN model ○ VGG16 on Imagenet • Hardware: ○ NVIDIA GTX 1080 • Software: Caffe
- 9. Sample result 30,000 iteration 8 classes
- 10. Test result mAP : mean Average Precision 0 0.2 0.4 0.6 0.8 1 1.2 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 110000 120000 130000 140000 Iteration mAP for each class 쌀밥 가츠동 자장면 냉면 라면 불고기 제육볶음 닭볶음탕 멸치볶음 배추김치 탕수육 만두 떡볶이 순대 콩나물무침 계란말이 돈까스 깍두기 후라이드치킨 갈비찜 갈비탕 김치전 김치찌개 깻잎무침 동그랑땡 된장찌개 미역국 비빔국수 비빔밥 삼계탕 스팸 육개장 잡채 콩나물국 새우튀김 샐러드 우동 채소튀김 장조림 보쌈
- 11. Test result mAP : mean Average Precision 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 110000 120000 130000 140000 OverallmAP Iteration Overall mAP
- 12. Issues
- 13. Test result mAP : mean Average Precision 0 0.2 0.4 0.6 0.8 1 1.2 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 110000 120000 130000 140000 Iteration mAP for each class 쌀밥 가츠동 자장면 냉면 라면 불고기 제육볶음 닭볶음탕 멸치볶음 배추김치 탕수육 만두 떡볶이 순대 콩나물무침 계란말이 돈까스 깍두기 후라이드치킨 갈비찜 갈비탕 김치전 김치찌개 깻잎무침 동그랑땡 된장찌개 미역국 비빔국수 비빔밥 삼계탕 스팸 육개장 잡채 콩나물국 새우튀김 샐러드 우동 채소튀김 장조림 보쌈
- 14. Too low mAP for Cooked Rice 11%
- 15. Why is it Cooked Rice?
- 16. Cropper Model
- 19. What actually happened… Hey… wait a minute.. I think I know that one too..!
- 21. What actually happened… Hey… wait a minute.. I think I know that one too..! ?????? ??????
- 22. What actually happened… 콩나물국쌀밥 닭볶음탕 동그랑땡
- 23. Re-evaluation Test Class_x just with Class_x test set Not with the whole test set
- 24. Test result 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 쌀밥 가츠동 자장면 냉면 라면 불고기 제육볶음 닭볶음탕 멸치볶음 배추김치 탕수육 만두 떡볶이 순대 콩나물무침 계란말이 돈까스 깍두기 후라이드치킨 갈비찜 갈비탕 김치전 김치찌개 깻잎무침 동그랑땡 된장찌개 미역국 비빔국수 비빔밥 삼계탕 스팸 육개장 잡채 콩나물국 새우튀김 샐러드 우동 채소튀김 장조림 보쌈 Estimated mean Average Precision Different test approach : tested menus separately mAP : 0.81 !!
- 26. Foods have hierarchy friedEgg 0.913 bibimbap 0.876 Some foods are on top of others as ingredients
- 27. Foods can be confusing yukgaejang 0.861 kimchiStew 0.783 Similar looking foods are hard to discriminate
- 28. Post-processing layer For practical application use
- 29. Post-processing layer 1.Hierarchy problem - Select the larger box
- 30. Post-processing layer 2. Confusion problem If IoU > 0.7 : Merge 2 boxes Select class with larger p(x) in a greedy way 𝐼𝑜𝑈 = 𝐴 ∩ 𝐵 𝐴 ∪ 𝐵 Intersection over Union
- 31. Result Examples
- 32. Results
- 33. Results
- 34. Results
- 35. Results
- 36. Results
- 38. DEMO
- 39. DEMO
- 40. DEMO
- 41. DEMO
- 42. DEMO
- 43. DEMO
- 44. DEMO
- 45. DEMO
- 46. Discussion 1. Faster-RCNN’s nature: • Learns multiple classes simultaneously • Needs high quality data set that has every class labeled in the image
- 47. Discussion 2. Discovered Faster-RCNN’s downside: • Even though it uses CNN, it does not consider the detected object’s visual context ○ Architectural trade-off • Vanilla CNN is better for catching context • Even naïve bayes would help: the combination of foods
- 48. Thank you