![Smoothing regularization
• ì¼ë°˜ì ì¸ Regularizationì˜ ì‘ìš©.
• ð‘šð‘–ð‘›ð‘–ð‘šð‘–ð‘§ð‘’ (ð¿ ð´, ð’™, ð‘ , ∥ 𒙠∥)
• ð’™ ê°€ ì‹œê°„ì— ë”°ë¥¸ ë³€ìˆ˜ì¼ ë•Œ
• (ex, ð‘¥d : [0, 1]ì„ në¶„í• í–ˆì„ ë•Œ, i/n ì‹œì ì˜ ì˜¨ë„)
• | ð’™ | ëŒ€ì‹ ì— | ð·ð’™ | 를 넣는다.
• ð·ð‘¥ : ð‘¥ ì˜ ë³€ë™ì„±ì´ë‚˜ 매ë„러운 ì •ë„를 ì¸¡ì •í•˜ëŠ” 함수](https://image.slidesharecdn.com/vectoroptimization-200118074429/85/Vector-Optimization-21-320.jpg)
![[í™ëŒ€ ë¨¸ì‹ ëŸ¬ë‹ ìŠ¤í„°ë”” - 핸즈온 ë¨¸ì‹ ëŸ¬ë‹] 4장. ëª¨ë¸ í›ˆë ¨](https://cdn.slidesharecdn.com/ss_thumbnails/handson-mlch-180814064959-thumbnail.jpg?width=560&fit=bounds)
![[한글] Tutorial: Sparse variational dropout](https://cdn.slidesharecdn.com/ss_thumbnails/tutorialsparsevariationaldropout-190728122300-thumbnail.jpg?width=560&fit=bounds)
![[밑러ë‹] Chap06 í•™ìŠµê´€ë ¨ê¸°ìˆ ë“¤](https://cdn.slidesharecdn.com/ss_thumbnails/chap06-171119110341-thumbnail.jpg?width=560&fit=bounds)
![[GomGuard] 뉴런부터 YOLO 까지 - ë”¥ëŸ¬ë‹ ì „ë°˜ì— ëŒ€í•œ ì´ì•¼ê¸°](https://cdn.slidesharecdn.com/ss_thumbnails/part1mlp2cnn-slidesharev-180308015531-thumbnail.jpg?width=560&fit=bounds)
More Related Content
Similar to Vector Optimization (20)
More from SEMINARGROOT (20)
Vector Optimization
- 1. Vector optimization ì„ ì§„ 환
- 2. ìš©ì–´ ì •ë¦¬ • Convex • Regularization • L1, L2
- 3. 최ì í™” ë¬¸ì œ 표준ì 형태 Q : What is optimal( ì œì¼ ì¢‹ì€ ) vector x* in this primal problem? Step 1. Existence ( Slater’s Condition ) Step 2. Where? ( KKT Condition ) Step 3. How? ( Backtracking Line Search, Gradient Descent, … )
- 5. example
- 6. ì„ í˜¸ 체계가 í•ìƒ 명확한 ê²ƒì€ ì•„ë‹ˆë‹¤. 확실한 ê²ƒì€ ê³ ë“±ì–´ 4마리와 2000ì›ì´ ë‘ ê²½ìš°ë³´ë‹¤ ë” ì¢‹ë‹¤ëŠ” 것!
- 7. 짬뽕과 짜장면 ì¤‘ì— í•˜ë‚˜ë¥¼ ì„ íƒí•´ì•¼í•˜ëŠ” 순간.. Regularization termì„ ì¶”ê°€í•˜ëŠ” ì´ìœ - ëª¨ìˆ˜ì˜ ì ˆëŒ€ê°’ì„ ì¤„ì´ê¸° 위함. - 수리ì í‘œí˜„ì— ëŒ€í•´ ê³ ë¯¼í•´ë³´ìž.
- 8. Standard form of “Vector optimization†Proper coneì€ ë¶€ë¶„ì 으로 대소 관계를 만들어준다.
- 9. ‘(ì œì¼) 좋ì€â€™ ì˜ ì •ì˜, Optimal and Pareto optimal ìµœê³ ì™€ ìµœì„ ì˜ ì°¨ì´
- 11. Remark. • Vector optimization ì—서는 optimal(최ì ) point 를 찾기보다는 Pareto(ìµœì„ ) optimal ì„ ì°¾ëŠ” ê²ƒì´ ì ì ˆí•˜ë‹¤. • Pareto optimal value lies in the boundary of the set of achievable object values. • Optimal point is Pareto optimal. • Pareto optimal point 를 ëª¨ë‘ ì°¾ìž. • ê·¸ ì¤‘ì— Optimal pointê°€ 있다면 optimal value를 쉽게 ì•Œ 수 있다.
- 12. Pareto optimal ì„ ì°¾ëŠ” 방법 – Scalarization ì´ ë°©ë²•ìœ¼ë¡œ ëª¨ë“ pareto optimal ì„ ì°¾ì„ ìˆ˜ 있는 ê²ƒì€ ì•„ë‹ˆë‹¤.
- 13. • Lemma ð¿ð‘’𑡠𑆠ð‘ð‘’ ð‘Ž ð‘ð‘œð‘›ð‘£ð‘’ð‘¥ ð‘ ð‘’ð‘¡. ð¹ð‘œð‘Ÿ ð‘Žð‘›ð‘¦ ð‘šð‘–ð‘›ð‘–ð‘šð‘Žð‘™ ð‘’ð‘™ð‘’ð‘šð‘’ð‘›ð‘¡ ð‘‹ ð‘œð‘“ ð‘†, ð‘‡â„Žð‘’ð‘Ÿð‘’ ð‘’ð‘¥ð‘–ð‘ ð‘¡ð‘ ð‘Ž ð‘›ð‘œð‘›ð‘§ð‘’ð‘Ÿð‘œ 𜆠≽<∗ 0 ð‘ ð‘¢ð‘â„Ž ð‘¡â„Žð‘Žð‘¡ ð‘‹ ð‘šð‘–ð‘›ð‘–ð‘šð‘–ð‘§ð‘’ð‘ ðœ†@ 𑧠ð‘œð‘£ð‘’𑟠𑧠∈ ð‘†. pf ) easy • Proposition ð¹ð‘œð‘Ÿ ð¶ð‘œð‘›ð‘£ð‘’ð‘¥ ð‘£ð‘’ð‘ð‘¡ð‘œð‘Ÿ ð‘œð‘ð‘¡ð‘–ð‘šð‘–ð‘§ð‘Žð‘¡ð‘–ð‘œð‘› ð‘ð‘Ÿð‘œð‘ð‘™ð‘’ð‘š, for every Pareto optimal point ð‘¥RS , ð‘¡â„Žð‘’ð‘Ÿð‘’ ð‘–ð‘ ð‘Ž ð‘›ð‘œð‘›ð‘§ð‘’ð‘Ÿð‘œ 𜆠≽<∗ 0 ð‘ ð‘¢ð‘â„Ž ð‘¡â„Žð‘Žð‘¡ ð‘¥RS ð‘šð‘–ð‘›ð‘–ð‘šð‘–ð‘§ð‘’ð‘ ð‘¡â„Žð‘’ ð‘ ð‘ð‘Žð‘™ð‘Žð‘Ÿð‘–ð‘§ð‘’ð‘‘ ð‘ð‘Ÿð‘œð‘ð‘™ð‘’ð‘š. • Remark! • Optimal of the scalarized problem ⟶ Pareto optimal ⋯ 𜆠≻<∗ 0 • Pareto optimal ⟶ Optimal of the scalarized problem ⋯ 𜆠≽<∗ 0 초과ì¸ì§€ ì´ìƒì¸ì§€ëŠ” 실무ì 으로 별 ë¬¸ì œ 아니다 !!
- 14. 지금까지 í•œ ê²ƒë“¤ì„ ì •ë¦¬ë¥¼ 하면 1. ì„ í˜¸ 체계 불확실 í• ìˆ˜ 있다. 2. Vector optimization 3. Optimal? No! Pareto optimal ! 4. 어떻게? Scalarization ! 5. 왜? Propositionì— ì˜í•´ !
- 15. Application • Regularized least squares • Smoothing regularization • Reconstruction, smoothing and de-noising
- 16. Application • Regularized least squares • Smoothing regularization • Reconstruction, smoothing and de-noising
- 17. Regularized least squares A : data set, b : label ê°€ ì£¼ì–´ì§€ê³ ì„ í˜•íšŒê·€ë¥¼ í–ˆì„ë•Œ, ì 당한 ì 당한 parameter 𑥠를 찾는 ë¬¸ì œ ð´ð‘–𑚠∶ ð‘“ð‘–ð‘›ð‘‘ ð‘¥ ð‘¡â„Žð‘Žð‘¡ ð‘”ð‘–ð‘£ð‘’ð‘ ð‘Ž ð‘”ð‘œð‘œð‘‘ ð‘“ð‘–ð‘¡ ð‘Žð‘›ð‘‘ ð‘¡â„Žð‘Žð‘¡ ð‘–ð‘ ð‘›ð‘œð‘¡ ð‘¡ð‘œð‘œ ð‘™ð‘Žð‘Ÿð‘”ð‘’ ì´ ë¬¸ì œì˜ Pareto optimalì„ ì°¾ëŠ” 법?? Scalarization !!
- 18. Recall : Pareto optimal ì„ ì°¾ëŠ” 방법 – Scalarization ì´ ë°©ë²•ìœ¼ë¡œ ëª¨ë“ pareto optimal ì„ ì°¾ì„ ìˆ˜ 있는 ê²ƒì€ ì•„ë‹ˆë‹¤.
- 19. Regularized least squares 극소ì ∶ 미분해서 0ì´ ë˜ëŠ” ì ðœ†[ 와 ðœ†ì˜ ë¹„ìœ¨ì„ ë‹¤ë¥´ê²Œí•´ì„œ 다른 결과를 만들 수 있다. - Fitting ì„ ìš°ì„ ì‹œí• ì§€ - Regularization ì„ ìš°ì„ ì‹œ í• ì§€ 𜇠≫ 1
- 20. Application • Regularized least squares • Smoothing regularization • Reconstruction, smoothing and de-noising
- 21. Smoothing regularization • ì¼ë°˜ì ì¸ Regularizationì˜ ì‘ìš©. • ð‘šð‘–ð‘›ð‘–ð‘šð‘–ð‘§ð‘’ (ð¿ ð´, ð’™, ð‘ , ∥ 𒙠∥) • ð’™ ê°€ ì‹œê°„ì— ë”°ë¥¸ ë³€ìˆ˜ì¼ ë•Œ • (ex, ð‘¥d : [0, 1]ì„ në¶„í• í–ˆì„ ë•Œ, i/n ì‹œì ì˜ ì˜¨ë„) • | ð’™ | ëŒ€ì‹ ì— | ð·ð’™ | 를 넣는다. • ð·ð‘¥ : ð‘¥ ì˜ ë³€ë™ì„±ì´ë‚˜ 매ë„러운 ì •ë„를 ì¸¡ì •í•˜ëŠ” 함수
- 22. Smoothing regularization ∆𑥠d = i번째 ì‹œì ì˜ ê³¡ë¥ ì˜ ê·¼ì‚¬ê°’
- 23. 𛿠= 0 𜇠= 0.005 𛿠= 0 𜇠= 0.05 𛿠= 0.3 𜇠= 0.05 𛿠가 ì»¤ì§ˆìˆ˜ë¡ smoothing í•´ì§€ê³ ðœ‡ ê°€ ì»¤ì§ˆìˆ˜ë¡ inputì˜ í¬ê¸°ê°€ 작아진다.
- 24. Application • Regularized least squares • Smoothing regularization • Reconstruction, smoothing and de-noising
- 25. Reconstruction, smoothing and de-noising • 𑥠∶ ð‘ ð‘–ð‘”ð‘›ð‘Žð‘™ ð‘Ÿð‘’ð‘ð‘Ÿð‘’ð‘ ð‘’ð‘›ð‘¡ð‘’ð‘‘ ð‘𑦠𑎠ð‘£ð‘’ð‘ð‘¡ð‘œð‘Ÿ ð‘–ð‘› â„m • ð’™ = ð‘¥[, ð‘¥, … , ð‘¥m , ð‘¥d = ð‘¥d(ð‘¡) • Assume that ð’™ usually doesn’t vary rapidly. • e.g. audio signal or video. • ð‘¥oSp = ð‘¥ + ð‘£, 𑣠∶ ð‘›ð‘œð‘–ð‘ 𑒠• ì›ëž˜ ì‹ í˜¸(ð‘¥)ì— ë…¸ì´ì¦ˆ(ð‘£)ê°€ ì„žì¸ ì‹ í˜¸(ð‘¥oSp) 를 ìƒ˜í”Œë§ í–ˆì„ ë•Œ, ì›ëž˜ ì‹ í˜¸(ð‘¥) ê°€ 무엇ì¸ì§€ ì¶”ì •í•˜ê³ ì‹¶ë‹¤. • ð‘¤ð‘’ ð‘¤ð‘Žð‘›ð‘¡ ð‘¡ð‘œ ð‘’ð‘ ð‘¡ð‘–ð‘šð‘Žð‘¡ð‘’ sð‘¥ ð‘œð‘“ ð‘¡â„Žð‘’ ð‘œð‘Ÿð‘–ð‘”ð‘–ð‘›ð‘Žð‘™ ð‘ ð‘–ð‘”ð‘›ð‘Žð‘™ ð‘¥, ð‘”ð‘–ð‘£ð‘’ð‘› ð‘¡â„Žð‘’ ð‘ð‘œð‘Ÿð‘Ÿð‘¢ð‘ð‘¡ð‘’ð‘‘ ð‘ ð‘–ð‘”ð‘›ð‘Žð‘™ ð‘¥oSp.
- 26. Reconstruction, smoothing and de-noising 𜙠∶ ð‘Ÿð‘’ð‘”ð‘¢ð‘™ð‘Žð‘Ÿð‘–ð‘§ð‘Žð‘¡ð‘–ð‘œð‘› ð‘“ð‘¢ð‘›ð‘ð‘¡ð‘–ð‘œð‘› ð‘œð‘Ÿ ð‘ ð‘šð‘œð‘œð‘¡â„Žð‘–ð‘›ð‘” ð‘œð‘ð‘—ð‘’ð‘ð‘¡ð‘–ð‘£ð‘’ 𜙠ð‘šð‘’ð‘Žð‘ ð‘¢ð‘Ÿð‘’ð‘ ð‘Ÿð‘œð‘¢ð‘”â„Žð‘›ð‘’ð‘ ð‘ ð‘œð‘Ÿ ð‘™ð‘Žð‘𑘠ð‘œð‘“ ð‘ ð‘šð‘œð‘œð‘¡â„Žð‘›ð‘’ð‘ ð‘ ð‘œð‘“ sð‘¥ + convex
- 27. Reconstruction, smoothing and de-noising
- 28. Reconstruction, smoothing and de-noising
- 29. Reconstruction, smoothing and de-noising Rapid variationì´ ìžˆëŠ” 경우
- 30. ë.