[ICCV2021] Learning with Memory-based Virtual Classes for Deep Metric Learning (PPT)
- 1. Learning with Memory-based Virtual Classes for Deep Metric Learning Byungsoo Ko*, Geonmo Gu*, Han-Gyu Kim NAVER/LINE Corp. * Equal contribution memory-based virtual classes for deep metric learning企朱 覈 MemVir朱 覦覯.
- 2. Metric learning problem requires decent generalization performance. This is because of the class difference between training and test. Classification Metric Learning Dog Wolf Cat Fox Lion Tiger Dog Wolf Cat Train class = Test class Train class Test class Motivation Introduction
- 3. Motivation Introduction During training, models from the different training steps are different networks. Training steps 50 100 150 Model weights Embedding space CNN CNN CNN A B C A B C A B C
- 4. Contribution Introduction MemVir utilizes previous steps embeddings and proxies as virtual classes. Training steps 50 100 150 Model weights Embedding space CNN CNN CNN A B C A B C A B C A B A B C A B C C
- 5. Contribution Introduction MemVir trains with augmented information. Adding virtual classes
- 6. Contribution Introduction MemVir increases the learning difficulty gradually. Increasing learning difficulty
- 7. Conventional approach Proposed Method Computes relations between actual embeddings and class weights (proxies) Embeddings 巨 倹 Weights 駒 倹 Mini-batch 巨 CNN Network Actual 巨 倹 駒 倹
- 8. MemVir Proposed Method Computes relations including actual and virtual classes Embeddings + 1 巨 倹 Weights ( + 1)駒 倹 Mini-batch 巨 CNN Network Actual Virtual 倹 倹 巨 倹 駒 倹
- 9. MemVir Proposed Method Computes relations including actual and virtual classes Embeddings + 1 巨 倹 Weights ( + 1)駒 倹 Embeddings Queue 腫 Weights Queue Step -1 -(+1) - -(+1) Mini-batch 巨 CNN Network Actual Virtual 倹 倹 巨 倹 駒 倹 -(+1)
- 10. MemVir Proposed Method Computes relations including actual and virtual classes Embeddings + 1 巨 倹 Weights ( + 1)駒 倹 Embeddings Queue 腫 Weights Queue Step Margin -1 -(+1) - -(+1) Mini-batch 巨 CNN Network Actual Virtual 倹 倹 巨 倹 駒 倹 -(+1)
- 11. MemVir Proposed Method Computes relations including actual and virtual classes Embeddings + 1 巨 倹 Weights ( + 1)駒 倹 Embeddings Queue 腫 Weights Queue Step Margin -1 -(+1) - -(+1) Mini-batch 巨 CNN Network Actual Virtual 倹 倹 巨 倹 駒 倹 # of steps to use (colored) -(+1)
- 12. Discussion: Alleviating a strong focus on the seen classes Proposed Method
- 13. Comparison with SOTA Experiments MemVir improves performance for every metric and benchmark.