際際滷

際際滷Share a Scribd company logo

Deep Learning on the Mobile Devices

?
?
Ryosuke Tanno
Ryosuke Tanno

This document describes research on running deep learning models on mobile devices. The researchers created Caffe2C, which converts Caffe models and parameters into a single C source code. This allows deep learning models trained in Caffe to run efficiently on mobile. Caffe2C is 15x faster than OpenCV DNN. Four mobile apps were created demonstrating Caffe2C: DeepFoodCam for food recognition, DeepStyleCam for neural style transfer, DeepMaterialCam for image translation, and DeepTextInpaintCam for text inpainting.

1 of 17
Download to read offline
2016 UEC Tokyo. Deep Learning on the Mobile Devices Ryosuke Tanno and Keiji Yanai Department of Informatics, The University of Electro-Communications, Tokyo, Japan
? 2016 UEC Tokyo. 1. INTRODUCTION
? 2016 UEC Tokyo. ? There are many attempts to archive CNN on the mobile C Require a high computational power and memory Bring to CNN to Mobile CNN into mobile ! High Computational Power and Memory are Bottleneck!!
? 2016 UEC Tokyo. ? We create a Caffe2C which converts the CNN model definition files and the parameter files trained by Caffe to a single C language code that can run on mobile devices ? Caffe2C makes it easy to use deep learning on the C language operating environment ? Caffe2C achieves faster runtime in comparison to the existing OpenCV DNN module Objective Network Mean Label Model 4 files Caffe2C Single C code
? 2016 UEC Tokyo. ? In order to demonstrate the utilization of the Caffe2C, we have implemented 4 kinds of mobile CNN-based apps on iOS. Objective
? 2016 UEC Tokyo. 2. CONSTRUCTION OF CNN- BASED MOBILE RECOGNITION SYSTEM
? 2016 UEC Tokyo. ? In order to use the learned parameters by Caffe on mobile devices, it is necessary to currently use the OpenCV DNN module not optimized, relatively slow ? We create a Caffe2C which converts the CNN model definition files and the parameter files trained by Caffe to a single C language code C We can use parameter files trained by Caffe on mobile devices Caffe2C
? 2016 UEC Tokyo. ? Caffe2C achieves faster execution speed in comparison to the existing OpenCV DNN module Caffe2C Caffe2C OpenCV DNN AlexNet iPhone 7 Plus 106.9 1663.8 iPad Pro 141.5 1900.1 iPhone SE 141.5 2239.8 Runtime[ms] Caffe2C vs. OpenCV DNN(Input size: 227x227) Speedup Rate: About 15X
? 2016 UEC Tokyo. 1. Caffe2C directly converts the Deep Neural Network to a C source code Reasons for Fast Execution Caffe2C OpenCV DNN Network Mean Label Model Caffe2C Single C code Execution like Compiler Execution like Interpreter
? 2016 UEC Tokyo. 2. Caffe2C performs the pre-processing of the CNN as much as possible to reduce the amount of online computation C Compute batch normalization in advance for conv weight. 3. Caffe2C effectively uses NEON/BLAS by multi-threading Reasons for Fast Execution Network Mean Label Model 4 files Caffe2C Single C code
? 2016 UEC Tokyo. 3. EXAMPLES OF MOBILE APPLICATIONS
? 2016 UEC Tokyo. ? We have implemented 4 kinds of mobile CNN-based apps on iOS C Food recognition app: ＾DeepFoodCam￣ C Neural Style Transfer app: ＾DeepStyleCam￣ C Material Transfer app: ＾DeepMaterialCam￣ C Text Inpaint app: ＾DeepTextInpaintCam￣ 4 iOS Applications
? 2016 UEC Tokyo. DeepFoodCam ? Recognize 101 classes including 100 food classes and one nonfood class Training Phase ? fine-tuned the CNN with 101 class images C totally 20,000 images C UECFOOD-100 and non-food collected from Twitter Target Top-1 Top-5 Food 101 class 74.5% 93.5% Accuracy
? 2016 UEC Tokyo. DeepStyleCam ? Neural Style Transfer C Synthesize an image which has the style of a given style image and the contents of a given content image using Convolutional Neural Network
? 2016 UEC Tokyo. DeepMaterialCam ? Translate an image from a source domain X to a target domain Y in the absence of paired examples.
? 2016 UEC Tokyo. 1. We create a Caffe2C which converts the model definition files and the parameter files of Caffe into a single C code that can run on mobile devices 2. We explain the flow of construction of recognition app using Caffe2C 3. We have implemented 4 kinds of mobile CNN-based apps on iOS. Conclusions
2016 UEC Tokyo. Thank you for listening iOS App is Available ! ＾DeepFoodCam＾ iOS App is Available ! ＾DeepStyleCam￣

More Related Content

Deep Learning on the Mobile Devices

  • 1. 2016 UEC Tokyo. Deep Learning on the Mobile Devices Ryosuke Tanno and Keiji Yanai Department of Informatics, The University of Electro-Communications, Tokyo, Japan
  • 2. ? 2016 UEC Tokyo. 1. INTRODUCTION
  • 3. ? 2016 UEC Tokyo. ? There are many attempts to archive CNN on the mobile C Require a high computational power and memory Bring to CNN to Mobile CNN into mobile ! High Computational Power and Memory are Bottleneck!!
  • 4. ? 2016 UEC Tokyo. ? We create a Caffe2C which converts the CNN model definition files and the parameter files trained by Caffe to a single C language code that can run on mobile devices ? Caffe2C makes it easy to use deep learning on the C language operating environment ? Caffe2C achieves faster runtime in comparison to the existing OpenCV DNN module Objective Network Mean Label Model 4 files Caffe2C Single C code
  • 5. ? 2016 UEC Tokyo. ? In order to demonstrate the utilization of the Caffe2C, we have implemented 4 kinds of mobile CNN-based apps on iOS. Objective
  • 6. ? 2016 UEC Tokyo. 2. CONSTRUCTION OF CNN- BASED MOBILE RECOGNITION SYSTEM
  • 7. ? 2016 UEC Tokyo. ? In order to use the learned parameters by Caffe on mobile devices, it is necessary to currently use the OpenCV DNN module not optimized, relatively slow ? We create a Caffe2C which converts the CNN model definition files and the parameter files trained by Caffe to a single C language code C We can use parameter files trained by Caffe on mobile devices Caffe2C
  • 8. ? 2016 UEC Tokyo. ? Caffe2C achieves faster execution speed in comparison to the existing OpenCV DNN module Caffe2C Caffe2C OpenCV DNN AlexNet iPhone 7 Plus 106.9 1663.8 iPad Pro 141.5 1900.1 iPhone SE 141.5 2239.8 Runtime[ms] Caffe2C vs. OpenCV DNN(Input size: 227x227) Speedup Rate: About 15X
  • 9. ? 2016 UEC Tokyo. 1. Caffe2C directly converts the Deep Neural Network to a C source code Reasons for Fast Execution Caffe2C OpenCV DNN Network Mean Label Model Caffe2C Single C code Execution like Compiler Execution like Interpreter
  • 10. ? 2016 UEC Tokyo. 2. Caffe2C performs the pre-processing of the CNN as much as possible to reduce the amount of online computation C Compute batch normalization in advance for conv weight. 3. Caffe2C effectively uses NEON/BLAS by multi-threading Reasons for Fast Execution Network Mean Label Model 4 files Caffe2C Single C code
  • 11. ? 2016 UEC Tokyo. 3. EXAMPLES OF MOBILE APPLICATIONS
  • 12. ? 2016 UEC Tokyo. ? We have implemented 4 kinds of mobile CNN-based apps on iOS C Food recognition app: ＾DeepFoodCam￣ C Neural Style Transfer app: ＾DeepStyleCam￣ C Material Transfer app: ＾DeepMaterialCam￣ C Text Inpaint app: ＾DeepTextInpaintCam￣ 4 iOS Applications
  • 13. ? 2016 UEC Tokyo. DeepFoodCam ? Recognize 101 classes including 100 food classes and one nonfood class Training Phase ? fine-tuned the CNN with 101 class images C totally 20,000 images C UECFOOD-100 and non-food collected from Twitter Target Top-1 Top-5 Food 101 class 74.5% 93.5% Accuracy
  • 14. ? 2016 UEC Tokyo. DeepStyleCam ? Neural Style Transfer C Synthesize an image which has the style of a given style image and the contents of a given content image using Convolutional Neural Network
  • 15. ? 2016 UEC Tokyo. DeepMaterialCam ? Translate an image from a source domain X to a target domain Y in the absence of paired examples.
  • 16. ? 2016 UEC Tokyo. 1. We create a Caffe2C which converts the model definition files and the parameter files of Caffe into a single C code that can run on mobile devices 2. We explain the flow of construction of recognition app using Caffe2C 3. We have implemented 4 kinds of mobile CNN-based apps on iOS. Conclusions
  • 17. 2016 UEC Tokyo. Thank you for listening iOS App is Available ! ＾DeepFoodCam＾ iOS App is Available ! ＾DeepStyleCam￣