際際滷

際際滷Share a Scribd company logo

2018/03/28 Sony's deep learning software "Neural Network Libraries/Console＾ and its use cases in Sony

?Download as PPTX, PDF?
?
Sony Network Communications Inc.
Sony Network Communications Inc.

This document discusses Sony's deep learning software, including the Neural Network Libraries and Neural Network Console. The Neural Network Libraries are an open source deep learning framework with Python API, while the Neural Network Console is a GUI-based deep learning IDE. These tools are being used by Sony in various products and services for applications like real estate price estimation, gesture recognition in Xperia Ear, and image recognition in aibo. The document provides information on the features and capabilities of the software.

1 of 21
Downloaded 33 times
Sony's deep learning software "Neural Network Libraries/Console＾ and its use cases in Sony Yoshiyuki Kobayashi C Senior Machine Learning Researcher Sony Network Communications Inc. / Sony Corporation GTC 2018
2 Neural Network Libraries Neural Network Console nnabla.org dl.sony.com Windows version is now available for free.Open source (Apache 2.0 license) Deep learning framework with Python API GUI based deep learning IDE Sony's deep learning software
3 Sony real estate AR Effect Xperia Ear aiboXperia Hello ? Used since 2011 and more than 1,000 users in Sony group. ? Just released in 2017 ? Already utilized in many products and services. 2011゛ 1st gen framework 2013゛ 2nd gen framework 2016゛ 3rd gen framework Neural Network Libraries 2015゛ GUI Tool Neural Network Console Open Sourced at Jul. 2017 Released Windows Version at Aug. 2017 for Free
4 Motivation Deep learning engineer is lacking worldwide Deep Learning is great and demand is explosively growing ? Need to make deep learning research & development more efficient ? Also, human resource development needs to be accelerated SOLD
5 Introduction of Neural Network Console https://www.youtube.com/watch?v=-lXjnaUSEtM
6 Application Example #1Sony Real Estate Real Estate Price Estimate Neural Network Libraries is used in Real Estate Price Estimate Engine of Sony Real Estate Corporation. the Library realizes the solution that statistically estimates signed price in buying and selling real estate, analyzing massive data with unique algorism developed based on evaluation know-how and knowledge of Sony Real Estate Corporation. The solution is utilized in various businesses of Sony Real Estate Corporation such as ￣Ouchi Direct￣, ＾Real Estate Property Search Map￣ and ＾Automatic Evaluation￣. ´ Input Bunch of features about a real estate Total floor area Floor plan Street address ´ Output Price of a real estate
7 Application Example #2Xperia Ear Gesture Sensitivity The Library is used in an intuitive gesture sensitivity function of Sony Mobile Communications ＾Xperia Ear￣. Based on data from several sensors embedded in Xperia Ear, you can just use a nod of the head to confirm a command - answering Yes/No, answering/declining the phone call, cancelling text-to-speech reading of notifications, skipping/rewinding a song track. ´ Input Sensor data Output Head gestures such as Yes and No
8 Application Example #3aibo Image Recognition (User Identification, Face Tracking, etc.) The Library is used to realize image recognition of Sony¨s Entertainment Robot "aibo" 〆ERS-1000〇. In the image recognition through fish-eye cameras installed at the nose, the Library is actively utilized for user identification, face tracking, charge stand recognition, generic object recognition, etc. These features and various inbuilt sensors enable its adaptable behavior. ´ 秘薦 Image Output Face, object, Charge stand, etc´
9 Application of DL spreading depending on input / output ´Input Output Function Input Output Image recognition Image Category of image Image filtering Source image target image Speech recognition Speech String Machine translation (En->Ja) English word string Japanese word string Chat-bot Input word string String of response Sensor error detection Sensor signal Abnormality level Robot control Robot's sensor Robot actuator ´ The possibilities of application are infinite Function
10 Feature 1. Ideal for deep learning beginners ? Easy to setup ? Just unzip the downloaded file and run the application ? Select ＾GPU￣ in Setup window to use NVIDIA GPU ? You can learn deep learning visually ? Trial and error in a short time Provides shortcut for skill improvement on deep learning Logistic regression Multi layer perceptron Convolutional Neural Networks
11 Feature 2. Realize very efficient development ? Easy debugging ? Instantly locate the error on GUI ? Easy to deploy ? Created model can be deployed immediately by using Neural Network Libraries ? NNL offers both Python and C++ API ? Automatic structure search ? search for better neural network structure automatically
12 More features ? Functions ? Management of trial and error history ? Visualization of weights ? Export python code ? Supports ´ ? Various data as well as images, matrices and vectors ? Various objectives such as classification, detection, signal processing, regression, etc. ? Multiple inputs and outputs for multi-modal application ? Training of huge neural networks like ResNet-152 ? Recurrent neural networks (RNN) ? Generative Adversarial Networks (GAN) ? Semi-supervised learning ? Transfer learning etc´
13 Neural Network Libraries ? Lightweight C++ core library with Python API ? Flexible for arbitrary NNs ? Cross-platform & Cross-device ? Easy to add a new NN operator ? Portable to C++ ? Sophisticated ? Easy-to-write & easy-to-read ? Fast training and inference with CUDA/cuDNN ? Supports both static and dynamic NNs ? Supports distributed training ? Supports fp16 and Tensor Cores ? Resources ? NNabla-Examles https://github.com/sony/nnabla-examples/ ResNet, GANs, CapsNet, Quantized Nets, ´ ? Jupyter Notebook Tutorials https://github.com/sony/nnabla/tree/master/tutorial https://github/sony/nnabla/ pip install nnabla import nnabla as nn import nnabla.functions as F import nnabla.parametric_functions as PF x = nnabla.Variable((batch_size, 1, 28, 28)) c1 = PF.convolution(x, 16, (5, 5), name='c1') c1 = F.relu(F.max_pooling(c1, (2, 2))) c2 = PF.convolution(c1, 16, (5, 5), name='c2') c2 = F.relu(F.max_pooling(c2, (2, 2))) f3 = F.relu(PF.affine(c2, 50, name='f3')) y = PF.affine(f3, 50, name='f4') t = Variable((batch_size, 1)) loss = F.mean(F.softmax_cross_entropy(y, t)) ´ Suitable for both research and production use
14 Neural Network Libraries Neural Network Console Sony is providing an integrated development environment with hopes that a wider range of developers and researchers will build on its programs, and with the aim of contributing to the development of society. nnabla.org dl.sony.com Windows version is now available for free.Open source (Apache 2.0 license) Deep learning framework with Python API GUI based deep learning IDE Sony's deep learning software
2018/03/28 Sony's deep learning software "Neural Network Libraries/Console＾ and its use cases in Sony
16 Appendix
17 Tutorial Image classification (hand-writing digits recognition) ´ ´ ´ ´ ´ ´ ´ Classifier Neural Network Input : image Output : Classification result Take a typical application ＾Hand-writing digits recognition￣ as an example 2 Number of input neuron is size of input data 28x28 pixel 784 Number of output neural is number of categories of target data 10
18 Work required to create a image classifier ´ ´ ´ ´ ´ ´ ´ Classifier Neural Network Input : Image Output : Classification result 2 1. Prepare dataset Prepare as much data as possible Each data includes image and its class index 2. Design neural network structure You can use sample project 3. Train designed neural network using prepared dataset Basically you can create image classifier with these three steps ´ 仝0々 ´ 仝1々 ´ 仝2々 ´ 仝3々 ´ 仝4々 ´ 仝5々 ´ 仝6々 ´ 仝7々 ´ 仝8々 ´ 仝9々
19 1. Prepare dataset ? Create dataset CSV files according to a predetermined format x:image y:label ./training/5/0.png 5 ./training/0/1.png 0 ./training/4/2.png 4 ./training/1/3.png 1 ./training/9/4.png 9 ./training/2/5.png 2 ./training/1/6.png 1 ´ File name of hand-writing digit class Index It can be created with Excel or simple script
20 2. Design neural network structure ? Convolutional Neural Networks ? Put layers as it is by drag & drop, and edit properties of layers Add layers by drag & drop Edit property of layer You can design the network structure very easily and intuitively 5 x 5 Convolution Activation 2 x 2 Pooling 3 x 3 Convolution Activation 2 x 2 Pooling Affine Activation Fully connected Softmax Categorical Loss
21 3. Train designed neural network using prepared dataset ? Training starts just by pushing the training button When the training is complete, an image classifier is obtained loss training iteration

More Related Content

2018/03/28 Sony's deep learning software "Neural Network Libraries/Console＾ and its use cases in Sony

  • 1. Sony's deep learning software "Neural Network Libraries/Console＾ and its use cases in Sony Yoshiyuki Kobayashi C Senior Machine Learning Researcher Sony Network Communications Inc. / Sony Corporation GTC 2018
  • 2. 2 Neural Network Libraries Neural Network Console nnabla.org dl.sony.com Windows version is now available for free.Open source (Apache 2.0 license) Deep learning framework with Python API GUI based deep learning IDE Sony's deep learning software
  • 3. 3 Sony real estate AR Effect Xperia Ear aiboXperia Hello ? Used since 2011 and more than 1,000 users in Sony group. ? Just released in 2017 ? Already utilized in many products and services. 2011゛ 1st gen framework 2013゛ 2nd gen framework 2016゛ 3rd gen framework Neural Network Libraries 2015゛ GUI Tool Neural Network Console Open Sourced at Jul. 2017 Released Windows Version at Aug. 2017 for Free
  • 4. 4 Motivation Deep learning engineer is lacking worldwide Deep Learning is great and demand is explosively growing ? Need to make deep learning research & development more efficient ? Also, human resource development needs to be accelerated SOLD
  • 5. 5 Introduction of Neural Network Console https://www.youtube.com/watch?v=-lXjnaUSEtM
  • 6. 6 Application Example #1Sony Real Estate Real Estate Price Estimate Neural Network Libraries is used in Real Estate Price Estimate Engine of Sony Real Estate Corporation. the Library realizes the solution that statistically estimates signed price in buying and selling real estate, analyzing massive data with unique algorism developed based on evaluation know-how and knowledge of Sony Real Estate Corporation. The solution is utilized in various businesses of Sony Real Estate Corporation such as ￣Ouchi Direct￣, ＾Real Estate Property Search Map￣ and ＾Automatic Evaluation￣. ´ Input Bunch of features about a real estate Total floor area Floor plan Street address ´ Output Price of a real estate
  • 7. 7 Application Example #2Xperia Ear Gesture Sensitivity The Library is used in an intuitive gesture sensitivity function of Sony Mobile Communications ＾Xperia Ear￣. Based on data from several sensors embedded in Xperia Ear, you can just use a nod of the head to confirm a command - answering Yes/No, answering/declining the phone call, cancelling text-to-speech reading of notifications, skipping/rewinding a song track. ´ Input Sensor data Output Head gestures such as Yes and No
  • 8. 8 Application Example #3aibo Image Recognition (User Identification, Face Tracking, etc.) The Library is used to realize image recognition of Sony¨s Entertainment Robot "aibo" 〆ERS-1000〇. In the image recognition through fish-eye cameras installed at the nose, the Library is actively utilized for user identification, face tracking, charge stand recognition, generic object recognition, etc. These features and various inbuilt sensors enable its adaptable behavior. ´ 秘薦 Image Output Face, object, Charge stand, etc´
  • 9. 9 Application of DL spreading depending on input / output ´Input Output Function Input Output Image recognition Image Category of image Image filtering Source image target image Speech recognition Speech String Machine translation (En->Ja) English word string Japanese word string Chat-bot Input word string String of response Sensor error detection Sensor signal Abnormality level Robot control Robot's sensor Robot actuator ´ The possibilities of application are infinite Function
  • 10. 10 Feature 1. Ideal for deep learning beginners ? Easy to setup ? Just unzip the downloaded file and run the application ? Select ＾GPU￣ in Setup window to use NVIDIA GPU ? You can learn deep learning visually ? Trial and error in a short time Provides shortcut for skill improvement on deep learning Logistic regression Multi layer perceptron Convolutional Neural Networks
  • 11. 11 Feature 2. Realize very efficient development ? Easy debugging ? Instantly locate the error on GUI ? Easy to deploy ? Created model can be deployed immediately by using Neural Network Libraries ? NNL offers both Python and C++ API ? Automatic structure search ? search for better neural network structure automatically
  • 12. 12 More features ? Functions ? Management of trial and error history ? Visualization of weights ? Export python code ? Supports ´ ? Various data as well as images, matrices and vectors ? Various objectives such as classification, detection, signal processing, regression, etc. ? Multiple inputs and outputs for multi-modal application ? Training of huge neural networks like ResNet-152 ? Recurrent neural networks (RNN) ? Generative Adversarial Networks (GAN) ? Semi-supervised learning ? Transfer learning etc´
  • 13. 13 Neural Network Libraries ? Lightweight C++ core library with Python API ? Flexible for arbitrary NNs ? Cross-platform & Cross-device ? Easy to add a new NN operator ? Portable to C++ ? Sophisticated ? Easy-to-write & easy-to-read ? Fast training and inference with CUDA/cuDNN ? Supports both static and dynamic NNs ? Supports distributed training ? Supports fp16 and Tensor Cores ? Resources ? NNabla-Examles https://github.com/sony/nnabla-examples/ ResNet, GANs, CapsNet, Quantized Nets, ´ ? Jupyter Notebook Tutorials https://github.com/sony/nnabla/tree/master/tutorial https://github/sony/nnabla/ pip install nnabla import nnabla as nn import nnabla.functions as F import nnabla.parametric_functions as PF x = nnabla.Variable((batch_size, 1, 28, 28)) c1 = PF.convolution(x, 16, (5, 5), name='c1') c1 = F.relu(F.max_pooling(c1, (2, 2))) c2 = PF.convolution(c1, 16, (5, 5), name='c2') c2 = F.relu(F.max_pooling(c2, (2, 2))) f3 = F.relu(PF.affine(c2, 50, name='f3')) y = PF.affine(f3, 50, name='f4') t = Variable((batch_size, 1)) loss = F.mean(F.softmax_cross_entropy(y, t)) ´ Suitable for both research and production use
  • 14. 14 Neural Network Libraries Neural Network Console Sony is providing an integrated development environment with hopes that a wider range of developers and researchers will build on its programs, and with the aim of contributing to the development of society. nnabla.org dl.sony.com Windows version is now available for free.Open source (Apache 2.0 license) Deep learning framework with Python API GUI based deep learning IDE Sony's deep learning software
  • 17. 17 Tutorial Image classification (hand-writing digits recognition) ´ ´ ´ ´ ´ ´ ´ Classifier Neural Network Input : image Output : Classification result Take a typical application ＾Hand-writing digits recognition￣ as an example 2 Number of input neuron is size of input data 28x28 pixel 784 Number of output neural is number of categories of target data 10
  • 18. 18 Work required to create a image classifier ´ ´ ´ ´ ´ ´ ´ Classifier Neural Network Input : Image Output : Classification result 2 1. Prepare dataset Prepare as much data as possible Each data includes image and its class index 2. Design neural network structure You can use sample project 3. Train designed neural network using prepared dataset Basically you can create image classifier with these three steps ´ 仝0々 ´ 仝1々 ´ 仝2々 ´ 仝3々 ´ 仝4々 ´ 仝5々 ´ 仝6々 ´ 仝7々 ´ 仝8々 ´ 仝9々
  • 19. 19 1. Prepare dataset ? Create dataset CSV files according to a predetermined format x:image y:label ./training/5/0.png 5 ./training/0/1.png 0 ./training/4/2.png 4 ./training/1/3.png 1 ./training/9/4.png 9 ./training/2/5.png 2 ./training/1/6.png 1 ´ File name of hand-writing digit class Index It can be created with Excel or simple script
  • 20. 20 2. Design neural network structure ? Convolutional Neural Networks ? Put layers as it is by drag & drop, and edit properties of layers Add layers by drag & drop Edit property of layer You can design the network structure very easily and intuitively 5 x 5 Convolution Activation 2 x 2 Pooling 3 x 3 Convolution Activation 2 x 2 Pooling Affine Activation Fully connected Softmax Categorical Loss
  • 21. 21 3. Train designed neural network using prepared dataset ? Training starts just by pushing the training button When the training is complete, an image classifier is obtained loss training iteration

Editor's Notes