際際滷

際際滷Share a Scribd company logo

????? 2.0 ???? - RNN

?Download as PPTX, PDF?
?
H
Hwanhee Kim

2019? 4? 27?? ??? ?????? ?????? ??? ??? ???? <Tensorflow 2.0 ???? - RNN> ???????. RNN ? ?? ??? ??? ?? RNN ? ?? ??? ????, Google Colab Sample Code ? ?? 2.0 ?? RNN ? ???? ??? ???? ?????.

1 of 29
Downloaded 78 times
TensorFlow 2.0 Tutorial RNN ??? 2019.05.25
?? ? RNN ?? ? Tensorflow 2.0 - RNN ? Tensorflow 2.0 Sample code - RNN
RNN ??
RNN ? Recurrent Neural Network ? Recurrent : ????, ???? ?? X ?? Y ?? Y ?? X DNN RNN ???? ??? ?? RNN ? ??? ??
RNN ? Recurrent Neural Network ? ?? ????? ???? ??? ?? ??? ??? ? ?? ???? ??? ????? ?? ?? Y ?? X RNN ?? ?? RNN ?? ?? RNN ?? ?? RNN X1 X2 X3 Y1 Y2 Y3 Y1 Y2
RNN ? ?? ???? ??(Sequence)? ???? ?? ? ??, ??, ?? ? https://seanwes.com/book/ https://md2biz.tistory.com/148
RNN ? RNN ? ??? ??? LSTM ?? ?? ?? ??? ???? ?? https://github.com/NourozR/Stock-Price-Prediction-LSTM
RNN ? ??? ???? ??? ??? ???? ? ?? ??? ??? ? http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture10.pdf Image Captioning Sentiment Classification Machine Translation Video Classification (Frame level)
Image Captioning ? ???? CNN ?? ?? ? RNN ?? caption ?? https://www.analyticsvidhya.com/blog/2018/04/solving-an-image-captioning-task-using-deep-learning/
Sentiment Classification ? ??? ??/?? ?? https://openai.com/blog/unsupervised-sentiment-neuron/
Machine Translation ? Encoder ? ?? ??? ?? Decoder ?? ?? ? ??? : Decoder ? ???? ??? ??? ? ?? ???? ?? ? ??? Attention, Transformer, BERT ? ?? ????? ??? https://medium.com/@gautam.karmakar/attention-for-neural-connectionist-machine-translation-b833d1e085a3
Video Classification ? ???? ? ?????? ??? ?? ??? ?? https://arxiv.org/abs/1411.4389
RNN? ?? ? ??(Vanilla) RNN ? ??? ??? ?? ? 3?? ???(weights) ?? https://aikorea.org/blog/rnn-tutorial-1/ https://ratsgo.github.io/natural%20language%20processing/2017/03/09/rnnlstm/
RNN? ?? ? ?? ??? 1?? RNN? ???? ?? 3?? ?? ??? ? ? ?
RNN? ?? ? ?? ??? 2?? ?, 3?? ? ????? ?? ??
RNN? ?? ? ????? Fully connected ?? ? ?? ??? ?? = ???? ?? RNN units = 2 Param # = 8 RNN units = 3 Param # = 15 RNN units = 1 Param # = 3 bias bias bias
Tensorflow 2.0 - RNN
SimpleRNN ? tf.keras.layers ?? import ? ? ??
return_sequences ? RNN ?? ??? ?? hidden state ? ??? ???? ?? ?? ? ?? RNN ?? one-to-many, many-to-many ??? ?? ?? ?? Y1 ?? X1 RNN ?? Y2 ?? X2 RNN ?? Y3 ?? X3 RNN ?? Y1 ?? X1 RNN ?? Y2 ?? X2 RNN ?? Y3 ?? X3 RNN return_sequences = False (?? ???) return_sequences = True Y1 Y2 Y1 Y2
return_state ? RNN ? ??? ??? LSTM ? ??? ??? cell_state ? ?? ? ??? cell_state ? ??? ???? ?? ?? ? ??? ???? ???? ??? https://excelsior-cjh.tistory.com/185 SimpleRNN LSTM
Embedding ? ??? ??(character)? ??? ? ?? ???? ?? n? ?? ??? ?? [[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]]
Embedding ? tf.keras.layers ?? import ? ? ?? ? input_dim : ?? ? ? output_dim : embedding ?? ??
Embedding my cat is smart 0 1 2 3 [0.76, -0.11, 0.36, ´, -0.93] [0.7, 0.42, -0.1, ´, -0.34] [0.83, 0.75, 0.64, ´, -0.4] [-0.33, 0.56, 0.82, ´, 0.71] ?? ?? ??? Embedding ?? ?? output_dim=64 input_dim =1000
Dense(softmax) ? RNN ???? ??? ??? softmax ? ?? ?? https://www.dlology.com/blog/how-to-generate-realistic-yelp-restaurant-reviews-with-keras/
Tensorflow 2.0 sample code - RNN
Text generation ? RNN ? ??? Text generation ? ???? ?? Andrej Karpathy ? <The Unreasonable Effectiveness of Recurrent Neural Networks> ? ??? ?? ?? ?? RNN ??? ????? ??, ????? ??, LaTeX, ?? ?? ?? ?? ??? ?? ??
????? 2.0 ???? - RNN
RNN with KoreanHiphop Dataset ? ?? ?? ?? RNN C Google Colab ?? ?? ? ?? ?? ?? RNN C Google Colab ?? ??
Thank you!

More Related Content

????? 2.0 ???? - RNN

  • 2. ?? ? RNN ?? ? Tensorflow 2.0 - RNN ? Tensorflow 2.0 Sample code - RNN
  • 4. RNN ? Recurrent Neural Network ? Recurrent : ????, ???? ?? X ?? Y ?? Y ?? X DNN RNN ???? ??? ?? RNN ? ??? ??
  • 5. RNN ? Recurrent Neural Network ? ?? ????? ???? ??? ?? ??? ??? ? ?? ???? ??? ????? ?? ?? Y ?? X RNN ?? ?? RNN ?? ?? RNN ?? ?? RNN X1 X2 X3 Y1 Y2 Y3 Y1 Y2
  • 6. RNN ? ?? ???? ??(Sequence)? ???? ?? ? ??, ??, ?? ? https://seanwes.com/book/ https://md2biz.tistory.com/148
  • 7. RNN ? RNN ? ??? ??? LSTM ?? ?? ?? ??? ???? ?? https://github.com/NourozR/Stock-Price-Prediction-LSTM
  • 8. RNN ? ??? ???? ??? ??? ???? ? ?? ??? ??? ? http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture10.pdf Image Captioning Sentiment Classification Machine Translation Video Classification (Frame level)
  • 9. Image Captioning ? ???? CNN ?? ?? ? RNN ?? caption ?? https://www.analyticsvidhya.com/blog/2018/04/solving-an-image-captioning-task-using-deep-learning/
  • 10. Sentiment Classification ? ??? ??/?? ?? https://openai.com/blog/unsupervised-sentiment-neuron/
  • 11. Machine Translation ? Encoder ? ?? ??? ?? Decoder ?? ?? ? ??? : Decoder ? ???? ??? ??? ? ?? ???? ?? ? ??? Attention, Transformer, BERT ? ?? ????? ??? https://medium.com/@gautam.karmakar/attention-for-neural-connectionist-machine-translation-b833d1e085a3
  • 12. Video Classification ? ???? ? ?????? ??? ?? ??? ?? https://arxiv.org/abs/1411.4389
  • 13. RNN? ?? ? ??(Vanilla) RNN ? ??? ??? ?? ? 3?? ???(weights) ?? https://aikorea.org/blog/rnn-tutorial-1/ https://ratsgo.github.io/natural%20language%20processing/2017/03/09/rnnlstm/
  • 14. RNN? ?? ? ?? ??? 1?? RNN? ???? ?? 3?? ?? ??? ? ? ?
  • 15. RNN? ?? ? ?? ??? 2?? ?, 3?? ? ????? ?? ??
  • 16. RNN? ?? ? ????? Fully connected ?? ? ?? ??? ?? = ???? ?? RNN units = 2 Param # = 8 RNN units = 3 Param # = 15 RNN units = 1 Param # = 3 bias bias bias
  • 19. return_sequences ? RNN ?? ??? ?? hidden state ? ??? ???? ?? ?? ? ?? RNN ?? one-to-many, many-to-many ??? ?? ?? ?? Y1 ?? X1 RNN ?? Y2 ?? X2 RNN ?? Y3 ?? X3 RNN ?? Y1 ?? X1 RNN ?? Y2 ?? X2 RNN ?? Y3 ?? X3 RNN return_sequences = False (?? ???) return_sequences = True Y1 Y2 Y1 Y2
  • 20. return_state ? RNN ? ??? ??? LSTM ? ??? ??? cell_state ? ?? ? ??? cell_state ? ??? ???? ?? ?? ? ??? ???? ???? ??? https://excelsior-cjh.tistory.com/185 SimpleRNN LSTM
  • 21. Embedding ? ??? ??(character)? ??? ? ?? ???? ?? n? ?? ??? ?? [[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]]
  • 22. Embedding ? tf.keras.layers ?? import ? ? ?? ? input_dim : ?? ? ? output_dim : embedding ?? ??
  • 23. Embedding my cat is smart 0 1 2 3 [0.76, -0.11, 0.36, ´, -0.93] [0.7, 0.42, -0.1, ´, -0.34] [0.83, 0.75, 0.64, ´, -0.4] [-0.33, 0.56, 0.82, ´, 0.71] ?? ?? ??? Embedding ?? ?? output_dim=64 input_dim =1000
  • 24. Dense(softmax) ? RNN ???? ??? ??? softmax ? ?? ?? https://www.dlology.com/blog/how-to-generate-realistic-yelp-restaurant-reviews-with-keras/
  • 25. Tensorflow 2.0 sample code - RNN
  • 26. Text generation ? RNN ? ??? Text generation ? ???? ?? Andrej Karpathy ? <The Unreasonable Effectiveness of Recurrent Neural Networks> ? ??? ?? ?? ?? RNN ??? ????? ??, ????? ??, LaTeX, ?? ?? ?? ?? ??? ?? ??
  • 28. RNN with KoreanHiphop Dataset ? ?? ?? ?? RNN C Google Colab ?? ?? ? ?? ?? ?? RNN C Google Colab ?? ??

Editor's Notes

  1. ?????. ?? ?? Tensorflow 2.0 Tutorial, RNN ??? ??? ??? ?? ?????? ????? ???.
  2. ??? ??? ????. ?? RNN ? ?? ??? ?? ???? ?? ???? ??? ??? ?? ????, Tensorflow 2.0 ?? RNN ? ???? ??? ??? ?? ?? ??? ???, Google Colab ?? ??? ?? ??? ?? ?? ???? generation ?? ??? ???????.
  3. ?? RNN ?????.
  4. RNN ? Recurrent Neural Network ? ???, Recurrent ? ????, ???? ??? ????. ???? ??? ??? ??? RNN ? ??? ??? ??? ????.
  5. ??? ? ??? ????, RNN ? ?? ????? ???? ??? ??? ???? ?????. ??? ??? ??? ?? ??? ???? ???. ?? ??? ??? ??? ?? ????? ???? ??? ????? ?? ??? ?? ??? ???? ???.
  6. ???? ???? ?? ?? ????? ?? ???? ??? ?? ?????. ??, ??, ?? ? ??? ??? ?? ??? ???? ???, ?? ???? ???? ??? ?? ???? ??? ??? ???? ??? ??? ???.
  7. RNN ?? ?? ?? ??? ???? ?????. ???? ???? ??? ?? ??? ??? ??? ?????.
  8. RNN ? ??? ???? ??? ??? ???? ? ?? ??? ??? ??? ????. ?? ??? ???? RNN ?????, one-to-many, many-to-one ? ??? ??? ??? ?? ????.
  9. Image captioning ? ???? cnn ?? ???? feature ? ??? ? ? ??? ???? RNN ?? ??? caption ? ???? ???????.
  10. Sentiment classification ? ??? ??? ??? ??/??? ???? ?????. Openai ? ? ???? ????? ??? ????? ???? ?? ?? ???? ??, ???? ???? ??? ??? ?? ?? ??? ??? ?? ? ? ????.
  11. Machine translation ? ? ??? ?? ?????. ???? ?? ??? encoder / decoder ??? encoder ?? rnn ? ?? ??? ??? ????? ??? ???, decoder rnn ?? ?? ??? ??? ???? ???????. ??? ? ??? decoder ? ???? ??? ??? ???? ???? ??? ???? ?????. ??? attention, transformer, bert ? ?? ?? ??? ???????.
  12. Video classification ? ??? ??? ??? ?? ???? ?? ??? ??? ???? ?? ???? ???? ?????.
  13. Rnn ? ??? ?????, ?? RNN ? ??? ??? ??? ??????. ?? ??? ??? ?? ????? ???.
  14. RNN ??? ??? ?? ? ???? ????????. ?? colab ?? ?? ??? RNN ????? ???? ?????, ???? ?? 3?? ?? ? ? ????.
  15. ??? ??? ??? ?? 2, 3 ?? ??? ???? ?? ?? 8 ?, 15 ?? ????.
  16. ??? ? ??? ? ? ????. RNN ? Dense Layer ?? ????? Fully Connected ???? ??? RNN ? ?? ?? ??? ? ???? ?? ???? ???? ??? ???? ???. ??? ?? ??? ?? ???? ??? ???.
  17. ?? Tensorflow 2.0 ?? RNN ? ?? ??? ???? ???????.
  18. ?? ???? RNN ???? ??? SimpleRNN ???. ??? ?? ??? ???? ?? ??? ?? ????. Units ? rnn ? ??? ???? ?? ????. Activation ? sigmoid ? ????? ???? -1 ?? 1 ??? tanh ??? ?????. ? ??? ???? ??????. ?? ????? return_sequences ? return_state ? ???? ?? ????????.
  19. Return_sequences ? rnn ?? ???? hidden state ? ??? ???? ???? ?????. ??? ? ????? ?? ?? ??? ??? one-to-many, many-to-many ??? rnn ?? ?? ? ??? ?? rnn ???? rnn ??? ?? ?? ??? ?? ?? ???? ?????. ??? ?? ?????? return_sequences ? True ? ?? ????.
  20. Return_state ? ??? ???? ???? ??? ?? ??? ???? LSTM ? ??? ?? ??? ??? ????????. ??? simpleRNN ? ???? h ? ??? hidden state ? ????? ?? ??? ?????. ?? ?? lstm ? c ?? ?? ?? ????? ??? cell ????? ???? cell_state ???. ??? return_state ? ? cell_state ? ??? ???? ??? ?????.
  21. Rnn ? ??? ???? ?? 2?? layer ? ??? ? ??? ??????. Embedding layer ? ??? ???? ????? ??? layer ???. ?????? ??? ?? n ?? ?? ??? ?????.
  22. Embedding ???? tf.keras.layers ?? import ? ? ????. Input dimension ? output dimension ? ?? ???? ?? ?? embedding ?? ??? ?????.
  23. ??? ? embedding layer ? ??? ??? ???.
  24. Dense ? softmax activation ? rnn ?? ??? ??? ??? ???? ??? ????.
  25. ?? ?? ?? ?? ??? ?? Tensorflow 2.0 ?? CNN ? ???? ??? ??? ?????? ?????.
  26. Rnn ? ??? ?? ?? ?????? ??? text generation ? ????. ? ? ???? ?? ?? ???? ai ???? ?? Andrej karpathy ? ???? ???? ??? ??? ??? ??? rnn ?????. ??? ???? ?? ??? ?? ???? ??? ?? ?? ?? rnn ? ????? ????. ? ????? ??, ?? ??, latex ? ?? ?? ??? ???? ?? ??????.
  27. ?? ???? ??? LaTex ?? ?????. ??? ?? ??? ???? ???? ?? ??? ????? ?????. ???? ?????? ???? ?? ??? ? ????.
  28. ??? ?? ???? ?? ???? text generation ? ?? ??? ??? ?????. ??? ???? ?????? 11,000 ? ?? ?? ?? ?? ??? ?? Korean hiphop ??????. ? ???? ????? ???????. ? ? ?? ?? rnn ? ?? ????, ?? ?? rnn ? ??? ??? ?? ???????.