際際滷

際際滷Share a Scribd company logo

Chapter 10 sequence modeling recurrent and recursive nets

Download as PPTX, PDF
K
KyeongUkJang

Chapter 10 sequence modeling recurrent and recursive nets

1 of 44
Download to read offline
10 RNN 蟾焔 襯燕
磯Μ る RNN(Recurrent neural network) 伎 覦一 蟆企. 願姥襦 覓伎 讌襯 螳螻 企 朱 企 蟲譟郁 蟆朱讌 覲伎.
RNN 磯 覿 一危郁 谿朱 蟇磯 蟯螻螳 譴 蟆曙一 RNN 覦.
蠏碁る RNN 覘蠍碁 襷 螳 企 蟆手?? 讀 RNN 蟯螻螳 譴 一危一 伎 ル 蟆. 蟆暑(RNN) 螻 一危一 螳 螳 襴 磯 覲 一危磯ゼ 牛蠍 ル 覈碁, 蠍一 (t)螻 れ (t+1) ろ語襯 郁屋 蟲燕 瑚概蟆暑
- 一危一 譴讌 . - れ 襷 蠍一ヾ DNN 螳 襯 螻ろ讌 螻 譯殊伎 一危磯 螳讌螻 . - 一危郁 覺れ 一危一語 蠍一牛 れ . - 蠏語朱襦 る襷 DNN, NN 蟆曙, 一危磯ゼ ロ覃 一一 レ元 豸(hidden layers)襯 蟇一 豢リ讌 谿蠏殊姶蠏 讌. 螻殊 一危磯 覈 碁襯 覯 讌螳蟆 . RNN螻 NN 谿伎?? DNN
ｼ豺蠍 RNN螻 NN 谿伎?? RNN 蟲譟 : 伎 豸旧 螳 れ 豸旧朱 . 讀 蠍一 RNN(Recurrent Neural Network) 企 覿. 覦覃伎 RNN 讌蠍 ル 一危一 螻手碓 ル 一危磯ゼ 螻ろ蟆 . 蠍磯蓋 蟲譟 れ螻 螳.
れ元 蟆暑 れ螻 螳 蟲譟磯ゼ 覲伎譴.
RNN 蟲譟一 蟆曙 朱誤磯 5螳螳 譟伎. 螳譴豺 3螳 1. レ元螻 豸旧 螳譴豺 モ 2. 豸糾骸 豸旧 螳譴豺 3. 豸糾骸 豢レ元 螳譴豺 Bias 2螳 1. レ元 豸旧朱 願 伎朱 2. 豸旧 豢レ元朱 願 伎朱
RNN 蠍磯蓋 螻 レ元 : 豸旧 : モ + ≠1 + = 豸旧 豢 : = 豢レ元 : + = 豢レ元 豢 : ( ) , 燕
蠍磯蓋 RNN蟲譟一 伎 覲願鍵 レ元 : 豸旧 : モ + ≠1 + = 豸旧 豢 : = 豢レ元 : + = 豢レ元 豢 : = , 燕
蠍 ロ伎 Hello朱 企ゼ 豸″ 豌 h[1,0,0,0]企朱 レ朱 e[0,1,0,0]企朱 豢レ 豸″ 覦レ朱 旧 讌. 蠏 れ l[0,0,1,0]螻 o[0,0,0,1] 伎 襷谿螳讌襦 旧 讌.
企 RNN 轟 螳讌螳 , 襷り覲襯 螻旧る 蟆. 讀 リ朱 ≠1 ｋ, +4襯 ｋ 螳譴豺 モ 螳 蟆 . 襷谿螳讌襦 ′ +1襦 願 , +17 +18襦 願 , 螳 襯 . 企 るジ 襷り覲 襷谿螳讌企. 企 轟朱 誤 旧 觜襯願 讀螳讌襷, れ 豕襦 襷り覲襯 覦蟆 螻殊 伎 豕螳 れ 企れ る . RNN 轟
覦 RNN 伎 覲企れ RNN豌企 譬 焔レ 蠍磯 讌襷, 危 覲企ゼ 譬 蟆郁骸襯 蠍磯 . Ex) 碁ジ _____ も 朱 覓語レ 螳企 觜豺語 れ願 企ゼ 豸″ , 碁ジ, 企朱 覲企 蟲襴 豸″ 讌襷, 危 覲伎 も襯 牛伎 覲企 蟆 蟲襴企朱 企ゼ 豸″ .
覦 RNN 蟲譟 豸旧 危 覲企覿 レ 覦 蟆 覲 .
RNN 1. 螻蠍 & 豢. ) 企語襯 ロ伎 企語 る 覓語レ朱 豢ロ 企語 貂′ 2. & 螻蠍 豢. ) 覓語レ ロ伎 蠍覿 襯 豢ロ 螳 覿蠍 3. & 豢. ) 企ゼ 蟲朱 覯 覯蠍
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory The clouds Are in The ? ? 覘蟾?
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory "the clouds are in the 朱 リ 覦螻 襷讌襷 企ゼ 豸″伎 る, 覲危 磯Μ 伎 覓碁Д(覲) 讌 . 覈蟆 れ ル 企 sky螳 襯 . 企 蟆曙一 , 螻給 一危一 豸″伎 覲伎 豺 谿(Gap)螳 讌 る, RNN 螻手碓 一 磯ゼ 蠍磯朱 所 牛 . The clouds Are in The ?
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory 2. ? 襯 襷豢蠍 伎 覓碁Д 危危 も 朱蟾 れ器襯 谿渚蟆 蟆蟲! 朱 伎 . 3. 讌襷 リ 谿朱 れ伎り鍵 覓語 France朱 伎 れ 襷豢一 ? 蟇磯Μ螳 覃伎覃 RNN 覲伎 覓碁Д 郁屋蠍郁 れ伎. grew up France I speak fluent ? (I grew up in France ... I speak fluent French) 1. 蠏碁る ル 覓語レ 蠍語伎り 企慨 I
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory grew up France I speak fluent ? (I grew up in France ... I speak fluent French) 蟇磯Μ螳 覃伎覃 襷豢蠍郁 れ?
RNN 襷レ . 蟇煙蟇磯Μ螳 . 蠍一ヾ RNN 螳 れ 螳 螳 朱誤磯ゼ 螻旧螻 . れ 郁屋 螻 Recurrent螻 蟆企蟾(蠏企.) Long Short Term Memory
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory 磯殊 螳譴豺()螳 1覲企 譟郁企朱 => 伎 蟇磯Μ螳 覃伎. = 覦 覯 螻燕 => 0 螳蟾 覃誤伎 1覲企 譟郁企朱 => 伎 蟇磯Μ螳 覃伎. = 覦 覯 螻燕 => 覓 貉れ 覦. 襷豢蠍 企糾, 豕 企給. (蟲 : 覿螳ロ 蟇 旧 譯 る 蟇碁Π.) 蠍一ヾ RNN 螳 れ 螳 螳 朱誤一企. れ 郁屋 螻 Recurrent螻 蟆企蟾(蠏企.)
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory 豌伎語 れ願 螳 螻 覦蠖譯朱 蟆 => 螳 譯殊 襷螻 control unit 譯殊! 蠏碁 煙 : Long Short Term Memory network(LSTM) 誤 覯 企慨
RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory network 襷覃 LSTM襷螻 るジ 覦覯朱 願屋 蠍磯 . 10.8 覦 覦 蟆暑 10.9 豢 覦 れ 螳 豢 But 螳 10.10 LSTM 伎手鍵覃伎 豈 磯 , れ 一企 螳 螻殊 谿 覈 LSTM企. 手 覦給. 磯殊 10.8螻 10.9襯 牛螻 10.10 LSTM朱 願 蟆 覦.
Long Short Term Memory 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 朱 讌 襯 牛 蟆)
Long Short Term Memory 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎. 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 朱 讌 襯 牛 蟆)
Long Short Term Memory 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 襯 牛 蟆) 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎. forget gate : - 螻手碓 覲企ゼ 蠍謂 蟆危 - 螳 覯 0 ~ 1 - 0 => 伎 覲企 . - 1 => 伎 覲企ゼ 蠍一牛.
Long Short Term Memory 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 襯 牛 蟆) 蠍一ヾ Rnn豌 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎. input gate : - 覲企ゼ 蠍一牛蠍謂 蟆危 - 螳 覯 0 ~ 1 - 0 => 覲企 . - 1 => 覲企ゼ 蠍一牛. => 襦 襷 cell 螳
Long Short Term Memory - 襯 sigmoid襦 蠍 蠍 覓語 0~1 伎 螳 豢 ＄ * 蠍一ヾ RNN - リ朱 伎 豢リ ≠1, ヰ°ゼ 覦. レ元 : ヰ 豸旧 : モ ヰ + ≠1 + = 豸旧 豢 : = 豢レ元 : + = 豢レ元 豢 : = , 燕
Long Short Term Memory i ＄ (襦 襷 cell 螳) 襷 tanh襯 磯? 1. Gateれ 一危碁ゼ 蟆企 0~1 螳 豢ロ伎 伎 蠏碁企 2. tanh螳 譬り .
Long Short Term Memory O＄ ＄ - C(cell) 襷 - C 蠍一/襷螳 襯 牛 蟆 - C 豕譬 豢リ , C襯 伎伎 豕譬 豸旧 豢リ 蟆一. 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 襯 牛 蟆) 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎.
讌蠍蟾讌 LSTM 蟲ロ 覯 LSTM 覈 LSTM 狩 蟲譟磯 x (蠏碁觜結訣蠍磯 伎) Long Short Term Memory 襴
讌蠍蟾讌 LSTM 蟲ロ 覯 LSTM 覈 LSTM 狩 蟲譟磯 x (蠏碁觜結訣蠍磯 伎) Long Short Term Memory
Gate 豌伎 伎 螻 Cell ｋ. Long Short Term Memory Gers & Schmidhuber (2000), is adding peephole connections.
Input gate 1 伎 語一 襷 覿譟燕 覿覿 襦 語一朱 豈企. Make sense Long Short Term Memory coupled forget and input gates
Long Short Term Memory Gated Recurrent Unit(GRU) - 螳 豕(2014) - LSTM レ 讌覃伎 螻磯概′煙 豢 蟲譟 : update 蟆危~ input gate : reset 蟆危~ forget gate : (t) 蠍一牛企 襷 覲 : れ (state)襦 一危
覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive recurrent neural network - リ 襦 覦 谿朱 豌襴 ろ語 蟲譟 - リ 譴螳 蟇企郁碓 覿覿 螻 煙レ襦 蠏碁襦 豌襴 蟲譟 - 襷讌襷 碁(2.5, 3.8) 伎蟾讌 覈 襷ル(the, country, of, my)螻 蟷 リ(birth) 覲願 覈 覦.
覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive convolutional neural network - リ 旧 覈 覦る Recurrent Neural Networks 谿 企 - 讌襷 リ (the, country) 覲企 Recurrent Neural Networks 襴 CNN 蠏碁殊 覲企 螳 伎(the country, country of, of my) 覯 覿 - 覯 譯殊 螻給れ, 磯朱 豺蟲螳 2螳 伎 覲企ゼ 豢伎 覲企ゼ 企. 蠏 朱 危危覃 .
覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive recursive neural network - 讌襷 CNN 覈 覲企ゼ 旧 覦 觜 RNN 朱 覲企 ろ る 谿 - the country of my birth 覦 蟲譟 - CNN 覦豌 the country, country of, of my 企蟆 覈 覿 螳 - RNN 企 語伎 螻豸旧 煙 ろ語 蟲譟一 蠏 谿 覈- 一 豌襴 襷 - Recursive Neural Networks(RNN) リ朱 譯殊伎 覈 螳 企ゼ 覓苦 覿る 伎CNN螻 convolutional neural network
覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive - Recurrent Neural Networks Recursive Neural Networks 轟 貅伎 - 襷 Recursive Neural Networks螳 覈 覲企ゼ 襦 觜讌 覦り 覃 殊 蠏碁 螳 蟲譟
企ゼ 螳 伎 覲企 覲語朱 Recurrent Neural Networks 螳 覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive 讌 蠍..伎れ 覦 覯
覿襦 襭 谿城り 蟆 る convolutional + recursive 蠍一ヾ cnn -> 讌 れ伎る => 蟆 覓伎瑚? 覿襯
覿襦 襭 谿城り 蟆 る convolutional + recursive 螻 一危 語 螳 一危一 label 蟆 一 label 豸′ 朱 一危 語螻 るゴ蟆 螻 企語 一危磯ゼ 語螻 矩. CNN + RNN ! => CRNN
覿襦 襭 谿城り 蟆 る convolutional + recursive 覓碁Д 磯ジ 蠍 , 覓語 語 覲 覓碁Д 語 焔レ 覲伎碁り
!

Recommended

Chapter 7 Regularization for deep learning - 2
Chapter 7 Regularization for deep learning - 2Chapter 7 Regularization for deep learning - 2
Chapter 7 Regularization for deep learning - 2
KyeongUkJang
油
伎螻 貅殊る 覦一磯 螳 濠
伎螻 貅殊る 覦一磯 螳 濠伎螻 貅殊る 覦一磯 螳 濠
伎螻 貅殊る 覦一磯 螳 濠
Woong won Lee
油
Chapter 19 Variational Inference
Chapter 19 Variational InferenceChapter 19 Variational Inference
Chapter 19 Variational Inference
KyeongUkJang
油
ddpg seminar
ddpg seminarddpg seminar
ddpg seminar
覩殊
油
Graph Neural Network (蟲)
Graph Neural Network (蟲)Graph Neural Network (蟲)
Graph Neural Network (蟲)
Jungwon Kim
油
Deep-Learning Based Stereo Super-Resolution
Deep-Learning Based Stereo Super-ResolutionDeep-Learning Based Stereo Super-Resolution
Deep-Learning Based Stereo Super-Resolution
NAVER Engineering
油
螳蟾螻 襾 Trpo
螳蟾螻 襾 Trpo螳蟾螻 襾 Trpo
螳蟾螻 襾 Trpo
Woong won Lee
油
ル螻 螳 旧朱 覲企 貎る AI 蟲蠍 DEVIEW 2016
ル螻 螳 旧朱 覲企 貎る AI 蟲蠍 DEVIEW 2016ル螻 螳 旧朱 覲企 貎る AI 蟲蠍 DEVIEW 2016
ル螻 螳 旧朱 覲企 貎る AI 蟲蠍 DEVIEW 2016
Taehoon Kim
油
螳 螻襴讀 襴 Part 2
螳 螻襴讀 襴 Part 2螳 螻襴讀 襴 Part 2
螳 螻襴讀 襴 Part 2
Dongmin Lee
油
Continuous control with deep reinforcement learning (DDPG)
Continuous control with deep reinforcement learning (DDPG)Continuous control with deep reinforcement learning (DDPG)
Continuous control with deep reinforcement learning (DDPG)
Taehoon Kim
油
Densely Connected Convolutional Networks
Densely Connected Convolutional NetworksDensely Connected Convolutional Networks
Densely Connected Convolutional Networks
harmonylab
油
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
Kyunghwan Kim
油
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement Learning
NAVER Engineering
油
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
Shuai Zhang
油
ろ語 覈 蟆
ろ語 覈 蟆ろ語 覈 蟆
ろ語 覈 蟆
NAVER Engineering
油
Safe Reinforcement Learning
Safe Reinforcement LearningSafe Reinforcement Learning
Safe Reinforcement Learning
Dongmin Lee
油
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning
DongHyun Kwak
油
Introduction to A3C model
Introduction to A3C modelIntroduction to A3C model
Introduction to A3C model
WEBFARMER. ltd.
油
Boosting_suman
Boosting_sumanBoosting_suman
Boosting_suman
suman_lim
油
螳旧 襴 Part 1
螳旧 襴 Part 1螳旧 襴 Part 1
螳旧 襴 Part 1
Dongmin Lee
油
RLCode A3C 所 蟾蟆 危危蠍
RLCode A3C 所 蟾蟆 危危蠍RLCode A3C 所 蟾蟆 危危蠍
RLCode A3C 所 蟾蟆 危危蠍
Woong won Lee
油
蠏碁 蠏碁Μ AI
蠏碁 蠏碁Μ AI蠏碁 蠏碁Μ AI
蠏碁 蠏碁Μ AI
NAVER Engineering
油
Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)
Suhyun Cho
油
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
油
Understanding cnn
Understanding cnnUnderstanding cnn
Understanding cnn
Rucha Gole
油
Backpropagation in Convolutional Neural Network
Backpropagation in Convolutional Neural NetworkBackpropagation in Convolutional Neural Network
Backpropagation in Convolutional Neural Network
Hiroshi Kuwajima
油
ル - 企 蠍一
ル - 企 蠍一ル - 企 蠍一
ル - 企 蠍一
Hyungsoo Ryoo
油
[蠍一螳] Graph Convolutional Network (GCN)
[蠍一螳] Graph Convolutional Network (GCN)[蠍一螳] Graph Convolutional Network (GCN)
[蠍一螳] Graph Convolutional Network (GCN)
Donghyeon Kim
油
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
Lee Seungeun
油
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
jdo
油

More Related Content

What's hot (20)

螳 螻襴讀 襴 Part 2
螳 螻襴讀 襴 Part 2螳 螻襴讀 襴 Part 2
螳 螻襴讀 襴 Part 2
Dongmin Lee
油
Continuous control with deep reinforcement learning (DDPG)
Continuous control with deep reinforcement learning (DDPG)Continuous control with deep reinforcement learning (DDPG)
Continuous control with deep reinforcement learning (DDPG)
Taehoon Kim
油
Densely Connected Convolutional Networks
Densely Connected Convolutional NetworksDensely Connected Convolutional Networks
Densely Connected Convolutional Networks
harmonylab
油
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
Kyunghwan Kim
油
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement Learning
NAVER Engineering
油
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
Shuai Zhang
油
ろ語 覈 蟆
ろ語 覈 蟆ろ語 覈 蟆
ろ語 覈 蟆
NAVER Engineering
油
Safe Reinforcement Learning
Safe Reinforcement LearningSafe Reinforcement Learning
Safe Reinforcement Learning
Dongmin Lee
油
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning
DongHyun Kwak
油
Introduction to A3C model
Introduction to A3C modelIntroduction to A3C model
Introduction to A3C model
WEBFARMER. ltd.
油
Boosting_suman
Boosting_sumanBoosting_suman
Boosting_suman
suman_lim
油
螳旧 襴 Part 1
螳旧 襴 Part 1螳旧 襴 Part 1
螳旧 襴 Part 1
Dongmin Lee
油
RLCode A3C 所 蟾蟆 危危蠍
RLCode A3C 所 蟾蟆 危危蠍RLCode A3C 所 蟾蟆 危危蠍
RLCode A3C 所 蟾蟆 危危蠍
Woong won Lee
油
蠏碁 蠏碁Μ AI
蠏碁 蠏碁Μ AI蠏碁 蠏碁Μ AI
蠏碁 蠏碁Μ AI
NAVER Engineering
油
Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)
Suhyun Cho
油
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
油
Understanding cnn
Understanding cnnUnderstanding cnn
Understanding cnn
Rucha Gole
油
Backpropagation in Convolutional Neural Network
Backpropagation in Convolutional Neural NetworkBackpropagation in Convolutional Neural Network
Backpropagation in Convolutional Neural Network
Hiroshi Kuwajima
油
ル - 企 蠍一
ル - 企 蠍一ル - 企 蠍一
ル - 企 蠍一
Hyungsoo Ryoo
油
[蠍一螳] Graph Convolutional Network (GCN)
[蠍一螳] Graph Convolutional Network (GCN)[蠍一螳] Graph Convolutional Network (GCN)
[蠍一螳] Graph Convolutional Network (GCN)
Donghyeon Kim
油
螳 螻襴讀 襴 Part 2
螳 螻襴讀 襴 Part 2螳 螻襴讀 襴 Part 2
螳 螻襴讀 襴 Part 2
Dongmin Lee
油
Continuous control with deep reinforcement learning (DDPG)
Continuous control with deep reinforcement learning (DDPG)Continuous control with deep reinforcement learning (DDPG)
Continuous control with deep reinforcement learning (DDPG)
Taehoon Kim
油
Densely Connected Convolutional Networks
Densely Connected Convolutional NetworksDensely Connected Convolutional Networks
Densely Connected Convolutional Networks
harmonylab
油
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
螳 企 蟲: Rainbow 企覿 蟲蟾讌 (2nd dlcat in Daejeon)
Kyunghwan Kim
油
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement Learning
NAVER Engineering
油
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
Shuai Zhang
油
ろ語 覈 蟆
ろ語 覈 蟆ろ語 覈 蟆
ろ語 覈 蟆
NAVER Engineering
油
Safe Reinforcement Learning
Safe Reinforcement LearningSafe Reinforcement Learning
Safe Reinforcement Learning
Dongmin Lee
油
Reinforcement Learning
Reinforcement LearningReinforcement Learning
Reinforcement Learning
DongHyun Kwak
油
Introduction to A3C model
Introduction to A3C modelIntroduction to A3C model
Introduction to A3C model
WEBFARMER. ltd.
油
Boosting_suman
Boosting_sumanBoosting_suman
Boosting_suman
suman_lim
油
螳旧 襴 Part 1
螳旧 襴 Part 1螳旧 襴 Part 1
螳旧 襴 Part 1
Dongmin Lee
油
RLCode A3C 所 蟾蟆 危危蠍
RLCode A3C 所 蟾蟆 危危蠍RLCode A3C 所 蟾蟆 危危蠍
RLCode A3C 所 蟾蟆 危危蠍
Woong won Lee
油
蠏碁 蠏碁Μ AI
蠏碁 蠏碁Μ AI蠏碁 蠏碁Μ AI
蠏碁 蠏碁Μ AI
NAVER Engineering
油
Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)
Suhyun Cho
油
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
油
Understanding cnn
Understanding cnnUnderstanding cnn
Understanding cnn
Rucha Gole
油
Backpropagation in Convolutional Neural Network
Backpropagation in Convolutional Neural NetworkBackpropagation in Convolutional Neural Network
Backpropagation in Convolutional Neural Network
Hiroshi Kuwajima
油
ル - 企 蠍一
ル - 企 蠍一ル - 企 蠍一
ル - 企 蠍一
Hyungsoo Ryoo
油
[蠍一螳] Graph Convolutional Network (GCN)
[蠍一螳] Graph Convolutional Network (GCN)[蠍一螳] Graph Convolutional Network (GCN)
[蠍一螳] Graph Convolutional Network (GCN)
Donghyeon Kim
油

Similar to Chapter 10 sequence modeling recurrent and recursive nets (20)

CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
Lee Seungeun
油
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
jdo
油
Attention is all you need る
Attention is all you need るAttention is all you need る
Attention is all you need る
Junho Lee
油
R.T.Bach
R.T.BachR.T.Bach
R.T.Bach
豌 覦
油
LSTM ろ語 危危蠍
LSTM ろ語 危危蠍LSTM ろ語 危危蠍
LSTM ろ語 危危蠍
Mad Scientists
油
Final project v0.84
Final project v0.84Final project v0.84
Final project v0.84
Soukwon Jun
油
Computational Complexity
Computational ComplexityComputational Complexity
Computational Complexity
skku_npc
油
檎稼稼螳襴
檎稼稼螳襴檎稼稼螳襴
檎稼稼螳襴
譬 豕
油
Rnn keras
Rnn kerasRnn keras
Rnn keras
Park Seong Hyeon
油
Sequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural NetworksSequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural Networks
Hoon Heo
油
貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic
貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic
貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic
讌 譟
油
Seed2016 - 螳覩語 譯殊 (annotated)
Seed2016 - 螳覩語 譯殊 (annotated)Seed2016 - 螳覩語 譯殊 (annotated)
Seed2016 - 螳覩語 譯殊 (annotated)
Jooyung Han
油
Deep Learning for Chatbot (2/4)
Deep Learning for Chatbot (2/4)Deep Learning for Chatbot (2/4)
Deep Learning for Chatbot (2/4)
Jaemin Cho
油
Cnn 螳
Cnn 螳Cnn 螳
Cnn 螳
油
伎螻 一 5 | ル
伎螻 一 5 | ル伎螻 一 5 | ル
伎螻 一 5 | ル
蟾覯 | 覓伎誤磯伎朱
油
2.supervised learning(epoch#2)-3
2.supervised learning(epoch#2)-32.supervised learning(epoch#2)-3
2.supervised learning(epoch#2)-3
Haesun Park
油
Text2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to ActionText2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to Action
NAVER Engineering
油
07. PCA
07. PCA07. PCA
07. PCA
Jeonghun Yoon
油
Lecture 4: Neural Networks I
Lecture 4: Neural Networks ILecture 4: Neural Networks I
Lecture 4: Neural Networks I
Sang Jun Lee
油
Recurrent Neural Net 企螻 る
Recurrent Neural Net 企螻 るRecurrent Neural Net 企螻 る
Recurrent Neural Net 企螻 る
覦 蟾
油
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
CNN 豐覲伎螳 襷 豐覲伎 螳企 (VGG 所 )
Lee Seungeun
油
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
[貉危磯螻 瑚概讌] 7. 炎咳 蟆暑 2
jdo
油
Attention is all you need る
Attention is all you need るAttention is all you need る
Attention is all you need る
Junho Lee
油
R.T.Bach
R.T.BachR.T.Bach
R.T.Bach
豌 覦
油
LSTM ろ語 危危蠍
LSTM ろ語 危危蠍LSTM ろ語 危危蠍
LSTM ろ語 危危蠍
Mad Scientists
油
Final project v0.84
Final project v0.84Final project v0.84
Final project v0.84
Soukwon Jun
油
Computational Complexity
Computational ComplexityComputational Complexity
Computational Complexity
skku_npc
油
檎稼稼螳襴
檎稼稼螳襴檎稼稼螳襴
檎稼稼螳襴
譬 豕
油
Rnn keras
Rnn kerasRnn keras
Rnn keras
Park Seong Hyeon
油
Sequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural NetworksSequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural Networks
Hoon Heo
油
貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic
貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic
貂觜襾誤 襾語 Kevin Murphy Machine Learning Statistic
讌 譟
油
Seed2016 - 螳覩語 譯殊 (annotated)
Seed2016 - 螳覩語 譯殊 (annotated)Seed2016 - 螳覩語 譯殊 (annotated)
Seed2016 - 螳覩語 譯殊 (annotated)
Jooyung Han
油
Deep Learning for Chatbot (2/4)
Deep Learning for Chatbot (2/4)Deep Learning for Chatbot (2/4)
Deep Learning for Chatbot (2/4)
Jaemin Cho
油
Cnn 螳
Cnn 螳Cnn 螳
Cnn 螳
油
伎螻 一 5 | ル
伎螻 一 5 | ル伎螻 一 5 | ル
伎螻 一 5 | ル
蟾覯 | 覓伎誤磯伎朱
油
2.supervised learning(epoch#2)-3
2.supervised learning(epoch#2)-32.supervised learning(epoch#2)-3
2.supervised learning(epoch#2)-3
Haesun Park
油
Text2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to ActionText2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to Action
NAVER Engineering
油
07. PCA
07. PCA07. PCA
07. PCA
Jeonghun Yoon
油
Lecture 4: Neural Networks I
Lecture 4: Neural Networks ILecture 4: Neural Networks I
Lecture 4: Neural Networks I
Sang Jun Lee
油
Recurrent Neural Net 企螻 る
Recurrent Neural Net 企螻 るRecurrent Neural Net 企螻 る
Recurrent Neural Net 企螻 る
覦 蟾
油

More from KyeongUkJang (20)

Photo wake up - 3d character animation from a single photo
Photo wake up - 3d character animation from a single photoPhoto wake up - 3d character animation from a single photo
Photo wake up - 3d character animation from a single photo
KyeongUkJang
油
YOLO
YOLOYOLO
YOLO
KyeongUkJang
油
AlphagoZero
AlphagoZeroAlphagoZero
AlphagoZero
KyeongUkJang
油
GoogLenet
GoogLenetGoogLenet
GoogLenet
KyeongUkJang
油
GAN - Generative Adversarial Nets
GAN - Generative Adversarial NetsGAN - Generative Adversarial Nets
GAN - Generative Adversarial Nets
KyeongUkJang
油
Distilling the knowledge in a neural network
Distilling the knowledge in a neural networkDistilling the knowledge in a neural network
Distilling the knowledge in a neural network
KyeongUkJang
油
Latent Dirichlet Allocation
Latent Dirichlet AllocationLatent Dirichlet Allocation
Latent Dirichlet Allocation
KyeongUkJang
油
Gaussian Mixture Model
Gaussian Mixture ModelGaussian Mixture Model
Gaussian Mixture Model
KyeongUkJang
油
CNN for sentence classification
CNN for sentence classificationCNN for sentence classification
CNN for sentence classification
KyeongUkJang
油
Visualizing data using t-SNE
Visualizing data using t-SNEVisualizing data using t-SNE
Visualizing data using t-SNE
KyeongUkJang
油
Playing atari with deep reinforcement learning
Playing atari with deep reinforcement learningPlaying atari with deep reinforcement learning
Playing atari with deep reinforcement learning
KyeongUkJang
油
Chapter 20 - GAN
Chapter 20 - GANChapter 20 - GAN
Chapter 20 - GAN
KyeongUkJang
油
Chapter 20 - VAE
Chapter 20 - VAEChapter 20 - VAE
Chapter 20 - VAE
KyeongUkJang
油
Chapter 20 Deep generative models
Chapter 20 Deep generative modelsChapter 20 Deep generative models
Chapter 20 Deep generative models
KyeongUkJang
油
Natural Language Processing(NLP) - basic 2
Natural Language Processing(NLP) - basic 2Natural Language Processing(NLP) - basic 2
Natural Language Processing(NLP) - basic 2
KyeongUkJang
油
Natural Language Processing(NLP) - Basic
Natural Language Processing(NLP) - BasicNatural Language Processing(NLP) - Basic
Natural Language Processing(NLP) - Basic
KyeongUkJang
油
Chapter 17 monte carlo methods
Chapter 17 monte carlo methodsChapter 17 monte carlo methods
Chapter 17 monte carlo methods
KyeongUkJang
油
Chapter 16 structured probabilistic models for deep learning - 2
Chapter 16 structured probabilistic models for deep learning - 2Chapter 16 structured probabilistic models for deep learning - 2
Chapter 16 structured probabilistic models for deep learning - 2
KyeongUkJang
油
Chapter 16 structured probabilistic models for deep learning - 1
Chapter 16 structured probabilistic models for deep learning - 1Chapter 16 structured probabilistic models for deep learning - 1
Chapter 16 structured probabilistic models for deep learning - 1
KyeongUkJang
油
Chapter 15 Representation learning - 2
Chapter 15 Representation learning - 2Chapter 15 Representation learning - 2
Chapter 15 Representation learning - 2
KyeongUkJang
油
Photo wake up - 3d character animation from a single photo
Photo wake up - 3d character animation from a single photoPhoto wake up - 3d character animation from a single photo
Photo wake up - 3d character animation from a single photo
KyeongUkJang
油
YOLO
YOLOYOLO
YOLO
KyeongUkJang
油
AlphagoZero
AlphagoZeroAlphagoZero
AlphagoZero
KyeongUkJang
油
GoogLenet
GoogLenetGoogLenet
GoogLenet
KyeongUkJang
油
GAN - Generative Adversarial Nets
GAN - Generative Adversarial NetsGAN - Generative Adversarial Nets
GAN - Generative Adversarial Nets
KyeongUkJang
油
Distilling the knowledge in a neural network
Distilling the knowledge in a neural networkDistilling the knowledge in a neural network
Distilling the knowledge in a neural network
KyeongUkJang
油
Latent Dirichlet Allocation
Latent Dirichlet AllocationLatent Dirichlet Allocation
Latent Dirichlet Allocation
KyeongUkJang
油
Gaussian Mixture Model
Gaussian Mixture ModelGaussian Mixture Model
Gaussian Mixture Model
KyeongUkJang
油
CNN for sentence classification
CNN for sentence classificationCNN for sentence classification
CNN for sentence classification
KyeongUkJang
油
Visualizing data using t-SNE
Visualizing data using t-SNEVisualizing data using t-SNE
Visualizing data using t-SNE
KyeongUkJang
油
Playing atari with deep reinforcement learning
Playing atari with deep reinforcement learningPlaying atari with deep reinforcement learning
Playing atari with deep reinforcement learning
KyeongUkJang
油
Chapter 20 - GAN
Chapter 20 - GANChapter 20 - GAN
Chapter 20 - GAN
KyeongUkJang
油
Chapter 20 - VAE
Chapter 20 - VAEChapter 20 - VAE
Chapter 20 - VAE
KyeongUkJang
油
Chapter 20 Deep generative models
Chapter 20 Deep generative modelsChapter 20 Deep generative models
Chapter 20 Deep generative models
KyeongUkJang
油
Natural Language Processing(NLP) - basic 2
Natural Language Processing(NLP) - basic 2Natural Language Processing(NLP) - basic 2
Natural Language Processing(NLP) - basic 2
KyeongUkJang
油
Natural Language Processing(NLP) - Basic
Natural Language Processing(NLP) - BasicNatural Language Processing(NLP) - Basic
Natural Language Processing(NLP) - Basic
KyeongUkJang
油
Chapter 17 monte carlo methods
Chapter 17 monte carlo methodsChapter 17 monte carlo methods
Chapter 17 monte carlo methods
KyeongUkJang
油
Chapter 16 structured probabilistic models for deep learning - 2
Chapter 16 structured probabilistic models for deep learning - 2Chapter 16 structured probabilistic models for deep learning - 2
Chapter 16 structured probabilistic models for deep learning - 2
KyeongUkJang
油
Chapter 16 structured probabilistic models for deep learning - 1
Chapter 16 structured probabilistic models for deep learning - 1Chapter 16 structured probabilistic models for deep learning - 1
Chapter 16 structured probabilistic models for deep learning - 1
KyeongUkJang
油
Chapter 15 Representation learning - 2
Chapter 15 Representation learning - 2Chapter 15 Representation learning - 2
Chapter 15 Representation learning - 2
KyeongUkJang
油

Chapter 10 sequence modeling recurrent and recursive nets

  • 2. 磯Μ る RNN(Recurrent neural network) 伎 覦一 蟆企. 願姥襦 覓伎 讌襯 螳螻 企 朱 企 蟲譟郁 蟆朱讌 覲伎.
  • 3. RNN 磯 覿 一危郁 谿朱 蟇磯 蟯螻螳 譴 蟆曙一 RNN 覦.
  • 4. 蠏碁る RNN 覘蠍碁 襷 螳 企 蟆手?? 讀 RNN 蟯螻螳 譴 一危一 伎 ル 蟆. 蟆暑(RNN) 螻 一危一 螳 螳 襴 磯 覲 一危磯ゼ 牛蠍 ル 覈碁, 蠍一 (t)螻 れ (t+1) ろ語襯 郁屋 蟲燕 瑚概蟆暑
  • 5. - 一危一 譴讌 . - れ 襷 蠍一ヾ DNN 螳 襯 螻ろ讌 螻 譯殊伎 一危磯 螳讌螻 . - 一危郁 覺れ 一危一語 蠍一牛 れ . - 蠏語朱襦 る襷 DNN, NN 蟆曙, 一危磯ゼ ロ覃 一一 レ元 豸(hidden layers)襯 蟇一 豢リ讌 谿蠏殊姶蠏 讌. 螻殊 一危磯 覈 碁襯 覯 讌螳蟆 . RNN螻 NN 谿伎?? DNN
  • 6. ｼ豺蠍 RNN螻 NN 谿伎?? RNN 蟲譟 : 伎 豸旧 螳 れ 豸旧朱 . 讀 蠍一 RNN(Recurrent Neural Network) 企 覿. 覦覃伎 RNN 讌蠍 ル 一危一 螻手碓 ル 一危磯ゼ 螻ろ蟆 . 蠍磯蓋 蟲譟 れ螻 螳.
  • 7. れ元 蟆暑 れ螻 螳 蟲譟磯ゼ 覲伎譴.
  • 8. RNN 蟲譟一 蟆曙 朱誤磯 5螳螳 譟伎. 螳譴豺 3螳 1. レ元螻 豸旧 螳譴豺 モ 2. 豸糾骸 豸旧 螳譴豺 3. 豸糾骸 豢レ元 螳譴豺 Bias 2螳 1. レ元 豸旧朱 願 伎朱 2. 豸旧 豢レ元朱 願 伎朱
  • 9. RNN 蠍磯蓋 螻 レ元 : 豸旧 : モ + ≠1 + = 豸旧 豢 : = 豢レ元 : + = 豢レ元 豢 : ( ) , 燕
  • 10. 蠍磯蓋 RNN蟲譟一 伎 覲願鍵 レ元 : 豸旧 : モ + ≠1 + = 豸旧 豢 : = 豢レ元 : + = 豢レ元 豢 : = , 燕
  • 11. 蠍 ロ伎 Hello朱 企ゼ 豸″ 豌 h[1,0,0,0]企朱 レ朱 e[0,1,0,0]企朱 豢レ 豸″ 覦レ朱 旧 讌. 蠏 れ l[0,0,1,0]螻 o[0,0,0,1] 伎 襷谿螳讌襦 旧 讌.
  • 12. 企 RNN 轟 螳讌螳 , 襷り覲襯 螻旧る 蟆. 讀 リ朱 ≠1 ｋ, +4襯 ｋ 螳譴豺 モ 螳 蟆 . 襷谿螳讌襦 ′ +1襦 願 , +17 +18襦 願 , 螳 襯 . 企 るジ 襷り覲 襷谿螳讌企. 企 轟朱 誤 旧 觜襯願 讀螳讌襷, れ 豕襦 襷り覲襯 覦蟆 螻殊 伎 豕螳 れ 企れ る . RNN 轟
  • 13. 覦 RNN 伎 覲企れ RNN豌企 譬 焔レ 蠍磯 讌襷, 危 覲企ゼ 譬 蟆郁骸襯 蠍磯 . Ex) 碁ジ _____ も 朱 覓語レ 螳企 觜豺語 れ願 企ゼ 豸″ , 碁ジ, 企朱 覲企 蟲襴 豸″ 讌襷, 危 覲伎 も襯 牛伎 覲企 蟆 蟲襴企朱 企ゼ 豸″ .
  • 14. 覦 RNN 蟲譟 豸旧 危 覲企覿 レ 覦 蟆 覲 .
  • 15. RNN 1. 螻蠍 & 豢. ) 企語襯 ロ伎 企語 る 覓語レ朱 豢ロ 企語 貂′ 2. & 螻蠍 豢. ) 覓語レ ロ伎 蠍覿 襯 豢ロ 螳 覿蠍 3. & 豢. ) 企ゼ 蟲朱 覯 覯蠍
  • 16. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory The clouds Are in The ? ? 覘蟾?
  • 17. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory "the clouds are in the 朱 リ 覦螻 襷讌襷 企ゼ 豸″伎 る, 覲危 磯Μ 伎 覓碁Д(覲) 讌 . 覈蟆 れ ル 企 sky螳 襯 . 企 蟆曙一 , 螻給 一危一 豸″伎 覲伎 豺 谿(Gap)螳 讌 る, RNN 螻手碓 一 磯ゼ 蠍磯朱 所 牛 . The clouds Are in The ?
  • 18. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory 2. ? 襯 襷豢蠍 伎 覓碁Д 危危 も 朱蟾 れ器襯 谿渚蟆 蟆蟲! 朱 伎 . 3. 讌襷 リ 谿朱 れ伎り鍵 覓語 France朱 伎 れ 襷豢一 ? 蟇磯Μ螳 覃伎覃 RNN 覲伎 覓碁Д 郁屋蠍郁 れ伎. grew up France I speak fluent ? (I grew up in France ... I speak fluent French) 1. 蠏碁る ル 覓語レ 蠍語伎り 企慨 I
  • 19. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory grew up France I speak fluent ? (I grew up in France ... I speak fluent French) 蟇磯Μ螳 覃伎覃 襷豢蠍郁 れ?
  • 20. RNN 襷レ . 蟇煙蟇磯Μ螳 . 蠍一ヾ RNN 螳 れ 螳 螳 朱誤磯ゼ 螻旧螻 . れ 郁屋 螻 Recurrent螻 蟆企蟾(蠏企.) Long Short Term Memory
  • 21. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory 磯殊 螳譴豺()螳 1覲企 譟郁企朱 => 伎 蟇磯Μ螳 覃伎. = 覦 覯 螻燕 => 0 螳蟾 覃誤伎 1覲企 譟郁企朱 => 伎 蟇磯Μ螳 覃伎. = 覦 覯 螻燕 => 覓 貉れ 覦. 襷豢蠍 企糾, 豕 企給. (蟲 : 覿螳ロ 蟇 旧 譯 る 蟇碁Π.) 蠍一ヾ RNN 螳 れ 螳 螳 朱誤一企. れ 郁屋 螻 Recurrent螻 蟆企蟾(蠏企.)
  • 22. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory 豌伎語 れ願 螳 螻 覦蠖譯朱 蟆 => 螳 譯殊 襷螻 control unit 譯殊! 蠏碁 煙 : Long Short Term Memory network(LSTM) 誤 覯 企慨
  • 23. RNN 襷レ . 蟇煙蟇磯Μ螳 . Long Short Term Memory network 襷覃 LSTM襷螻 るジ 覦覯朱 願屋 蠍磯 . 10.8 覦 覦 蟆暑 10.9 豢 覦 れ 螳 豢 But 螳 10.10 LSTM 伎手鍵覃伎 豈 磯 , れ 一企 螳 螻殊 谿 覈 LSTM企. 手 覦給. 磯殊 10.8螻 10.9襯 牛螻 10.10 LSTM朱 願 蟆 覦.
  • 24. Long Short Term Memory 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 朱 讌 襯 牛 蟆)
  • 25. Long Short Term Memory 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎. 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 朱 讌 襯 牛 蟆)
  • 26. Long Short Term Memory 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 襯 牛 蟆) 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎. forget gate : - 螻手碓 覲企ゼ 蠍謂 蟆危 - 螳 覯 0 ~ 1 - 0 => 伎 覲企 . - 1 => 伎 覲企ゼ 蠍一牛.
  • 27. Long Short Term Memory 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 襯 牛 蟆) 蠍一ヾ Rnn豌 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎. input gate : - 覲企ゼ 蠍一牛蠍謂 蟆危 - 螳 覯 0 ~ 1 - 0 => 覲企 . - 1 => 覲企ゼ 蠍一牛. => 襦 襷 cell 螳
  • 28. Long Short Term Memory - 襯 sigmoid襦 蠍 蠍 覓語 0~1 伎 螳 豢 ＄ * 蠍一ヾ RNN - リ朱 伎 豢リ ≠1, ヰ°ゼ 覦. レ元 : ヰ 豸旧 : モ ヰ + ≠1 + = 豸旧 豢 : = 豢レ元 : + = 豢レ元 豢 : = , 燕
  • 29. Long Short Term Memory i ＄ (襦 襷 cell 螳) 襷 tanh襯 磯? 1. Gateれ 一危碁ゼ 蟆企 0~1 螳 豢ロ伎 伎 蠏碁企 2. tanh螳 譬り .
  • 30. Long Short Term Memory O＄ ＄ - C(cell) 襷 - C 蠍一/襷螳 襯 牛 蟆 - C 豕譬 豢リ , C襯 伎伎 豕譬 豸旧 豢リ 蟆一. 旧 : h(hidden unit) 襷螻 C(cell)襦 (C 蠍一/襷螳 襯 牛 蟆) 螻手碓 覲企ゼ 覦る(駒≠1) 企ゼ 螳螻 螻燕蠍 蠍 一一 牛伎 螻手碓襯 蟇磯 覲企ゼ 蠍一牛伎 駒′ 襷れ伎.
  • 31. 讌蠍蟾讌 LSTM 蟲ロ 覯 LSTM 覈 LSTM 狩 蟲譟磯 x (蠏碁觜結訣蠍磯 伎) Long Short Term Memory 襴
  • 32. 讌蠍蟾讌 LSTM 蟲ロ 覯 LSTM 覈 LSTM 狩 蟲譟磯 x (蠏碁觜結訣蠍磯 伎) Long Short Term Memory
  • 33. Gate 豌伎 伎 螻 Cell ｋ. Long Short Term Memory Gers & Schmidhuber (2000), is adding peephole connections.
  • 34. Input gate 1 伎 語一 襷 覿譟燕 覿覿 襦 語一朱 豈企. Make sense Long Short Term Memory coupled forget and input gates
  • 35. Long Short Term Memory Gated Recurrent Unit(GRU) - 螳 豕(2014) - LSTM レ 讌覃伎 螻磯概′煙 豢 蟲譟 : update 蟆危~ input gate : reset 蟆危~ forget gate : (t) 蠍一牛企 襷 覲 : れ (state)襦 一危
  • 36. 覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive recurrent neural network - リ 襦 覦 谿朱 豌襴 ろ語 蟲譟 - リ 譴螳 蟇企郁碓 覿覿 螻 煙レ襦 蠏碁襦 豌襴 蟲譟 - 襷讌襷 碁(2.5, 3.8) 伎蟾讌 覈 襷ル(the, country, of, my)螻 蟷 リ(birth) 覲願 覈 覦.
  • 37. 覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive convolutional neural network - リ 旧 覈 覦る Recurrent Neural Networks 谿 企 - 讌襷 リ (the, country) 覲企 Recurrent Neural Networks 襴 CNN 蠏碁殊 覲企 螳 伎(the country, country of, of my) 覯 覿 - 覯 譯殊 螻給れ, 磯朱 豺蟲螳 2螳 伎 覲企ゼ 豢伎 覲企ゼ 企. 蠏 朱 危危覃 .
  • 38. 覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive recursive neural network - 讌襷 CNN 覈 覲企ゼ 旧 覦 觜 RNN 朱 覲企 ろ る 谿 - the country of my birth 覦 蟲譟 - CNN 覦豌 the country, country of, of my 企蟆 覈 覿 螳 - RNN 企 語伎 螻豸旧 煙 ろ語 蟲譟一 蠏 谿 覈- 一 豌襴 襷 - Recursive Neural Networks(RNN) リ朱 譯殊伎 覈 螳 企ゼ 覓苦 覿る 伎CNN螻 convolutional neural network
  • 39. 覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive - Recurrent Neural Networks Recursive Neural Networks 轟 貅伎 - 襷 Recursive Neural Networks螳 覈 覲企ゼ 襦 觜讌 覦り 覃 殊 蠏碁 螳 蟲譟
  • 40. 企ゼ 螳 伎 覲企 覲語朱 Recurrent Neural Networks 螳 覿襦 襭 谿城り 蟆 る recurrent vs convolutional vs recursive 讌 蠍..伎れ 覦 覯
  • 41. 覿襦 襭 谿城り 蟆 る convolutional + recursive 蠍一ヾ cnn -> 讌 れ伎る => 蟆 覓伎瑚? 覿襯
  • 42. 覿襦 襭 谿城り 蟆 る convolutional + recursive 螻 一危 語 螳 一危一 label 蟆 一 label 豸′ 朱 一危 語螻 るゴ蟆 螻 企語 一危磯ゼ 語螻 矩. CNN + RNN ! => CRNN
  • 43. 覿襦 襭 谿城り 蟆 る convolutional + recursive 覓碁Д 磯ジ 蠍 , 覓語 語 覲 覓碁Д 語 焔レ 覲伎碁り
  • 44. !

Editor's Notes

  • #25: Hidden unit企朱 蟆 伎 cell企朱 蟆 . 螻燕蠍磯 蠍郁 覲伎伎? 願 覘覃 蠍一ヾ 蟆, Ct-1蟆企 Ct襯 伎 蟇一 蟇磯 蟇磯 伎朱 蟇 => long term memory 蟆伎. 蠏瑚姥 螳 ? Cell .
  • #27: Hidden unit企朱 蟆 伎 cell企朱 蟆 . 螻燕蠍磯 蠍郁 覲伎伎? 願 覘覃 蠍一ヾ 蟆, Ct-1蟆企 Ct襯 伎 蟇一 蟇磯 蟇磯 伎 long term memory襯 襷. 蠍 螳 蠍一牛 覃覈襴襯 襷. 蠏瑚姥 螳 ? Cell .
  • #28: Output gates 朱 覦朱 豢讌 企螳 0 1 蟆郁骸朱 蠍一ヾ rnn 觜伎 cell企朱 蟆 襷 cell 襦燕 覃覈襴襯 蠍一牛 t-1螻 t襯 譟壱伎 output 襷. 蠍一牛讌 襷讌 一危一 伎 給螻 給る 蟇 gate螳 給.
  • #29: 誤 x(t) 伎 覃覈襴 h(t-1) W[x,h]+b = W_x*x + W_h*h + b 襯 螳牛 蟆.鏤 Ht => short term memory 蠏碁蠏 螳 蟆郁骸 螳企手 覲企 る
  • #30: Tanh 襦 ろ危語 伎 襦 覲 螳 襷れ企碁り 覺 蟯谿 蟆 螳.
  • #38: 覯譯朱 rgb貊 企 the country of my birth朱 一危謂 蟲譟磯ゼ 覲願 一危磯 覓 蟆曙一 襷蠍
AboutCopyright
息 2025 際際滷