際際滷

際際滷Share a Scribd company logo

20170227 ????? ??_???

Kim Sungdong
Kim Sungdong

Building a Chatbot with Natural Language Interface using Python

Software
1 of 49
Downloaded 836 times
1
2
3
Most read
4
5
Most read
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
????? ?? ??? (With Natural Language Interface) 2017.02.27 ??? : ???
?? : ???(@DSKSD) ??? : tjdehd1222@gmail.com ??? ??/?????? (????) ??? ?????? ??? ??????:) Hello !!
?? 1.Introduction 2.Main Components of Chatbot 3.Deep Learning for Chatbot 4.Conclusion
Introduction ＾Facebook's future is in Chatbot￣ 2016.4, Mark Zuckerberg Inspiration C ??? ?? Lock-In effect(?? ???) Automation(?? ??) ´
Introduction Inspiration C ???? ?? ??? ??? ??? ??.(??? ??? ???.) ????? ???? ??? ? ?? ??. (Dialogue Design ?? ???) ?? ???? ??? ?? ??? (??? ???? ?????..!) ?? ????? ?? Challenging? ??!! (??? Pioneer? ? ??..?) ??(??)? ??? ??? ?? ??? ???? ???? ??? ??? ?? ?? (???? ??...)
Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´)
Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process * ?? 3?? ??? ???? ?? ???? ???? ??
Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process ???(??)???? ??? ????? ?? ??? ???/???(#??) ??(?? ?? ? ??) ???(Natural Language Understanding) * ?? 3?? ??? ???? ?? ???? ???? ?? ?? ??
Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process ?? Closed Domain(Goal based) Open Domain(General Bot) ??? ??? ????? ?? Rule based Retrieval Inference * ?? 3?? ??? ???? ?? ???? ???? ?? ?? ??
Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process ??? ??? ??? ??? ??? ???? ?? ?? ??(Database Retrieval) #??? ???(Entity/Information ????) ???(Natural Language Generation) * ?? 3?? ??? ???? ?? ???? ???? ?? ?? ??
Introduction ??? ?? ???? ???? ?? ??? ??!! ?? Natural Language Interface ???? ??? ??? Inference based Answer Maker (Closed domain) ?? ??? ??? ????
Main Components of Chatbot ??? ????? ???? ?? ??? ?? ??? ??? ????. ???? ?? ?? ??(Intent) ??? ??? ??(Context) ???? ?? ? ??? ??(Entity) (??? ??(Inference)) ?? ?? ???? ...! * ? Components? ??? ?? ?? ?? ??..!!
Main Components of Chatbot Intent U: ?? B: ?????. U: ?? ?? ??? B: ???? U: ???, ???? B: ?? ?? ??? ?????. U: ??? B: ?????? ?????. 1. Greeting() 2. AskWeather() 2. AskWeather() ?? ??? Goal Bot(?? ???? ?)? ??? ?? ? Component?? ???? ~ ?? ?? Intent? ? 5??? ????. 3. Appreciate() 4. Bye() 5. Other()
Main Components of Chatbot Intent ?? ??? ?? : Keyword Matching def Greeting(sentence): # ?? Intent? ???? ?? ???? GREETING_KEYWORDS = ["??", "??", "??", "??", "??＾] # ?? ?? GREETING_RESPONSES = ["??", "??","??", "?????"] # ???? ???? ??? ??. for word in sentence: if word.lower() in GREETING_KEYWORDS: return random.choice(GREETING_RESPONSES) ??? ??? ???? ?? ??? ? ???, ?? ???? ???? ?? ??? Intent? ??? ? ??. ????? ??? ??? ?? ??. (Rule based? ??)
Main Components of Chatbot Intent ?? ?? ?? : Intent Classifier 1. Raw text? ????? ??? (konlpy, genism) 2. Intent Classifier ???? (sklearn) 3. Classifier??? ?? ?? Raw text Preprocessing Vectorize Classifier 1 2 3 4 5
Main Components of Chatbot Intent - Preprocessing ?? ?? ?? : Intent Classifier from konlpy.tag import Twitter tagger = Twitter() def Preprocessing(sentences): """raw_text? ????? ??? ???(??-><NUM>) """ result = [] for sent in sentences: processed_sent = [] pos_tagged = tagger.pos(sent) # ???? -> [('??', 'Noun'), ('??', 'KoreanParticle')] for token in pos_tagged: if token[0].isdigit(): token[0] = '<NUM>' processed_sent.append(token[0]+'/'+token[1]) result.append(processed_sent) return result
Main Components of Chatbot Intent - Vectorize ?? ?? ?? : Intent Classifier import gensim def Vectorize(sentences): Text? ??? Input?? ?? ?? ??? ????. ?? ?? ?? 嗤 1> One-hot Vector 2> Bag-of-words 3> TFIDF 4> Word2Vec(Glove) 5> Paragraph2Vec (Doc2Vec) (link) 6> RNN Encoder 7> CNN for text (link) 8> FastText (link) ´ ?? : ???? NLTK, Gensim? ??, ???, 2015 Pycon
Main Components of Chatbot Intent - Vectorize ?? ?? ?? : Intent Classifier from gensim.models import doc2vec # ???? ?? ?? doc_vectorizer = doc2vec.Doc2Vec.load('doc2vec.model') # trained model def Vectorize(processed_sent): return doc_vectorizer.infer_vector(processed_sent) U: ?? ?? ??? Classifier (Logistic Reg, SVM, Random Forest, NN) 2 (AskWeather)
Main Components of Chatbot Intent - Classifier ?? ?? ?? : Intent Classifier from sklearn.linear_model import LogisticRegression classifier = LogisticRegression(random_state=1234) # classifier.fit(train_x, train_y) # ???? ?? ?? intent = classifier.predict(vectorized_sent) U: ?? ?? ??? Classifier (Logistic Reg, SVM, Random Forest, NN) 2 (AskWeather)
Main Components of Chatbot Entity ??? ??? ?? ??? ????..!! ?? ??? ?? ???.. U: ?? ?? ??? B: ???? ???? ??? ?? ???? '??'??? '??'? ?? ??? ???? ?? ?? ?? (??)? ??? ????? ???? ??? ? ???. ??? ?? ??? '??'? ?? ? ?? ???? ???? ?? ???? ?? ?? ???? ??. (?? ??(Context) ?? x) 2 (AskWeather)
Main Components of Chatbot Entity ???? ??(Entity)? ????! (Named Entity Recognition) A named entity is a collection of rigidly designated chunks of text that refer to exactly one or multiple identical, real or abstract concept instances. Person, Location, Organization, Date, Time, ... Ex) ??/Date ?? ? ??? ?? ??/Location?? ???/Organization ???? ???. ??/Date ???/Location ?? ? ???! ??(Entity)??
Main Components of Chatbot Entity ???? ?? ??? ??? NER ???? ???? ???? ?. ??? NER ??? ???? ?? ?? ???? ????? ??? ????? ???´´! ??? ??? ??? ??? ??. ?? ?? ??? ??? ?? ?? ????. => ???? ????? ???? ????! # ?? ?? Load NER_dict for token in sent: if token in NER_dict: Check
Main Components of Chatbot Entity ?? ???? ??? ??? ???!!! ???? NER ??? ??? ???? ??.. Sequence Labeling Model (HMM, CRF, RNN, ...) for POS, NER Tagging ?1 ?2 ?3 ?1 ?2 ?3???(Observation Sequence) ?? ?? (Latent state) ?? ??(T) ?? ??(T) ?? ??(E) ?? ??(E) ?? ??(E) https://github.com/krikit/annie * ANNIE [2016 ?? ?? ?? ??? C ??? ?? ???] Hidden Markov Model[Kaist ??? ???]
Main Components of Chatbot Entity BIO ?? for NER Model ???? ??(????) ? ?, ???? ???? ??? ?? ??? ??. ? ? ??? ??? ??? Entity? ???? ??? ?? ????. Ex> <?? ??>/Person, <??? ??? ???>/Location ??? ??? ???? ?? ???? BIO ??? ????. B : Entity? ?? ??, I : B ?? ???? Entity ??, O : Other Ex> ??/B-PER ??/I-PER, ???/B-LOC ???/I-LOC ???/I-LOC
Main Components of Chatbot Entity Conditional Random Field for NER http://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf
Main Components of Chatbot Entity Conditional Random Field for NER import pycrfsuite # pip install python-crfsuite def word2features(sent, i): 1> ??? ??? -2~2? ??? 2> ??? ??? -2~2? POS ?? 3> ??? ??? 4> ???? ?? 5> ?? ?? 6> ??, ??? ?? 7> ???? ?? ? ?? ´ return features CRF? ??? ??? ?? ????(???? ???)
Main Components of Chatbot Entity Conditional Random Field for NER import pycrfsuite # pip install python-crfsuite # ???? ?? ?? tagger = pycrfsuite.Tagger() tagger.open('myner.crfsuite') # trained model Predict = tagger.tag(sent2features(example_sent)) # ?? ?? ?? ?? ? # B-DATE B-LOC O O O CRF? ??? ??? ?? ????(???? ???) cf. ??? ??? Classification? ??? Precision/Recall/F1-Score? ?? ?? ???. ???? ?? ?? F1-Score? 80 ??? ?? ?... 20???? ??? ????? ?? ??..?! ?? state-of-the-art? ??? ??? Bidirectional LSTM + CRF https://github.com/rockingdingo/deepnlp
Main Components of Chatbot Entity ??? ????? Entity?? ??? ? ??. ??? ??! U: ??/Date ?? ??? B: ???? AskWeather(LOC, DATE=today) U: ???, ??/Location?? AskWeather(LOC=Seoul, DATE=today) B: ?? ?? ??? ?????. ?? ? ??? ???? ??? directly Inference ?? ?? Intent? ???? ??? ??? Entity? ?? ???? API Call? ?? ???? ???? ??.(Slot filling) (??? ??? Process ??? Inference)
Main Components of Chatbot Context API Call? ?? ??? ??? ?? Context? ???? ??? ?? ?? ????...!! (?, ? ??? ?? ??? ?? ??? ???? ?? ??) U: ??/Date ?? ??? B: ???? AskWeather(LOC, DATE=today) U: ???, ??/Location?? AskWeather(LOC=Seoul, DATE=today) B: ?? ?? ??? ?????. U: ??/Date?? B: ?..? Other()
Main Components of Chatbot Context API Call? ?? ??? ??? ?? Context? ???? ??? ?? ?? ????...!! (?, ? ??? ?? ??? ?? ??? ???? ?? ??) EX> ???? Slot filling (???? ??) B> ?????. ??? ??????? U> ?? ?? 7?? 6??? ???? ???? ?? ???. B> ?????. ???? ?? ??? ???? ???? U> 1?? 2?? ???. B> ??? ??? ???? U> ? ???. B> ?????. ? ??? ??? ?????? U> ? ???. B> API CALL LOCATION : ?? PRICE_RANGE : 2?? NUM_OF_PEAPLE : 6 TYPE_OF_CUISINE : ???? TIME : ?? ?? 7? Entities
Main Components of Chatbot Context API Call? ?? ??? ??? ?? Context? ???? ??? ?? ?? ????...!! (?, ? ??? ?? ??? ?? ??? ???? ?? ??) Entities EX> ???? Slot filling (Context ?? ?? ??) B> ?????. ??? ??????? U> ?? ?? 7?? 6??? ???? ???? ?? ???. B> ?????. ???? ?? ??? ???? ???? U> 1?? 2?? ???. B> ???? ???????? U> ?? ?? 7?????.. B> ?????. ? ??? ??? ?????? U> ??? ??? ?????? B> ????. ?????. U> ;; LOCATION : NULL PRICE_RANGE : 2?? NUM_OF_PEAPLE : 6 TYPE_OF_CUISINE : ???? TIME : NULL
Main Components of Chatbot Context ?? ??? ??? ?? ?? ??? ????? Intent? Entity? ????? ???? ?. def weatherBotEngine(User,userSay): context = stack(User).pop() # ????, ?? ??? ??? ???? entities = tagger.tag(sent2features(example_sent)) entities.extend(context['entities']) sent_vector = Vectorize(Processing(userSay)) intent = intentClassifier.predict(sent_vector) pre_intent = context[｀intent｀] # ?? Answer making # ?? ??? ?? ~_~ return reply
Deep Learning for Chatbot ??? ?? ??? Goal bot? ??? ???? ??? ??? ????... ?? ??? ??? ????. ??? ?? ? ????? ????? End-to-End ??? ???? ??!! (??? ?? ? ?? ??? ? ??..) ???? ??, ????? ?? ??? ???? ??. (?? ?? ??? ?? ??) ? ???? ??? Directly Inference?? Deep Learning model, ?? ?? ??? ?? ? ???? Deep Learning? ???? ???? model ? ? ???? ??.. ??? Deep learning model? ? ???? Machine Translation? Seq2Seq ??? ?? ?? ??? ??? ??? ?? ??.
Deep Learning for Chatbot Sequence2Sequence model Encoder? Decoder?? ?? 2?? RNN(LSTM)?? ???? ??. Encoder : ??(sequence)? ???? ??? ? timestep ? ??? word(token)?? ?? ??? ??. ?? words? ???? ??? ???? feature vector? ????. * ? feature vector? ??? ?? ??? ??? ?? ???? ???? ????! ??? feature vector? input sequence? ??? ??? ??? ? ? ??? context or thought vector?? ???. Decoder : ???? thought vector? ???? ??? sequence? generate??. Vocab? target?? Softmax!
Deep Learning for Chatbot Sequence2Sequence model ??? ??? ??? : 1> ??? ?? ?? 2> ??? ??? ??? ???? ???.(Context) 3> ???? ??(????? '???', '????' ?? ?? ???? ??? ?? ???? ??) ?? : Goblin(???)? Tensorflow tutorial 04. Chatbot ??
Deep Learning for Chatbot Hierarchical Recurrent Encoder Decoder HRED? Seq2Seq2 ??? 2???? ??????. ??? word level? sequence ??? Dialogue level? sequence * Context hidden state : m?? ????? context?? ???? ??! ? ? ?+? ??? ?? (Context? ?? ??? ??? ?? ??) Context hidden state https://arxiv.org/abs/1507.04808
Deep Learning for Chatbot Hierarchical Recurrent Encoder Decoder ?? ??? ???, ?? I don't know? I'm sorry? ?? ???? ??? ?? ??. ?, ???? ??.. 1> data scarcity 2> token? ???? ??? 3> triple ??? ?? ??. {?1, ?2, ?3} https://arxiv.org/abs/1507.04808
Deep Learning for Chatbot Attention https://arxiv.org/abs/1507.04808 Seq2seq ??? ? ??? ? ??? ?? ??? ??? ??? vector(context)? ??? ?? ??? ???. ???? ?? ??? ??? ??? ??, ?? ???? ????. ??? ? ??? ? ??? ???. Attention ????? ???? ???? ?? ?? ?? ??? ? ??? ????? ???? ???? ???? ??
Deep Learning for Chatbot Dialogue System as Reinforcement Learning POMDP(Partially Observable Markov Decision Process)? Reinforcement learning? ??? Task(Goal) Oriented Bot ??? ??? ?? ??? ???? action? state space? ?? hand- written (?)? ???? ??????? ???? ?????..!! (not scalable) http://mi.eng.cam.ac.uk/~sjy/papers/ygtw13.pdf
Deep Learning for Chatbot Dialogue System as Reinforcement Learning ??? ?? ??? ?? ?? ??? ???? ??. State? Action? ??? ??? ???? Directly Inference?? ??? ??? ?? ???? Reward? ??? ???..! (?? ??? ?? ????? ??? ??? ?? ??) https://arxiv.org/abs/1601.04574
Deep Learning for Chatbot Hybrid Model ??? ???? ??? ???(RL) ??? ??? ???? ??? ???? ?? ?? ???? ?? https://arxiv.org/abs/1606.01269 https://arxiv.org/abs/1604.04562
Deep Learning for Chatbot Memory Network ?? RNN ?? ??(LSTM Cell ??)?? ?? ?? ? long term memory? ?? ?? ??? ?? ??. ?? ?? ??? ??? ???? ?? ?? ???. (Memorization) ??? ??? ???? ?? Memory Network? ?? ???? ??? ???? ?? ?? ?? ???? ?? ??? ???? ????. https://arxiv.org/abs/1503.08895?? : ???? MemN2N Tensorflow Implementation ??
Deep Learning for Chatbot Learning through Dialogue Interaction Memory Network? ???? ??? ??? ?? ?? ??? ??? ?? ???. (???? ??, ?? ?? ?? ?...) ?? ?? ????? ??? ?? ??? ?? ???? ?????? ??? ???? ? ??. ?? ?? ??? ??? ?? ?????? ????. https://arxiv.org/abs/1612.04936
Conclusion ?? ??? ???? ???? ?? ??? ?? ????... ?..(?????.) ???? ??? ??? ???? ????, ??? ??? Issue?? ???? ?.(??? ????..?) ??? ??? ?? ??? ???? ???? ?? Bot Framework? ????. ??, 1> ?? ??. 2> ???? ? ???.(?? ? ??? ???..) 3> ??? ????? ???
Conclusion ??? ????? - ??? ???? ??? ??? ??? ?? ?? ??!! ??? ????? ???? ?? ????? ?? start> NLP with Python [link] ?? ??? ???? ????. nltk spaCy numpy konlpy gensim python-crfsuite ??, ??? ??? ?? ??? ??? ???? ???? ? ??.(???) (feat. Tensorflow) https://github.com/datanada/Awesome-Korean-NLP
Conclusion What is good conversation? ??? Inference(??)?? ???? ? ??? ?? ?? ??? ?? ????? ?? ?? ??? ????. Ex> Perplexity, (Entity) F1-Score, BLEU, Task-complete, DSTC ?? ???? ?? https://research.fb.com/downloads/babi/http://camdial.org/~mh521/dstc/
Conclusion Future work ??? ??? End-to-End Model? ??? ??? ??. ?? ??? ?? ?? '???? ?? ?' ? ?? ???? ??? ???? ? ??, ???? ??? ???? ?? ?? ??? ?? ?? ?? ?? ???? ??. ????? ???? Reinforcement Learning(Reward? ???? ?, ?? ??? ??? ??? ???) With External Memory(Memory Network ?) (???? ???? ????)? ??? ? ?? ?? ?? ??? ??? ???.
Reference ??? ?? ????/???[????? ??? ???] http://hunkim.github.io/ml/ CS-224d : Deep learning for NLP[Richard socher] http://cs224d.stanford.edu/syllabus.html Chatbot Fundamentals An interactive guide to writing bots in Python https://apps.worldwritable.com/tutorials/chatbot/ Building AI Chat bot using Python 3 & TensorFlow[??? ?] https://speakerdeck.com/inureyes/building-ai-chat-bot-using-python-3-and-tensorflow Developing Korean Chatbot 101 [??? ?] /JaeminCho6/developing-korean-chatbot-101-71013451 Deeplearning for Chatbot [Denny Britz] http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/ ?????? ?? ???? [??? ??? ???] /deview/f2-14341235 * ?? ?????? ???? ??? ???????!
Thank you for listening https://github.com/DSKSD

More Related Content

What's hot (20)

PPTX
???? ??? ??? ??? ?? ?? ?? ???? Ppt ???
Minji Kang
?
PPTX
Python? Tensorflow? ??? AI Chatbot ?? ? ?? ??
Susang Kim
?
PDF
ドメインl嘖O 云鯉秘T
弥 材
?
PDF
永霞岳鞄看稼による\徴宝秘壇
寄 弌}
?
PDF
[261] ???????? ??????????? ???????????? ?????????
NAVER D2
?
PDF
???? ??? ?? ??? ?? 3??
Taekyung Han
?
PDF
???????????????_????????_???????_?????.pdf
Seung kyoo Park
?
PDF
[NDC 2018] Spark, Flintrock, Airflow ? ???? ????? ??? ??? ???? ??? ??? ??
Juhong Park
?
PDF
[2D4]Python??? ???_???
NAVER D2
?
PDF
PHP の GC の
y-uti
?
PDF
Massive service basic
DaeMyung Kang
?
PPTX
及45指elasticsearch茶氏 BERTモデルを旋喘した猟蛍
shinhiguchi
?
PPTX
NLPにおけるAttention゛Seq2Seq から BERTまで゛
Takuya Ono
?
PPTX
??? ?? ???? ?? #1 ???? ?? ???
Terry Cho
?
PPTX
? 15? ???(BOAZ) ???? ???? - [YouPlace ?] : ???? ???? ??? ??? ?? ? ?? ?? ??
BOAZ Bigdata
?
PDF
[221] ??????????? ???????? ????? ?????????? ?????? ????????
NAVER D2
?
PPTX
Azure Search 冱ZI尖vBC嬬 ? アナライザ`、碧クエリ`、看、& ランキング, etc
Yoichi Kawasaki
?
PDF
リクル`トテクノロジ`ズがZる 二Iにおける、仝AIディ`プラ`ニング々試喘のリアル
Recruit Technologies
?
PDF
皆敬温乙乙艶姻て?の温沿庄蝕kよもやま三
KEISUKE KONISHI
?
PPTX
REST API ??
Terry Cho
?
???? ??? ??? ??? ?? ?? ?? ???? Ppt ???
Minji Kang
?
Python? Tensorflow? ??? AI Chatbot ?? ? ?? ??
Susang Kim
?
ドメインl嘖O 云鯉秘T
弥 材
?
永霞岳鞄看稼による\徴宝秘壇
寄 弌}
?
[261] ???????? ??????????? ???????????? ?????????
NAVER D2
?
???? ??? ?? ??? ?? 3??
Taekyung Han
?
???????????????_????????_???????_?????.pdf
Seung kyoo Park
?
[NDC 2018] Spark, Flintrock, Airflow ? ???? ????? ??? ??? ???? ??? ??? ??
Juhong Park
?
[2D4]Python??? ???_???
NAVER D2
?
PHP の GC の
y-uti
?
Massive service basic
DaeMyung Kang
?
及45指elasticsearch茶氏 BERTモデルを旋喘した猟蛍
shinhiguchi
?
NLPにおけるAttention゛Seq2Seq から BERTまで゛
Takuya Ono
?
??? ?? ???? ?? #1 ???? ?? ???
Terry Cho
?
? 15? ???(BOAZ) ???? ???? - [YouPlace ?] : ???? ???? ??? ??? ?? ? ?? ?? ??
BOAZ Bigdata
?
[221] ??????????? ???????? ????? ?????????? ?????? ????????
NAVER D2
?
Azure Search 冱ZI尖vBC嬬 ? アナライザ`、碧クエリ`、看、& ランキング, etc
Yoichi Kawasaki
?
リクル`トテクノロジ`ズがZる 二Iにおける、仝AIディ`プラ`ニング々試喘のリアル
Recruit Technologies
?
皆敬温乙乙艶姻て?の温沿庄蝕kよもやま三
KEISUKE KONISHI
?
REST API ??
Terry Cho
?

Viewers also liked (6)

PDF
?????? ???????????
?? ?
?
PDF
Developing Korean Chatbot 101
Jaemin Cho
?
PDF
?? ??? ?? ??? ?? api
NAVER D2
?
PPTX
???????????? ???????????? ???????????????
Sangmo Kang
?
PPTX
????? ?????? ?????_???? ???? ??? Buying_???
?? ?
?
PDF
[????] Ai?? ?? ??? ??? ????? 20170213_f
Lee Hansub
?
?????? ???????????
?? ?
?
Developing Korean Chatbot 101
Jaemin Cho
?
?? ??? ?? ??? ?? api
NAVER D2
?
???????????? ???????????? ???????????????
Sangmo Kang
?
????? ?????? ?????_???? ???? ??? Buying_???
?? ?
?
[????] Ai?? ?? ??? ??? ????? 20170213_f
Lee Hansub
?
Ad

Similar to 20170227 ????? ??_??? (20)

PDF
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Platform
?
PDF
1?????????? ???????? ???????????? ??????????? ??????
??? Jerry Jeong
?
PDF
Response Quality & BlahBlahbot
? ??
?
PDF
IBM Watson Assistant ?? ?? ?? ?? ?? (2019.11.18)
KYOYOON JUNG
?
PDF
AI ?????? ? ? ????
Myuserable
?
PDF
??, ??? ????
deepseaswjh
?
PDF
Deep Learning for Chatbot (1/4)
Jaemin Cho
?
PDF
Learning from dialogue after deployment feed yourself, chatbot
? ??
?
PDF
Open domain dialogue Chatbot(??? ???)
NAVER Engineering
?
PDF
Clova ai-business-day-session-4
Clova Platform
?
PDF
[IT21 ??? ????] ????? ??? ?? ??????? - ???
NAVER Engineering
?
PDF
Ipa definition
ssuser9a50211
?
PDF
hcik 2017 ?????????; ?????? ??? ???
Seoul National University
?
PDF
????? ??? ChatGPT
Tae Young Lee
?
PDF
?? ??????? ?? ??? ! WCS ???
HyeonJeong Jo
?
PDF
Ibm bluemix cloud ?? chatbot ???
Shaun LEE
?
PPTX
1?? ???? ? ??(Chotbot) ???
Seunghyeon Kim
?
PDF
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
Amazon Web Services Korea
?
PPTX
[???, kosena] ??AI ????? ??
kosena
?
PDF
???? ?? '???' ???
Yejee Choi
?
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Platform
?
1?????????? ???????? ???????????? ??????????? ??????
??? Jerry Jeong
?
Response Quality & BlahBlahbot
? ??
?
IBM Watson Assistant ?? ?? ?? ?? ?? (2019.11.18)
KYOYOON JUNG
?
AI ?????? ? ? ????
Myuserable
?
??, ??? ????
deepseaswjh
?
Deep Learning for Chatbot (1/4)
Jaemin Cho
?
Learning from dialogue after deployment feed yourself, chatbot
? ??
?
Open domain dialogue Chatbot(??? ???)
NAVER Engineering
?
Clova ai-business-day-session-4
Clova Platform
?
[IT21 ??? ????] ????? ??? ?? ??????? - ???
NAVER Engineering
?
Ipa definition
ssuser9a50211
?
hcik 2017 ?????????; ?????? ??? ???
Seoul National University
?
????? ??? ChatGPT
Tae Young Lee
?
?? ??????? ?? ??? ! WCS ???
HyeonJeong Jo
?
Ibm bluemix cloud ?? chatbot ???
Shaun LEE
?
1?? ???? ? ??(Chotbot) ???
Seunghyeon Kim
?
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
Amazon Web Services Korea
?
[???, kosena] ??AI ????? ??
kosena
?
???? ?? '???' ???
Yejee Choi
?
Ad

20170227 ????? ??_???

  • 1. ????? ?? ??? (With Natural Language Interface) 2017.02.27 ??? : ???
  • 2. ?? : ???(@DSKSD) ??? : tjdehd1222@gmail.com ??? ??/?????? (????) ??? ?????? ??? ??????:) Hello !!
  • 3. ?? 1.Introduction 2.Main Components of Chatbot 3.Deep Learning for Chatbot 4.Conclusion
  • 4. Introduction ＾Facebook's future is in Chatbot￣ 2016.4, Mark Zuckerberg Inspiration C ??? ?? Lock-In effect(?? ???) Automation(?? ??) ´
  • 5. Introduction Inspiration C ???? ?? ??? ??? ??? ??.(??? ??? ???.) ????? ???? ??? ? ?? ??. (Dialogue Design ?? ???) ?? ???? ??? ?? ??? (??? ???? ?????..!) ?? ????? ?? Challenging? ??!! (??? Pioneer? ? ??..?) ??(??)? ??? ??? ?? ??? ???? ???? ??? ??? ?? ?? (???? ??...)
  • 6. Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´)
  • 7. Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process * ?? 3?? ??? ???? ?? ???? ???? ??
  • 8. Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process ???(??)???? ??? ????? ?? ??? ???/???(#??) ??(?? ?? ? ??) ???(Natural Language Understanding) * ?? 3?? ??? ???? ?? ???? ???? ?? ?? ??
  • 9. Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process ?? Closed Domain(Goal based) Open Domain(General Bot) ??? ??? ????? ?? Rule based Retrieval Inference * ?? 3?? ??? ???? ?? ???? ???? ?? ?? ??
  • 10. Introduction What is Chatbot? ?? = ????? ?? = ???? ??(Chat)? ?? ? ?? ??? ????(? ?? ??? ?????´) User InterfaceUser Interface Input Output Answer Maker Process ??? ??? ??? ??? ??? ???? ?? ?? ??(Database Retrieval) #??? ???(Entity/Information ????) ???(Natural Language Generation) * ?? 3?? ??? ???? ?? ???? ???? ?? ?? ??
  • 11. Introduction ??? ?? ???? ???? ?? ??? ??!! ?? Natural Language Interface ???? ??? ??? Inference based Answer Maker (Closed domain) ?? ??? ??? ????
  • 12. Main Components of Chatbot ??? ????? ???? ?? ??? ?? ??? ??? ????. ???? ?? ?? ??(Intent) ??? ??? ??(Context) ???? ?? ? ??? ??(Entity) (??? ??(Inference)) ?? ?? ???? ...! * ? Components? ??? ?? ?? ?? ??..!!
  • 13. Main Components of Chatbot Intent U: ?? B: ?????. U: ?? ?? ??? B: ???? U: ???, ???? B: ?? ?? ??? ?????. U: ??? B: ?????? ?????. 1. Greeting() 2. AskWeather() 2. AskWeather() ?? ??? Goal Bot(?? ???? ?)? ??? ?? ? Component?? ???? ~ ?? ?? Intent? ? 5??? ????. 3. Appreciate() 4. Bye() 5. Other()
  • 14. Main Components of Chatbot Intent ?? ??? ?? : Keyword Matching def Greeting(sentence): # ?? Intent? ???? ?? ???? GREETING_KEYWORDS = ["??", "??", "??", "??", "??＾] # ?? ?? GREETING_RESPONSES = ["??", "??","??", "?????"] # ???? ???? ??? ??. for word in sentence: if word.lower() in GREETING_KEYWORDS: return random.choice(GREETING_RESPONSES) ??? ??? ???? ?? ??? ? ???, ?? ???? ???? ?? ??? Intent? ??? ? ??. ????? ??? ??? ?? ??. (Rule based? ??)
  • 15. Main Components of Chatbot Intent ?? ?? ?? : Intent Classifier 1. Raw text? ????? ??? (konlpy, genism) 2. Intent Classifier ???? (sklearn) 3. Classifier??? ?? ?? Raw text Preprocessing Vectorize Classifier 1 2 3 4 5
  • 16. Main Components of Chatbot Intent - Preprocessing ?? ?? ?? : Intent Classifier from konlpy.tag import Twitter tagger = Twitter() def Preprocessing(sentences): """raw_text? ????? ??? ???(??-><NUM>) """ result = [] for sent in sentences: processed_sent = [] pos_tagged = tagger.pos(sent) # ???? -> [('??', 'Noun'), ('??', 'KoreanParticle')] for token in pos_tagged: if token[0].isdigit(): token[0] = '<NUM>' processed_sent.append(token[0]+'/'+token[1]) result.append(processed_sent) return result
  • 17. Main Components of Chatbot Intent - Vectorize ?? ?? ?? : Intent Classifier import gensim def Vectorize(sentences): Text? ??? Input?? ?? ?? ??? ????. ?? ?? ?? 嗤 1> One-hot Vector 2> Bag-of-words 3> TFIDF 4> Word2Vec(Glove) 5> Paragraph2Vec (Doc2Vec) (link) 6> RNN Encoder 7> CNN for text (link) 8> FastText (link) ´ ?? : ???? NLTK, Gensim? ??, ???, 2015 Pycon
  • 18. Main Components of Chatbot Intent - Vectorize ?? ?? ?? : Intent Classifier from gensim.models import doc2vec # ???? ?? ?? doc_vectorizer = doc2vec.Doc2Vec.load('doc2vec.model') # trained model def Vectorize(processed_sent): return doc_vectorizer.infer_vector(processed_sent) U: ?? ?? ??? Classifier (Logistic Reg, SVM, Random Forest, NN) 2 (AskWeather)
  • 19. Main Components of Chatbot Intent - Classifier ?? ?? ?? : Intent Classifier from sklearn.linear_model import LogisticRegression classifier = LogisticRegression(random_state=1234) # classifier.fit(train_x, train_y) # ???? ?? ?? intent = classifier.predict(vectorized_sent) U: ?? ?? ??? Classifier (Logistic Reg, SVM, Random Forest, NN) 2 (AskWeather)
  • 20. Main Components of Chatbot Entity ??? ??? ?? ??? ????..!! ?? ??? ?? ???.. U: ?? ?? ??? B: ???? ???? ??? ?? ???? '??'??? '??'? ?? ??? ???? ?? ?? ?? (??)? ??? ????? ???? ??? ? ???. ??? ?? ??? '??'? ?? ? ?? ???? ???? ?? ???? ?? ?? ???? ??. (?? ??(Context) ?? x) 2 (AskWeather)
  • 21. Main Components of Chatbot Entity ???? ??(Entity)? ????! (Named Entity Recognition) A named entity is a collection of rigidly designated chunks of text that refer to exactly one or multiple identical, real or abstract concept instances. Person, Location, Organization, Date, Time, ... Ex) ??/Date ?? ? ??? ?? ??/Location?? ???/Organization ???? ???. ??/Date ???/Location ?? ? ???! ??(Entity)??
  • 22. Main Components of Chatbot Entity ???? ?? ??? ??? NER ???? ???? ???? ?. ??? NER ??? ???? ?? ?? ???? ????? ??? ????? ???´´! ??? ??? ??? ??? ??. ?? ?? ??? ??? ?? ?? ????. => ???? ????? ???? ????! # ?? ?? Load NER_dict for token in sent: if token in NER_dict: Check
  • 23. Main Components of Chatbot Entity ?? ???? ??? ??? ???!!! ???? NER ??? ??? ???? ??.. Sequence Labeling Model (HMM, CRF, RNN, ...) for POS, NER Tagging ?1 ?2 ?3 ?1 ?2 ?3???(Observation Sequence) ?? ?? (Latent state) ?? ??(T) ?? ??(T) ?? ??(E) ?? ??(E) ?? ??(E) https://github.com/krikit/annie * ANNIE [2016 ?? ?? ?? ??? C ??? ?? ???] Hidden Markov Model[Kaist ??? ???]
  • 24. Main Components of Chatbot Entity BIO ?? for NER Model ???? ??(????) ? ?, ???? ???? ??? ?? ??? ??. ? ? ??? ??? ??? Entity? ???? ??? ?? ????. Ex> <?? ??>/Person, <??? ??? ???>/Location ??? ??? ???? ?? ???? BIO ??? ????. B : Entity? ?? ??, I : B ?? ???? Entity ??, O : Other Ex> ??/B-PER ??/I-PER, ???/B-LOC ???/I-LOC ???/I-LOC
  • 25. Main Components of Chatbot Entity Conditional Random Field for NER http://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf
  • 26. Main Components of Chatbot Entity Conditional Random Field for NER import pycrfsuite # pip install python-crfsuite def word2features(sent, i): 1> ??? ??? -2~2? ??? 2> ??? ??? -2~2? POS ?? 3> ??? ??? 4> ???? ?? 5> ?? ?? 6> ??, ??? ?? 7> ???? ?? ? ?? ´ return features CRF? ??? ??? ?? ????(???? ???)
  • 27. Main Components of Chatbot Entity Conditional Random Field for NER import pycrfsuite # pip install python-crfsuite # ???? ?? ?? tagger = pycrfsuite.Tagger() tagger.open('myner.crfsuite') # trained model Predict = tagger.tag(sent2features(example_sent)) # ?? ?? ?? ?? ? # B-DATE B-LOC O O O CRF? ??? ??? ?? ????(???? ???) cf. ??? ??? Classification? ??? Precision/Recall/F1-Score? ?? ?? ???. ???? ?? ?? F1-Score? 80 ??? ?? ?... 20???? ??? ????? ?? ??..?! ?? state-of-the-art? ??? ??? Bidirectional LSTM + CRF https://github.com/rockingdingo/deepnlp
  • 28. Main Components of Chatbot Entity ??? ????? Entity?? ??? ? ??. ??? ??! U: ??/Date ?? ??? B: ???? AskWeather(LOC, DATE=today) U: ???, ??/Location?? AskWeather(LOC=Seoul, DATE=today) B: ?? ?? ??? ?????. ?? ? ??? ???? ??? directly Inference ?? ?? Intent? ???? ??? ??? Entity? ?? ???? API Call? ?? ???? ???? ??.(Slot filling) (??? ??? Process ??? Inference)
  • 29. Main Components of Chatbot Context API Call? ?? ??? ??? ?? Context? ???? ??? ?? ?? ????...!! (?, ? ??? ?? ??? ?? ??? ???? ?? ??) U: ??/Date ?? ??? B: ???? AskWeather(LOC, DATE=today) U: ???, ??/Location?? AskWeather(LOC=Seoul, DATE=today) B: ?? ?? ??? ?????. U: ??/Date?? B: ?..? Other()
  • 30. Main Components of Chatbot Context API Call? ?? ??? ??? ?? Context? ???? ??? ?? ?? ????...!! (?, ? ??? ?? ??? ?? ??? ???? ?? ??) EX> ???? Slot filling (???? ??) B> ?????. ??? ??????? U> ?? ?? 7?? 6??? ???? ???? ?? ???. B> ?????. ???? ?? ??? ???? ???? U> 1?? 2?? ???. B> ??? ??? ???? U> ? ???. B> ?????. ? ??? ??? ?????? U> ? ???. B> API CALL LOCATION : ?? PRICE_RANGE : 2?? NUM_OF_PEAPLE : 6 TYPE_OF_CUISINE : ???? TIME : ?? ?? 7? Entities
  • 31. Main Components of Chatbot Context API Call? ?? ??? ??? ?? Context? ???? ??? ?? ?? ????...!! (?, ? ??? ?? ??? ?? ??? ???? ?? ??) Entities EX> ???? Slot filling (Context ?? ?? ??) B> ?????. ??? ??????? U> ?? ?? 7?? 6??? ???? ???? ?? ???. B> ?????. ???? ?? ??? ???? ???? U> 1?? 2?? ???. B> ???? ???????? U> ?? ?? 7?????.. B> ?????. ? ??? ??? ?????? U> ??? ??? ?????? B> ????. ?????. U> ;; LOCATION : NULL PRICE_RANGE : 2?? NUM_OF_PEAPLE : 6 TYPE_OF_CUISINE : ???? TIME : NULL
  • 32. Main Components of Chatbot Context ?? ??? ??? ?? ?? ??? ????? Intent? Entity? ????? ???? ?. def weatherBotEngine(User,userSay): context = stack(User).pop() # ????, ?? ??? ??? ???? entities = tagger.tag(sent2features(example_sent)) entities.extend(context['entities']) sent_vector = Vectorize(Processing(userSay)) intent = intentClassifier.predict(sent_vector) pre_intent = context[｀intent｀] # ?? Answer making # ?? ??? ?? ~_~ return reply
  • 33. Deep Learning for Chatbot ??? ?? ??? Goal bot? ??? ???? ??? ??? ????... ?? ??? ??? ????. ??? ?? ? ????? ????? End-to-End ??? ???? ??!! (??? ?? ? ?? ??? ? ??..) ???? ??, ????? ?? ??? ???? ??. (?? ?? ??? ?? ??) ? ???? ??? Directly Inference?? Deep Learning model, ?? ?? ??? ?? ? ???? Deep Learning? ???? ???? model ? ? ???? ??.. ??? Deep learning model? ? ???? Machine Translation? Seq2Seq ??? ?? ?? ??? ??? ??? ?? ??.
  • 34. Deep Learning for Chatbot Sequence2Sequence model Encoder? Decoder?? ?? 2?? RNN(LSTM)?? ???? ??. Encoder : ??(sequence)? ???? ??? ? timestep ? ??? word(token)?? ?? ??? ??. ?? words? ???? ??? ???? feature vector? ????. * ? feature vector? ??? ?? ??? ??? ?? ???? ???? ????! ??? feature vector? input sequence? ??? ??? ??? ? ? ??? context or thought vector?? ???. Decoder : ???? thought vector? ???? ??? sequence? generate??. Vocab? target?? Softmax!
  • 35. Deep Learning for Chatbot Sequence2Sequence model ??? ??? ??? : 1> ??? ?? ?? 2> ??? ??? ??? ???? ???.(Context) 3> ???? ??(????? '???', '????' ?? ?? ???? ??? ?? ???? ??) ?? : Goblin(???)? Tensorflow tutorial 04. Chatbot ??
  • 36. Deep Learning for Chatbot Hierarchical Recurrent Encoder Decoder HRED? Seq2Seq2 ??? 2???? ??????. ??? word level? sequence ??? Dialogue level? sequence * Context hidden state : m?? ????? context?? ???? ??! ? ? ?+? ??? ?? (Context? ?? ??? ??? ?? ??) Context hidden state https://arxiv.org/abs/1507.04808
  • 37. Deep Learning for Chatbot Hierarchical Recurrent Encoder Decoder ?? ??? ???, ?? I don't know? I'm sorry? ?? ???? ??? ?? ??. ?, ???? ??.. 1> data scarcity 2> token? ???? ??? 3> triple ??? ?? ??. {?1, ?2, ?3} https://arxiv.org/abs/1507.04808
  • 38. Deep Learning for Chatbot Attention https://arxiv.org/abs/1507.04808 Seq2seq ??? ? ??? ? ??? ?? ??? ??? ??? vector(context)? ??? ?? ??? ???. ???? ?? ??? ??? ??? ??, ?? ???? ????. ??? ? ??? ? ??? ???. Attention ????? ???? ???? ?? ?? ?? ??? ? ??? ????? ???? ???? ???? ??
  • 39. Deep Learning for Chatbot Dialogue System as Reinforcement Learning POMDP(Partially Observable Markov Decision Process)? Reinforcement learning? ??? Task(Goal) Oriented Bot ??? ??? ?? ??? ???? action? state space? ?? hand- written (?)? ???? ??????? ???? ?????..!! (not scalable) http://mi.eng.cam.ac.uk/~sjy/papers/ygtw13.pdf
  • 40. Deep Learning for Chatbot Dialogue System as Reinforcement Learning ??? ?? ??? ?? ?? ??? ???? ??. State? Action? ??? ??? ???? Directly Inference?? ??? ??? ?? ???? Reward? ??? ???..! (?? ??? ?? ????? ??? ??? ?? ??) https://arxiv.org/abs/1601.04574
  • 41. Deep Learning for Chatbot Hybrid Model ??? ???? ??? ???(RL) ??? ??? ???? ??? ???? ?? ?? ???? ?? https://arxiv.org/abs/1606.01269 https://arxiv.org/abs/1604.04562
  • 42. Deep Learning for Chatbot Memory Network ?? RNN ?? ??(LSTM Cell ??)?? ?? ?? ? long term memory? ?? ?? ??? ?? ??. ?? ?? ??? ??? ???? ?? ?? ???. (Memorization) ??? ??? ???? ?? Memory Network? ?? ???? ??? ???? ?? ?? ?? ???? ?? ??? ???? ????. https://arxiv.org/abs/1503.08895?? : ???? MemN2N Tensorflow Implementation ??
  • 43. Deep Learning for Chatbot Learning through Dialogue Interaction Memory Network? ???? ??? ??? ?? ?? ??? ??? ?? ???. (???? ??, ?? ?? ?? ?...) ?? ?? ????? ??? ?? ??? ?? ???? ?????? ??? ???? ? ??. ?? ?? ??? ??? ?? ?????? ????. https://arxiv.org/abs/1612.04936
  • 44. Conclusion ?? ??? ???? ???? ?? ??? ?? ????... ?..(?????.) ???? ??? ??? ???? ????, ??? ??? Issue?? ???? ?.(??? ????..?) ??? ??? ?? ??? ???? ???? ?? Bot Framework? ????. ??, 1> ?? ??. 2> ???? ? ???.(?? ? ??? ???..) 3> ??? ????? ???
  • 45. Conclusion ??? ????? - ??? ???? ??? ??? ??? ?? ?? ??!! ??? ????? ???? ?? ????? ?? start> NLP with Python [link] ?? ??? ???? ????. nltk spaCy numpy konlpy gensim python-crfsuite ??, ??? ??? ?? ??? ??? ???? ???? ? ??.(???) (feat. Tensorflow) https://github.com/datanada/Awesome-Korean-NLP
  • 46. Conclusion What is good conversation? ??? Inference(??)?? ???? ? ??? ?? ?? ??? ?? ????? ?? ?? ??? ????. Ex> Perplexity, (Entity) F1-Score, BLEU, Task-complete, DSTC ?? ???? ?? https://research.fb.com/downloads/babi/http://camdial.org/~mh521/dstc/
  • 47. Conclusion Future work ??? ??? End-to-End Model? ??? ??? ??. ?? ??? ?? ?? '???? ?? ?' ? ?? ???? ??? ???? ? ??, ???? ??? ???? ?? ?? ??? ?? ?? ?? ?? ???? ??. ????? ???? Reinforcement Learning(Reward? ???? ?, ?? ??? ??? ??? ???) With External Memory(Memory Network ?) (???? ???? ????)? ??? ? ?? ?? ?? ??? ??? ???.
  • 48. Reference ??? ?? ????/???[????? ??? ???] http://hunkim.github.io/ml/ CS-224d : Deep learning for NLP[Richard socher] http://cs224d.stanford.edu/syllabus.html Chatbot Fundamentals An interactive guide to writing bots in Python https://apps.worldwritable.com/tutorials/chatbot/ Building AI Chat bot using Python 3 & TensorFlow[??? ?] https://speakerdeck.com/inureyes/building-ai-chat-bot-using-python-3-and-tensorflow Developing Korean Chatbot 101 [??? ?] /JaeminCho6/developing-korean-chatbot-101-71013451 Deeplearning for Chatbot [Denny Britz] http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/ ?????? ?? ???? [??? ??? ???] /deview/f2-14341235 * ?? ?????? ???? ??? ???????!
  • 49. Thank you for listening https://github.com/DSKSD
About
? 2025 際際滷