ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Python? Tensorflow? ??? AI Chatbot ?? ? ?? ??

Download as PPTX, PDF
S
Susang Kim

?? AI Chatbot ?? Chatbot Ecosystem Closed vs Open Domain Rule Based vs AI Chat IF Flow and Story Slot AI??? ??? ?? Data ?? ?? Data? ??? ? / Train? ?? Word Representation Data? ?? / Data Augmentation(Intent, NER) ????? ?? AI ?? ?? Intent (Char-CNN) / QnA (Seq2Seq) Named Entity Recognition (Bi-LSTM CRF) / Ontology (Graph DB) Chatbot Service? ?? Architecture ?? Chatbot Architecture NLP Architecture Web Service Architecture Bot builder / Chatbot API Test Codes for Chatbot ???? ???? ??? ?? Tips Ensemble and voting / Trigger / Synonym(N-Gram) Tone Generator / Parallel processing / Response Speed ??? [?? ??] Text Augmentation / Slot Bot / QA Bot / Graph DB / Response Generator

1 of 59
Downloaded 1,310 times
WRITTEN BY SuSang Kim healess1@gmail.com Python? Tensorflow? ??? AI Chatbot ?? ? ?? ??
1. ?????? https://github.com/TensorMSA/skp_edu_docker 2. ?????? - jupyter (/chap13_chatbot_lecture) git clone https://github.com/TensorMSA/tensormsa_jupyter.git 3. Python 3.5 / Tensorflow 1.2
I am Susang Kim as a developer ?Chatbot Develover - Released in POSCO (Find people using by NLP/AI) - Deep Learning MSA (ML,DNN, CNN, RNN) ?Agile Develover (Experienced in Pivotal Labs) - TDD, CI, Pair programming, User Story ?iOS Develover (Ranked App store in 100th - 2011 Korea) ?Front-End Developer (React, D3, Typescript and ES6) ?POSCO MES ... (working at POSCO ICT for 10 year)
Contents 1. ?? 2. AI Chatbot ?? Chatbot Ecosystem Closed vs Open Domain Rule Based vs AI Chat IF Flow and Story Slot 3. AI??? ??? ?? Data ?? ?? Data? ??? ? / Train? ?? Word Representation Data? ?? / Data Augmentation(Intent, NER) 4. ????? ?? AI ?? ?? Intent (Char-CNN) / QnA (Seq2Seq) Named Entity Recognition (Bi-LSTM CRF) / Ontology (Graph DB)
Contents 6. Chatbot Service? ?? Architecture ?? Chatbot Architecture NLP Architecture Web Service Architecture Bot builder / Chatbot API Test Codes for Chatbot 7. ???? ???? ??? ?? Tips Ensemble and voting / Trigger / Synonym(N-Gram) Tone Generator / Parallel processing / Response Speed 8. ??? [?? ??] Text Augmentation / Char-CNN / NER /Slot Bot / QA Bot / Graph DB / Response Generator
??
Facebook AI shut down after creating their own language ?? https://arxiv.org/abs/1706.05125
Chatbot? ????? ??? ? ?? ??? ?? (NLP, AI, F/W, Text Mining and ??? ?? skill) ? Deep Learning? ???? ???? ?? ??? ?? - ?? Computing?? ?? ???? ?? (Text ??) ? ??? ??(Micro Data??? ?? Biz dependency? ???) - ???(CNN)?? ??Data(DNN)??? Data??? ?? ???? ?? (??? ?????? ?? ??? ??? ????) ? ????? ?? (API??? ?????? ?? Smart Management) - Intent? Slot? ???? ?? ???? ???? ? ?? ????? ?? ???? (??? ??? ???????) - ???? ??? ??? ????? Text algorithm? ?? ???? ?? ?? ? Bot Service? ??? ????, ???? ???, Customize ??
Chatbot?? AI (??,??) ??? (??????) ????? (Data??-Python) Bot F/W (Story/Slot??) Architecture (????) Text Mining (Data??) Chatbot Chatbot ??? ???? ?? ??? ??? ?? ??
AI Chatbot ??
[?? Deview 2016 - https://deview.kr/2016/schedule#session/176] ?? ? Chatbot? ????? ???? UX ??? ?? ?? ??? ?? ?? ?? App ?? ?? ?? ??? ???? ?? ?? ?? Feedback ???? ??
Chatbot Ecosystem [?? Digital Landscapes - https://www.linkedin.com/pulse/curated-list-digital-landscapes-marek-rakowski] Chatbot ???? ?? ???? ?? ??? ????? ??? ???NLP? Bot Builder (Slot?? ?? / QA? ??)
??? Chatbot Platform? ????? ?? API.AI? ???? ?? ??? https://calyfactory.github.io/api.ai-chatbot/ ?? ???? ??? ????? ?? ?? ? ?? ???? Data? ???!!! api.ai? ???? ??? ?????? ??? ????? ??? ?
Closed Domain vs Open Domain Rule Based General (abstract) Open Closed Retrieval (accuracy) Impossible Strong AI Weak AI level of difficulty ?? Biz ????? ???? ???? ???? ?? Biz? ???? ??
Rule Based vs AI Computer Input Program Output Rule ??, ??, ?? ???? ??? rule? ?????? - ???? ???? ?? ??? ? ???? Computer Input Output Program AI ???? Data??? ??? ?? ? ?? ??? ?? ? ?? - ??? Data?? ????(W2V,GloVe) intent = ??? ???? ??? ??? => Intent : ?? ?? ?? ??? NER = ??? ???? ??? ??? => B-Loc O O B-Name O ??? ??? ?? ? ??? ?? ??? ?? ??? ??? ??? ??? ? ??? Data? ?? ?? ??? ??(????) If (loc = ?? and comp = ???ICT) person = ??? elif (loc = ?? and comp = SK Planet) person = ??? else person = ???
Chatbot Interface Flow NLP Context Analyzer Decision Maker ??? ???ICT? ???? Intent : ???? Entity : ?? = ?? ???ICT Service Manager Response Generator ??=null ??=null ???? Slot ??(Knowlodge Base) Entity : ??:Null, ??:null ???? ??? ???????. ???? Slot ?? ?? ??? ?????? ?? ??? ??? (Tone Gen) Slot OK
Story slot? ?? (Frame-based DM) ?? ???? ?? Pizza Slot Size Type Side menu ?? ?? ?? ?? ?? Bot? ??? ?? 1) ?? ???? ?????? 2) ?? ??? ?????? 3) ??? ??? ??????? ??? ?? - ???? ??? ?? ???? ???????? NER?? ? Slot ?? Pizza Slot Size Large Type Pepperoni Side menu cola ??? ?? (Slot API Call) ??? ?? Slot? ???? ?? ??
1. ?? ?? ???? 2. ??? -> ?? ?? NER 3. ???? -> ?? Entity?? -> MacBook Pro API Call 4. ???? ?? 5. ?? ??? ??? ?? Slot ?? 6. ??? ?? ?? ??? ?? Slot? ??? ? ?? ??? ?????? ??? ???? ?? ?? ?? ? ?? (?? Frame???? ??? ? ???¡­) ex) ¡°?? ???¡± ??? Slot? ?? ??? http://www.11st.co.kr/toc/bridge.tmall?method=chatPage Slot Trigger API
AI??? ??? ?? Data ????
????? Biz? ?? Text? ???? Deep Learning? ???? ???? ??? Text Tagging? ??? ?? ?? Data? ???? ??? Corpus? ????? ?? ???? ???? ???? Biz ??? ?? ????(???) - Corpus (annotation) ?????(2007 ) https://ithub.korean.go.kr/user/main.do - ??21 (2001~2014) ????? http://corpus.korea.ac.kr/ - Web Crawling or down (Wiki, Namu Wiki) - Domain Specific? ??? Text Data? ????? ??? ??? ?? ?? ?????? (??? , ????) ¡ù ????? ??? ??? ???? ?? ??? ????? Data? ??? ????
Word Represention? ?? (???? ? ???? ??) - One Hot? ??? ?? ??? ???? Train ? ??? (Scope? ????-Sparse) - Word ?? Embedding ? ??? ? ??? (But Sparse) / W2V (???) - GloVe? ??? ?? ????? ?? (Global Vector : ???-???) - Char ?? Embedding ? ??? ?? ??? ?? (Vector? ????? ????) - ??? ??? ?? Char ?? Embedding? ?? ?? ???? ?? ??? ?? Train? ?? Word Representation 15 ???? ??? ?? ??? ?? ? ???? ??? ?? ??.pdf
Train Vector? ?? ? Feature? ???? Cleansing -> Feature Engineering -> Train (??? ???? ??, ?? ?? ?? ?? - Tagging) ??? ??? ???? ??? ????? ??? ????Train Cost? ??? ??? ??? ??) ??? ??? ??? ?? (Dense Respresention-SVD) abcd~z, 0~9, ?, !, (,),¡¯,¡¯,??? ? 70?? ?????? ??? ????? ??? .lower()? ?????? ?? ?? ??? ???? Data? ??
Data Augmentation for Deep Learning (Intent - tag) ??? ?? ?? ???? Story Definition Intent Mapping?? ?? Entity Mapping ?? : ??, ?? : ??, ?? : ?? Pattern Generation 30% of Train Data ?? : ?? ?? (??) Preprocessing?? ?? ?? ?? Story key value (??) tagloc tagdate tagmenu ?? Model Train Evaluation tagloc tagdate tagmenu ?? tagloc tagdate ?? tagdate tagmenu ?? tagloc tagmenu ?? Predictiontagloc tagdate ?? tagmenu Hyper parameter Selection ?? = ??
Data flow for Model in Deep Learning (NER - BIO) ??? ?? ?? ???? Story Definition tagloc tagdate tagmenu ?? BIO-Mapping Preprocessing?? ?? ?? ?? B_Loc / B_Date / B_menu Model Train(Bi-LSTM) B-loc B-date B-menu O B-loc B-date O B-date B-menu O B-loc B-menu O Text Generator Pattern Matching tagloc tagdate tagmenu ?? tagloc tagdate ?? tagdate tagmenu ?? tagloc tagmenu ?? W2V 30% of Train Data Evaluation Prediction?? ?? ?? ?? Hyper parameter Selection ?? : 0.12 ?? : 0.7 ?? : 0.3 ???? B_loc B_Date B_menu O
????? ?? AI ?? ??
Intent? ????? (Text Classification) ???? ?? ?? / ?? ?? ??? / ?? ???? ??, ??, ??? 3?? ?? ?? ? Word???? ??? ??? ?? ??? ??? ?? ex) ?? ???? ?? / ?? ??? ???¡­. Deeplearning? ???? ?? ???? ?? ? ? ?? Char + CNN?? ????? (CNN - Feature ??, ??, ??) (Word Similarity ??, ?? / ??, ????)
Intent? ????? (Text Classification - Data ??) Word ?? ?? ?? ?? Vector? ??? ?????? PIJA JUMUN HAGO SIPO ??, ????,??? ?? ????? W2V(Pretrained) ?? (0.12, 0.54, 0.72) ??(0.56, 0.65, 0.64) ??(0.67, 0.91, 0.13) ??(0.89, 0.14, 0.11) Ont Hotencoding (Word?? or ????) (0100000000) (0000010000) (0010000000) (0000000100) Ont Hotencoding (A~Z Vector) (0100000000) (0000010000) (0010000000) (0000000100)
Char CNN? CNN? ????? ???? ??? ???? ????? ?? ??? ???? ??? Vector?? ???? Vector? ???? ???? Feature? ??? ? ??
Text Classification - Char CNN ?? ?? ?? ?? ?? [?? Convolutional Neural Networks for Sentence Classification - Yoon Kim - https://arxiv.org/abs/1408.5882] ?? ?? ?? Feature ?????? [3,4,5 filter] Vector (W2V) ??/??/??? Static / Non Static / Random pooling ??? classification ?? Char-CNN? ???? ??? ?????
Why Char-CNN?? Char-CNN? ???? ?? ????? ???? ??? ??? ?? ?? Convolutional Neural Networks for Sentence Classification - Yoon Kim - https://arxiv.org/abs/1408.5882
RNN? ?? ?? ??? Data? ?? ???? ?? ???? ???? ?? ??? Backpropagation? ??? ???? ??(BPTT) http://aikorea.org/blog/rnn-tutorial-3/
http://cs231n.stanford.edu/slides/2016/winter1516_lecture10.pdf Seq2Seq (RNN+RNN) ?? Chatbot??? Generator? ?? Sentence Generator ?? ???? ???? ???? ???? ? ?? (??? ???? input/output??)
Attention Mechanism?? /KeonKim/attention-mechanisms-with-tensorflow Attention? ?? Computing Cost? ??? ??? ??? ?? ???? ??
http://cs231n.stanford.edu/slides/2016/winter1516_lecture10.pdf LSTM? ?? ?? Cell State https://brunch.co.kr/@chris-song/9 updateforget out cell state
http://cs231n.stanford.edu/slides/2016/winter1516_lecture10.pdf ResNet? RNN? LSTM? ??? ??
Named Entity Recognition ???? Bidirectional LSTM (??? Layer) - RNN??? ?? - ????? ?? ??? ??? ?? ??? ?? ??? ?? ?? ???? ???? ?? [ ??? ???? ???? - https://sites.google.com/site/2016hclt/jalyosil]
Why Bi-LSTM CRF ? [ Bidirectional LSTM-CRF Models for Sequence Tagging - https://arxiv.org/pdf/1508.01991.pdf ]
?? ???? ?? B-Pizza B-Order O O ?? ?? ??? B-Travel B-Information O ?? ???? B-Hotel B-Reserve O Named Entity Recognition ???? brat? ?? BIO Tagging B-???? I-???? ?? O-????, ??(OUT) U-Unknown (Word Embedding? ???) ¡ùNew York?,????? Brat - http://brat.nlplab.org/examples.html / https://wapiti.limsi.fr/
Bi-LSTM?? ?? ?? -> ?? ?? ?? ???? ?? B-Pizza B-Order O O ?? ?? ??? B-Travel B-Info O ?? ???? B-Hotel B-Reserve O ??? ???? ?? ??? ?? ??? ?? ???? ?? ?? ?? Bi-LSTM? ??? ?? ??? ???? ??Data? ???? ??? ??? ????? ?? ??
????? with Ontology ??? ?? ?? ??? ???? ???? ??? ? ?????? ??? ??DB? ??? ?? ??? ????? ??? ?? ?? ???? ? ??? ?????? ??...
Ontology with syntaxnet Neo4j - https://github.com/TensorMSA/tensormsa_jupyter/blob/master/chap13_chatbot_lecture/6.Ontology(Neo4J).ipynb Ontology??? ?? Neo4j DB? ?? ?????? ?? ???? ??? Syntaxnet? ?? A?B?? B? C?? ?? : A? C??
Chatbot Service? ?? Architecture ??
ChatBot Layer Log File Chatbot Architecture Deep Learning Layer ?? ChatBot Layer ? ?? Application Layer ? ???? ? Application Layer ? ??? ??? DL Layer ? ??. DeepLearning Layer Bi-LSTM Char-CNN Seq2Seq NAS File Model Bot DB Residual NLP Context Analyzer Decision Maker Response Genertor ¡ù ?????? ?? Residual?? ?? ?? ?? Bot Builder GPU Deeplearning Predict Dict File Bot config Train Train Intent / NER
NLP Architecture Preprocessing Python 3.5 Konlpy Mecab (Sejong Corpus) Tensorflow 1.1 AutoEncoder Char-CNN Bi-LSTM CRF Gensim FastText User-Dic Synonym Voting Python 3.5 API Service (Swagger) ?? ???? ???ICT? ????? ?? ???? ??... [Intent ??] ?? ?? [NER ??] ?? - Loc ???ICT - Loc ??? - Name ???? ('???'ICT'', 'NNP'), ('???', 'NNP'), ¡ùMecab ?????? ?? ?????? , ????(...) ?? ?? ?? ?? ?? [('??', 'NNG'), ('??', 'NNG'), ('?', 'XSV'), ('?', 'ETM'), ('???'ICT'', 'NNP'), ('?', 'JKB'), ('???', 'NNP'), ('??', 'JKB'), ('??', 'NNG'), ('??', 'NNG'), ('?', 'XSV'), ('?', 'EC'), ('?', 'VX'), ('?', 'EC')] Intent Slot ? ?? ?? ???? Slot? Entity? NER ??? Input Data=¡¯¡¯ ?? ???? ???ICT? ????? ?? ???? ??¡­¡± Intent=¡¯????¡¯ Intent_History=[¡®????¡¯,¡¯¡¯] story_slot_entity { ¡®??¡¯:??¡¯¡¯, ¡®??¡¯ : ¡¯?? ???ICT¡¯, ¡®??¡¯ : ¡®???¡¯} request_type=¡¯text¡¯ service_type=¡¯pizza order¡¯ output_data=¡¯¡¯ } Meta
Docker (Ubuntu) in AWS EC2 (c4.8xlarge / p2.xlarge GPU) NAS DB Server Bot Builder (analysis) React Chatbot Server (Django) Python 3.5 Tensorflow Postgres SQL Bootstrap Web Service Architecture D3 SCSS Konlpy Nginx Celery Log File Model File Rabbit MQ Service Java Node Python Rest Gensim Front-End Java (Trigger) Rest LB Rest AP2 GPU Server (HDF5) GPU Server (HDF5) Dict File Hbase
Bot Builder and UX
ChatBot Definition ChatBot Intent ChatBot Service ChatBot Intent Entity ChatBot Story ChatBot Response ChatBot Model ChatBot Tagging ChatBot Entity Relation ChatBot Synonym Bot Builder DB Service? ??? ?? ???? Common?? ??
Rest API Client Input Data=???? ?? ???? Intent=¡¯¡¯ Intent_History=[¡® ¡¯,¡¯¡¯] story_slot_entity { ??:¡¯¡¯, ???:¡¯¡¯, ???:¡¯¡¯ } request_type=text service_type=¡¯¡¯ output_data=¡¯¡¯ Server Input Data= ???? ?? ???? Intent=???? Intent_History=[¡®????¡¯,¡¯¡¯] story_slot_entity { ??:??, ???:??, ???: ?? } request_type=text service_type=¡¯¡¯ output_data=???? Chatbot API ¡ù json? ??? ???? log??? ??
Case? Test Coverage ?? ?? 1. ?? ?? (?????) 2. Model ?? (Hyper Parameter) 3. Data ??(Slot, Dict, Entity,???) 4. ?? ? ?? (Threshold, Rule??) ?? ?? ???? ??? Data? Model? ????? ??? ?? ? ? ?? ?? ?? ?????? ???? ??? ??? Continous Integration? ?? (Jenkins / Travis CI?) Test Codes for Chatbot ???? ???? ????->NER??-> Slot?? ???? input ??? ?????? -> intent : ???? slot : {??,??,???-extra}
???? ???? ??? ?? Tips
??? ???? ??? ?? ???? ??? ???? ?? (Scoring / Voting) ??? ?? ?? ????? ???? ?? ??? ?? ??? Textming? ???? ???? ???? ??? ???ICT? ?? ?? ???? Char-CNN VotingSVM(Radial Basis/linear) Result naive_bayes.MultinomialNB (TF-IDF) ? ??? Slot ?? ??? ??? ??,??? ?? ???? ???? ???? ?? ?? Ensemble and Voting ??? ??? Slot ?? ?? ??
Trigger ?? (??, ??? ??) 1. ????? ??? ?? <?? ?? ??> ?? : XXX ???? ????? ??? ??? ?? : ?? ?? ???? ???. ?? : ?? ?? ? ??? ?? ?? : ????? ?? ? ???? ???. ?? : ???? ?? : ??? [??, ??, ???] ?? Trigger? ???? ?? ??? Data? Seq2Seq??? ???? NLP??? ??? ?? 2.??? ?? ?(ResNet Model Call)
??? Tone Generater? ?? ??? ????? (???, ??? , ???) ??? ??????? (??) ??? ?????? (??) ??? ?????? (??) ??? ??????? (??) Seq2Seq Model?? - Encoder? ??? ?? Decoder? ??+?? ?? Response Generator? ?? ??? ???? ?? https://github.com/TensorMSA/tensormsa_jupyter/blob/master/chap13_chatbot_lecture/4.Chatb ot(Response-seq2seq).ipynb
??? ??(N-Gram) ???? - Pepperoni, ????, ????..... / Mac Book Pro - ???, ????... ???? ??? ??? ???? API????? ?? ??? ?? ? N-Gram? ???? ???? ??? ??? Dict? ?? ?? (??? trigram) ?? https://www.simplicity.be/article/throwing-dices-recognizing-west-flemish-and-other-languages/ ? Entity? N? Threshold ?? ???? ?? ¡ù threshold : ???? ???? ??
Response Speed LB ?? Nginx ?? ??? ?? Thread? AP Caching of Data (Memory - API??) Chatbot?? ???? ?? MAX Time??
??? ?? ??? ?? Coding tf.device? ?? ??? Device? ?? CPU? GPU? ??? ?? GPU? ??? ??? ????...
??? ¡ñ ??? ??? ??? Hot? ??? ??? ????? ???? Domain? Data? ??? ?? ???? ? ??? ? ?? ??? ¡ñ ??? Data? ?? Data? ??? ????? ?? ??(???) ¡ñ ???? ??? ??? Data? ??? ??? ¡ñ ???? ????? ?? ??? ?? ??? ? ? ??
When the singularity comes...
Reference ??? ?? ??? http://hunkim.github.io/ml/ ?28? ?? ? ??? ???? ?? ?? https://sites.google.com/site/2016hclt/jalyosil Stanford University CS231n http://cs231n.stanford.edu/ Creating AI chat bot with Python 3 and Tensorflow[???] https://speakerdeck.com/inureyes/building-ai-chat-bot-using-python-3-and-tensorflow ????? ??_??? [???] /KimSungdong1/20170227-72644192?next_slideshow=1 ???? ??? ?? ???? ?? [???] http://www.slideshare.net/deview/221-67605830 Developing Korean Chatbot 101 [???] /JaeminCho6/developing-korean-chatbot-101-71013451 Tensorflow-Tutorials https://github.com/golbin/TensorFlow-Tutorials

Recommended

Python? ??? ?? ??? ?? 1??
Python? ??? ?? ??? ?? 1??Python? ??? ?? ??? ?? 1??
Python? ??? ?? ??? ?? 1??
Taekyung Han
?
???? ??? ?? ??? ?? 3??
???? ??? ?? ??? ?? 3?????? ??? ?? ??? ?? 3??
???? ??? ?? ??? ?? 3??
Taekyung Han
?
20170227 ????? ??_???
20170227 ????? ??_???20170227 ????? ??_???
20170227 ????? ??_???
Kim Sungdong
?
???? ? ???, ???? ? ???
???? ? ???, ???? ? ??????? ? ???, ???? ? ???
???? ? ???, ???? ? ???
Yongho Ha
?
1?????????? ???????? ???????????? ??????????? ??????
1?????????? ???????? ???????????? ??????????? ??????1?????????? ???????? ???????????? ??????????? ??????
1?????????? ???????? ???????????? ??????????? ??????
??? Jerry Jeong
?
??????????????????????? airs???????_?????????????????
??????????????????????? airs???????_???????????????????????????????????????? airs???????_?????????????????
??????????????????????? airs???????_?????????????????
NAVER D2
?
?? ??? ? ???? ???? ?? ?? (????? KUCC, 2022? 4?)
?? ??? ? ???? ???? ?? ?? (????? KUCC, 2022? 4?)?? ??? ? ???? ???? ?? ?? (????? KUCC, 2022? 4?)
?? ??? ? ???? ???? ?? ?? (????? KUCC, 2022? 4?)
Suhyun Park
?
??? ??? ???, ??? ?? ??? ?? ?????
??? ??? ???, ??? ?? ??? ?? ???????? ??? ???, ??? ?? ??? ?? ?????
??? ??? ???, ??? ?? ??? ?? ?????
Taehoon Kim
?
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ????? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
Hui Seo
?
???? ???? ????????, ???? ??? ??????.
???? ???? ????????, ???? ??? ??????.???? ???? ????????, ???? ??? ??????.
???? ???? ????????, ???? ??? ??????.
Yongho Ha
?
Python? ??? ?? ??? ?? 2??
Python? ??? ?? ??? ?? 2??Python? ??? ?? ??? ?? 2??
Python? ??? ?? ??? ?? 2??
Taekyung Han
?
???? ??? ??? ?? ??? ??.
???? ??? ??? ?? ??? ??.???? ??? ??? ?? ??? ??.
???? ??? ??? ?? ??? ??.
Yongho Ha
?
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
ke-m kamekoopa
?
???? / ????? ????
???? / ????? ???????? / ????? ????
???? / ????? ????
Yongha Kim
?
????? ??? ????(I)
????? ??? ????(I)????? ??? ????(I)
????? ??? ????(I)
?? ?
?
?? SW ??? ?? ??
?? SW ??? ?? ???? SW ??? ?? ??
?? SW ??? ?? ??
?? ?
?
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
WON JOON YOO
?
???, ?? ??? ?? - ???? ????, NDC2019
???, ?? ??? ?? - ???? ????, NDC2019???, ?? ??? ?? - ???? ????, NDC2019
???, ?? ??? ?? - ???? ????, NDC2019
devCAT Studio, NEXON
?
???, ??? ?? ???: ????, NDC2017
???, ??? ?? ???: ????, NDC2017???, ??? ?? ???: ????, NDC2017
???, ??? ?? ???: ????, NDC2017
devCAT Studio, NEXON
?
??? ?? ??? (??)
??? ?? ??? (??)??? ?? ??? (??)
??? ?? ??? (??)
Heungsub Lee
?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
Takahiro Inoue
?
RoFormer: Enhanced Transformer with Rotary Position Embedding
RoFormer: Enhanced Transformer with Rotary Position EmbeddingRoFormer: Enhanced Transformer with Rotary Position Embedding
RoFormer: Enhanced Transformer with Rotary Position Embedding
taeseon ryu
?
???? ????? - ????? ?????
???? ????? - ????? ????????? ????? - ????? ?????
???? ????? - ????? ?????
Seonggwan Lee
?
????, ????, IoT, ????. ? ??? ?? ??? ?????
????, ????, IoT, ????. ? ??? ?? ??? ?????????, ????, IoT, ????. ? ??? ?? ??? ?????
????, ????, IoT, ????. ? ??? ?? ??? ?????
Yongho Ha
?
??? ?? ???? AI ????- ???? ??
??? ?? ???? AI ????- ???? ????? ?? ???? AI ????- ???? ??
??? ?? ???? AI ????- ???? ??
Hangil Kim
?
[226]???????? ?????????????? ????? ???????
[226]???????? ?????????????? ????? ???????[226]???????? ?????????????? ????? ???????
[226]???????? ?????????????? ????? ???????
NAVER D2
?
???? ???? (Intelligence on Kakao Advertising)
???? ???? (Intelligence on Kakao Advertising)???? ???? (Intelligence on Kakao Advertising)
???? ???? (Intelligence on Kakao Advertising)
if kakao
?
???? ???? ??? ??? ???
???? ???? ??? ??? ??????? ???? ??? ??? ???
???? ???? ??? ??? ???
Kwangsik Lee
?
?11???sw????? ?? TensorMSA(??)
?11???sw????? ?? TensorMSA(??)?11???sw????? ?? TensorMSA(??)
?11???sw????? ?? TensorMSA(??)
Susang Kim
?
?????(??? ?? ???? ??)
?????(??? ?? ???? ??)?????(??? ?? ???? ??)
?????(??? ?? ???? ??)
SU BO KIM
?

More Related Content

What's hot (20)

??? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ????? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
Hui Seo
?
???? ???? ????????, ???? ??? ??????.
???? ???? ????????, ???? ??? ??????.???? ???? ????????, ???? ??? ??????.
???? ???? ????????, ???? ??? ??????.
Yongho Ha
?
Python? ??? ?? ??? ?? 2??
Python? ??? ?? ??? ?? 2??Python? ??? ?? ??? ?? 2??
Python? ??? ?? ??? ?? 2??
Taekyung Han
?
???? ??? ??? ?? ??? ??.
???? ??? ??? ?? ??? ??.???? ??? ??? ?? ??? ??.
???? ??? ??? ?? ??? ??.
Yongho Ha
?
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
ke-m kamekoopa
?
???? / ????? ????
???? / ????? ???????? / ????? ????
???? / ????? ????
Yongha Kim
?
????? ??? ????(I)
????? ??? ????(I)????? ??? ????(I)
????? ??? ????(I)
?? ?
?
?? SW ??? ?? ??
?? SW ??? ?? ???? SW ??? ?? ??
?? SW ??? ?? ??
?? ?
?
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
WON JOON YOO
?
???, ?? ??? ?? - ???? ????, NDC2019
???, ?? ??? ?? - ???? ????, NDC2019???, ?? ??? ?? - ???? ????, NDC2019
???, ?? ??? ?? - ???? ????, NDC2019
devCAT Studio, NEXON
?
???, ??? ?? ???: ????, NDC2017
???, ??? ?? ???: ????, NDC2017???, ??? ?? ???: ????, NDC2017
???, ??? ?? ???: ????, NDC2017
devCAT Studio, NEXON
?
??? ?? ??? (??)
??? ?? ??? (??)??? ?? ??? (??)
??? ?? ??? (??)
Heungsub Lee
?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
Takahiro Inoue
?
RoFormer: Enhanced Transformer with Rotary Position Embedding
RoFormer: Enhanced Transformer with Rotary Position EmbeddingRoFormer: Enhanced Transformer with Rotary Position Embedding
RoFormer: Enhanced Transformer with Rotary Position Embedding
taeseon ryu
?
???? ????? - ????? ?????
???? ????? - ????? ????????? ????? - ????? ?????
???? ????? - ????? ?????
Seonggwan Lee
?
????, ????, IoT, ????. ? ??? ?? ??? ?????
????, ????, IoT, ????. ? ??? ?? ??? ?????????, ????, IoT, ????. ? ??? ?? ??? ?????
????, ????, IoT, ????. ? ??? ?? ??? ?????
Yongho Ha
?
??? ?? ???? AI ????- ???? ??
??? ?? ???? AI ????- ???? ????? ?? ???? AI ????- ???? ??
??? ?? ???? AI ????- ???? ??
Hangil Kim
?
[226]???????? ?????????????? ????? ???????
[226]???????? ?????????????? ????? ???????[226]???????? ?????????????? ????? ???????
[226]???????? ?????????????? ????? ???????
NAVER D2
?
???? ???? (Intelligence on Kakao Advertising)
???? ???? (Intelligence on Kakao Advertising)???? ???? (Intelligence on Kakao Advertising)
???? ???? (Intelligence on Kakao Advertising)
if kakao
?
???? ???? ??? ??? ???
???? ???? ??? ??? ??????? ???? ??? ??? ???
???? ???? ??? ??? ???
Kwangsik Lee
?
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ????? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
??? ???? ?? SKILLSET? ??? ???? - ??? ??? ??
Hui Seo
?
???? ???? ????????, ???? ??? ??????.
???? ???? ????????, ???? ??? ??????.???? ???? ????????, ???? ??? ??????.
???? ???? ????????, ???? ??? ??????.
Yongho Ha
?
Python? ??? ?? ??? ?? 2??
Python? ??? ?? ??? ?? 2??Python? ??? ?? ??? ?? 2??
Python? ??? ?? ??? ?? 2??
Taekyung Han
?
???? ??? ??? ?? ??? ??.
???? ??? ??? ?? ??? ??.???? ??? ??? ?? ??? ??.
???? ??? ??? ?? ??? ??.
Yongho Ha
?
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
Éè¼Æ¤·¤Æ¤Þ¤¹¤«£¿
ke-m kamekoopa
?
???? / ????? ????
???? / ????? ???????? / ????? ????
???? / ????? ????
Yongha Kim
?
????? ??? ????(I)
????? ??? ????(I)????? ??? ????(I)
????? ??? ????(I)
?? ?
?
?? SW ??? ?? ??
?? SW ??? ?? ???? SW ??? ?? ??
?? SW ??? ?? ??
?? ?
?
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
? ?? ??? ??? ??? ?? ?????. (Deep Learning for Natural Language Processing)
WON JOON YOO
?
???, ?? ??? ?? - ???? ????, NDC2019
???, ?? ??? ?? - ???? ????, NDC2019???, ?? ??? ?? - ???? ????, NDC2019
???, ?? ??? ?? - ???? ????, NDC2019
devCAT Studio, NEXON
?
???, ??? ?? ???: ????, NDC2017
???, ??? ?? ???: ????, NDC2017???, ??? ?? ???: ????, NDC2017
???, ??? ?? ???: ????, NDC2017
devCAT Studio, NEXON
?
??? ?? ??? (??)
??? ?? ??? (??)??? ?? ??? (??)
??? ?? ??? (??)
Heungsub Lee
?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
¡¸³Ò°ù²¹±è³ó¶Ùµþ³¹µ×ÈëÃÅ¡¹?¹¹Ôì¤äÊË×é¤ßÀí½â¤«¤éʹ¤¤¤É¤³¤í?ÖÖ¡©¤Î³Ò°ù²¹±è³ó¶Ùµþ¤Î±È½Ï¤Þ¤Ç·ùŽÚ¤¯?
Takahiro Inoue
?
RoFormer: Enhanced Transformer with Rotary Position Embedding
RoFormer: Enhanced Transformer with Rotary Position EmbeddingRoFormer: Enhanced Transformer with Rotary Position Embedding
RoFormer: Enhanced Transformer with Rotary Position Embedding
taeseon ryu
?
???? ????? - ????? ?????
???? ????? - ????? ????????? ????? - ????? ?????
???? ????? - ????? ?????
Seonggwan Lee
?
????, ????, IoT, ????. ? ??? ?? ??? ?????
????, ????, IoT, ????. ? ??? ?? ??? ?????????, ????, IoT, ????. ? ??? ?? ??? ?????
????, ????, IoT, ????. ? ??? ?? ??? ?????
Yongho Ha
?
??? ?? ???? AI ????- ???? ??
??? ?? ???? AI ????- ???? ????? ?? ???? AI ????- ???? ??
??? ?? ???? AI ????- ???? ??
Hangil Kim
?
[226]???????? ?????????????? ????? ???????
[226]???????? ?????????????? ????? ???????[226]???????? ?????????????? ????? ???????
[226]???????? ?????????????? ????? ???????
NAVER D2
?
???? ???? (Intelligence on Kakao Advertising)
???? ???? (Intelligence on Kakao Advertising)???? ???? (Intelligence on Kakao Advertising)
???? ???? (Intelligence on Kakao Advertising)
if kakao
?
???? ???? ??? ??? ???
???? ???? ??? ??? ??????? ???? ??? ??? ???
???? ???? ??? ??? ???
Kwangsik Lee
?

Viewers also liked (16)

?11???sw????? ?? TensorMSA(??)
?11???sw????? ?? TensorMSA(??)?11???sw????? ?? TensorMSA(??)
?11???sw????? ?? TensorMSA(??)
Susang Kim
?
?????(??? ?? ???? ??)
?????(??? ?? ???? ??)?????(??? ?? ???? ??)
?????(??? ?? ???? ??)
SU BO KIM
?
Attention mechanisms with tensorflow
Attention mechanisms with tensorflowAttention mechanisms with tensorflow
Attention mechanisms with tensorflow
Keon Kim
?
1. ????????? ????? ????????
1. ????????? ????? ????????1. ????????? ????? ????????
1. ????????? ????? ????????
Terry Cho
?
Deep learning for 3-D Scene Reconstruction and Modeling
Deep learning for 3-D Scene Reconstruction and Modeling Deep learning for 3-D Scene Reconstruction and Modeling
Deep learning for 3-D Scene Reconstruction and Modeling
Yu Huang
?
DE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra Bagnato
DE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra BagnatoDE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra Bagnato
DE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra Bagnato
Alessandra Bagnato
?
Reduction of cogging torque in permanent magnet machine
Reduction of cogging torque in permanent magnet machineReduction of cogging torque in permanent magnet machine
Reduction of cogging torque in permanent magnet machine
Krithik Kumar Chandrashekar
?
R + Apache Spark
R + Apache SparkR + Apache Spark
R + Apache Spark
Dmitry Petukhov
?
??????? ????????? ???????? ????
??????? ????????? ???????? ??????????? ????????? ???????? ????
??????? ????????? ???????? ????
Kwangyoun Jung
?
2017 Business Platform Day
2017 Business Platform Day2017 Business Platform Day
2017 Business Platform Day
ROA Consulting Inc.
?
Micro Service Architecture? ??
Micro Service Architecture? ??Micro Service Architecture? ??
Micro Service Architecture? ??
Terry Cho
?
????, ???? ??? ???
????, ???? ??? ???????, ???? ??? ???
????, ???? ??? ???
Jinwon Lee
?
Ai ????
Ai ????Ai ????
Ai ????
?? ?
?
[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???
[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???
[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???
OpenStack Korea Community
?
Developing Korean Chatbot 101
Developing Korean Chatbot 101Developing Korean Chatbot 101
Developing Korean Chatbot 101
Jaemin Cho
?
? ???? ???: ?? ???? ????? ????? DEVIEW 2017
? ???? ???: ?? ???? ????? ????? DEVIEW 2017? ???? ???: ?? ???? ????? ????? DEVIEW 2017
? ???? ???: ?? ???? ????? ????? DEVIEW 2017
Taehoon Kim
?
?11???sw????? ?? TensorMSA(??)
?11???sw????? ?? TensorMSA(??)?11???sw????? ?? TensorMSA(??)
?11???sw????? ?? TensorMSA(??)
Susang Kim
?
?????(??? ?? ???? ??)
?????(??? ?? ???? ??)?????(??? ?? ???? ??)
?????(??? ?? ???? ??)
SU BO KIM
?
Attention mechanisms with tensorflow
Attention mechanisms with tensorflowAttention mechanisms with tensorflow
Attention mechanisms with tensorflow
Keon Kim
?
1. ????????? ????? ????????
1. ????????? ????? ????????1. ????????? ????? ????????
1. ????????? ????? ????????
Terry Cho
?
Deep learning for 3-D Scene Reconstruction and Modeling
Deep learning for 3-D Scene Reconstruction and Modeling Deep learning for 3-D Scene Reconstruction and Modeling
Deep learning for 3-D Scene Reconstruction and Modeling
Yu Huang
?
DE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra Bagnato
DE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra BagnatoDE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra Bagnato
DE-CPS 2017 The INTO-CPS Cyber-Physical System Profile Alessandra Bagnato
Alessandra Bagnato
?
Reduction of cogging torque in permanent magnet machine
Reduction of cogging torque in permanent magnet machineReduction of cogging torque in permanent magnet machine
Reduction of cogging torque in permanent magnet machine
Krithik Kumar Chandrashekar
?
R + Apache Spark
R + Apache SparkR + Apache Spark
R + Apache Spark
Dmitry Petukhov
?
??????? ????????? ???????? ????
??????? ????????? ???????? ??????????? ????????? ???????? ????
??????? ????????? ???????? ????
Kwangyoun Jung
?
2017 Business Platform Day
2017 Business Platform Day2017 Business Platform Day
2017 Business Platform Day
ROA Consulting Inc.
?
Micro Service Architecture? ??
Micro Service Architecture? ??Micro Service Architecture? ??
Micro Service Architecture? ??
Terry Cho
?
????, ???? ??? ???
????, ???? ??? ???????, ???? ??? ???
????, ???? ??? ???
Jinwon Lee
?
Ai ????
Ai ????Ai ????
Ai ????
?? ?
?
[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???
[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???
[OpenStack Days Korea 2016] Track3 - ????? UHD ???? ?? ? ???
OpenStack Korea Community
?
Developing Korean Chatbot 101
Developing Korean Chatbot 101Developing Korean Chatbot 101
Developing Korean Chatbot 101
Jaemin Cho
?
? ???? ???: ?? ???? ????? ????? DEVIEW 2017
? ???? ???: ?? ???? ????? ????? DEVIEW 2017? ???? ???: ?? ???? ????? ????? DEVIEW 2017
? ???? ???: ?? ???? ????? ????? DEVIEW 2017
Taehoon Kim
?

Similar to Python? Tensorflow? ??? AI Chatbot ?? ? ?? ?? (20)

Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Platform
?
LLM ????? ????? ?????? ?????? ??????
LLM ????? ????? ?????? ?????? ??????LLM ????? ????? ?????? ?????? ??????
LLM ????? ????? ?????? ?????? ??????
Tae Young Lee
?
[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???
[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???
[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???
ChangKyu Song
?
Slipp ????? ???? 20151212
Slipp ????? ???? 20151212Slipp ????? ???? 20151212
Slipp ????? ???? 20151212
Jinsoo Jung
?
Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...
Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...
Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...
hoondong kim
?
Definition of ipa
Definition of ipa Definition of ipa
Definition of ipa
ssuser9a50211
?
100% Serverless big data scale production Deep Learning System
100% Serverless big data scale production Deep Learning System100% Serverless big data scale production Deep Learning System
100% Serverless big data scale production Deep Learning System
hoondong kim
?
Ipa approach
Ipa approachIpa approach
Ipa approach
ssuser9a50211
?
H? IPA_Approach
H? IPA_ApproachH? IPA_Approach
H? IPA_Approach
ssuser9a50211
?
Ipa approach
Ipa approachIpa approach
Ipa approach
ssuser9a50211
?
??? ? ??? ??
??? ? ??? ????? ? ??? ??
??? ? ??? ??
if kakao
?
Ipa definition
Ipa definitionIpa definition
Ipa definition
ssuser9a50211
?
golang? websocket? ??? ??????? - ???? ?? ?? ???
golang? websocket? ??? ??????? - ???? ?? ?? ???golang? websocket? ??? ??????? - ???? ?? ?? ???
golang? websocket? ??? ??????? - ???? ?? ?? ???
Sangik Bae
?
20160414 ROS 2? ?? (for ???? ?)
20160414 ROS 2? ?? (for ???? ?)20160414 ROS 2? ?? (for ???? ?)
20160414 ROS 2? ?? (for ???? ?)
Yoonseok Pyo
?
Going asynchronous with netty - SOSCON 2015
Going asynchronous with netty - SOSCON 2015Going asynchronous with netty - SOSCON 2015
Going asynchronous with netty - SOSCON 2015
Kris Jeong
?
[E-commerce & Retail Day] ??????? ????
[E-commerce & Retail Day] ??????? ????[E-commerce & Retail Day] ??????? ????
[E-commerce & Retail Day] ??????? ????
Amazon Web Services Korea
?
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
Amazon Web Services Korea
?
H? IPA Definiton
H? IPA DefinitonH? IPA Definiton
H? IPA Definiton
ssuser9a50211
?
Ipa definition
Ipa definitionIpa definition
Ipa definition
ssuser9a50211
?
????? ??? ChatGPT
????? ??? ChatGPT????? ??? ChatGPT
????? ??? ChatGPT
Tae Young Lee
?
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Tech Summit 2: ???? ?? ??? Chatbot ?? ??? ??
Clova Platform
?
LLM ????? ????? ?????? ?????? ??????
LLM ????? ????? ?????? ?????? ??????LLM ????? ????? ?????? ?????? ??????
LLM ????? ????? ?????? ?????? ??????
Tae Young Lee
?
[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???
[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???
[NDC12] ??? ??? ???? ? ??? ???? ????? ????? ??? - ???
ChangKyu Song
?
Slipp ????? ???? 20151212
Slipp ????? ???? 20151212Slipp ????? ???? 20151212
Slipp ????? ???? 20151212
Jinsoo Jung
?
Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...
Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...
Auto Scalable ? Deep Learning Production ? ?? AI Serving Infra ?? ? AI DevOps...
hoondong kim
?
Definition of ipa
Definition of ipa Definition of ipa
Definition of ipa
ssuser9a50211
?
100% Serverless big data scale production Deep Learning System
100% Serverless big data scale production Deep Learning System100% Serverless big data scale production Deep Learning System
100% Serverless big data scale production Deep Learning System
hoondong kim
?
Ipa approach
Ipa approachIpa approach
Ipa approach
ssuser9a50211
?
H? IPA_Approach
H? IPA_ApproachH? IPA_Approach
H? IPA_Approach
ssuser9a50211
?
Ipa approach
Ipa approachIpa approach
Ipa approach
ssuser9a50211
?
??? ? ??? ??
??? ? ??? ????? ? ??? ??
??? ? ??? ??
if kakao
?
Ipa definition
Ipa definitionIpa definition
Ipa definition
ssuser9a50211
?
golang? websocket? ??? ??????? - ???? ?? ?? ???
golang? websocket? ??? ??????? - ???? ?? ?? ???golang? websocket? ??? ??????? - ???? ?? ?? ???
golang? websocket? ??? ??????? - ???? ?? ?? ???
Sangik Bae
?
20160414 ROS 2? ?? (for ???? ?)
20160414 ROS 2? ?? (for ???? ?)20160414 ROS 2? ?? (for ???? ?)
20160414 ROS 2? ?? (for ???? ?)
Yoonseok Pyo
?
Going asynchronous with netty - SOSCON 2015
Going asynchronous with netty - SOSCON 2015Going asynchronous with netty - SOSCON 2015
Going asynchronous with netty - SOSCON 2015
Kris Jeong
?
[E-commerce & Retail Day] ??????? ????
[E-commerce & Retail Day] ??????? ????[E-commerce & Retail Day] ??????? ????
[E-commerce & Retail Day] ??????? ????
Amazon Web Services Korea
?
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
[Partner TechForum] ??? ??? ?? ??? ??? ?? ??
Amazon Web Services Korea
?
H? IPA Definiton
H? IPA DefinitonH? IPA Definiton
H? IPA Definiton
ssuser9a50211
?
Ipa definition
Ipa definitionIpa definition
Ipa definition
ssuser9a50211
?
????? ??? ChatGPT
????? ??? ChatGPT????? ??? ChatGPT
????? ??? ChatGPT
Tae Young Lee
?

More from Susang Kim (15)

[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...
[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...
[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...
Susang Kim
?
[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)
Susang Kim
?
[Paper] dynamic routing between capsules
[Paper] dynamic routing between capsules[Paper] dynamic routing between capsules
[Paper] dynamic routing between capsules
Susang Kim
?
[Paper] anti spoofing for face recognition
[Paper] anti spoofing for face recognition[Paper] anti spoofing for face recognition
[Paper] anti spoofing for face recognition
Susang Kim
?
[Paper] attention mechanism(luong)
[Paper] attention mechanism(luong)[Paper] attention mechanism(luong)
[Paper] attention mechanism(luong)
Susang Kim
?
[Paper] shuffle net an extremely efficient convolutional neural network for ...
[Paper] shuffle net an extremely efficient convolutional neural network for ...[Paper] shuffle net an extremely efficient convolutional neural network for ...
[Paper] shuffle net an extremely efficient convolutional neural network for ...
Susang Kim
?
[Paper] EDA : easy data augmentation techniques for boosting performance on t...
[Paper] EDA : easy data augmentation techniques for boosting performance on t...[Paper] EDA : easy data augmentation techniques for boosting performance on t...
[Paper] EDA : easy data augmentation techniques for boosting performance on t...
Susang Kim
?
[Paper] auto ml part 1
[Paper] auto ml part 1[Paper] auto ml part 1
[Paper] auto ml part 1
Susang Kim
?
[Paper] eXplainable ai(xai) in computer vision
[Paper] eXplainable ai(xai) in computer vision[Paper] eXplainable ai(xai) in computer vision
[Paper] eXplainable ai(xai) in computer vision
Susang Kim
?
[Paper] learning video representations from correspondence proposals
[Paper] learning video representations from correspondence proposals[Paper] learning video representations from correspondence proposals
[Paper] learning video representations from correspondence proposals
Susang Kim
?
[Paper] DetectoRS for Object Detection
[Paper] DetectoRS for Object Detection[Paper] DetectoRS for Object Detection
[Paper] DetectoRS for Object Detection
Susang Kim
?
Long term feature banks for detailed video understanding (Action Recognition)
Long term feature banks for detailed video understanding (Action Recognition)Long term feature banks for detailed video understanding (Action Recognition)
Long term feature banks for detailed video understanding (Action Recognition)
Susang Kim
?
I3D and Kinetics datasets (Action Recognition)
I3D and Kinetics datasets (Action Recognition)I3D and Kinetics datasets (Action Recognition)
I3D and Kinetics datasets (Action Recognition)
Susang Kim
?
GroupFace (Face Recognition)
GroupFace (Face Recognition)GroupFace (Face Recognition)
GroupFace (Face Recognition)
Susang Kim
?
Sk t academy lecture note
Sk t academy lecture noteSk t academy lecture note
Sk t academy lecture note
Susang Kim
?
[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...
[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...
[Paper] GIRAFFE: Representing Scenes as Compositional Generative Neural Featu...
Susang Kim
?
[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)
Susang Kim
?
[Paper] dynamic routing between capsules
[Paper] dynamic routing between capsules[Paper] dynamic routing between capsules
[Paper] dynamic routing between capsules
Susang Kim
?
[Paper] anti spoofing for face recognition
[Paper] anti spoofing for face recognition[Paper] anti spoofing for face recognition
[Paper] anti spoofing for face recognition
Susang Kim
?
[Paper] attention mechanism(luong)
[Paper] attention mechanism(luong)[Paper] attention mechanism(luong)
[Paper] attention mechanism(luong)
Susang Kim
?
[Paper] shuffle net an extremely efficient convolutional neural network for ...
[Paper] shuffle net an extremely efficient convolutional neural network for ...[Paper] shuffle net an extremely efficient convolutional neural network for ...
[Paper] shuffle net an extremely efficient convolutional neural network for ...
Susang Kim
?
[Paper] EDA : easy data augmentation techniques for boosting performance on t...
[Paper] EDA : easy data augmentation techniques for boosting performance on t...[Paper] EDA : easy data augmentation techniques for boosting performance on t...
[Paper] EDA : easy data augmentation techniques for boosting performance on t...
Susang Kim
?
[Paper] auto ml part 1
[Paper] auto ml part 1[Paper] auto ml part 1
[Paper] auto ml part 1
Susang Kim
?
[Paper] eXplainable ai(xai) in computer vision
[Paper] eXplainable ai(xai) in computer vision[Paper] eXplainable ai(xai) in computer vision
[Paper] eXplainable ai(xai) in computer vision
Susang Kim
?
[Paper] learning video representations from correspondence proposals
[Paper] learning video representations from correspondence proposals[Paper] learning video representations from correspondence proposals
[Paper] learning video representations from correspondence proposals
Susang Kim
?
[Paper] DetectoRS for Object Detection
[Paper] DetectoRS for Object Detection[Paper] DetectoRS for Object Detection
[Paper] DetectoRS for Object Detection
Susang Kim
?
Long term feature banks for detailed video understanding (Action Recognition)
Long term feature banks for detailed video understanding (Action Recognition)Long term feature banks for detailed video understanding (Action Recognition)
Long term feature banks for detailed video understanding (Action Recognition)
Susang Kim
?
I3D and Kinetics datasets (Action Recognition)
I3D and Kinetics datasets (Action Recognition)I3D and Kinetics datasets (Action Recognition)
I3D and Kinetics datasets (Action Recognition)
Susang Kim
?
GroupFace (Face Recognition)
GroupFace (Face Recognition)GroupFace (Face Recognition)
GroupFace (Face Recognition)
Susang Kim
?
Sk t academy lecture note
Sk t academy lecture noteSk t academy lecture note
Sk t academy lecture note
Susang Kim
?

Python? Tensorflow? ??? AI Chatbot ?? ? ?? ??

  • 1. WRITTEN BY SuSang Kim healess1@gmail.com Python? Tensorflow? ??? AI Chatbot ?? ? ?? ??
  • 2. 1. ?????? https://github.com/TensorMSA/skp_edu_docker 2. ?????? - jupyter (/chap13_chatbot_lecture) git clone https://github.com/TensorMSA/tensormsa_jupyter.git 3. Python 3.5 / Tensorflow 1.2
  • 3. I am Susang Kim as a developer ?Chatbot Develover - Released in POSCO (Find people using by NLP/AI) - Deep Learning MSA (ML,DNN, CNN, RNN) ?Agile Develover (Experienced in Pivotal Labs) - TDD, CI, Pair programming, User Story ?iOS Develover (Ranked App store in 100th - 2011 Korea) ?Front-End Developer (React, D3, Typescript and ES6) ?POSCO MES ... (working at POSCO ICT for 10 year)
  • 4. Contents 1. ?? 2. AI Chatbot ?? Chatbot Ecosystem Closed vs Open Domain Rule Based vs AI Chat IF Flow and Story Slot 3. AI??? ??? ?? Data ?? ?? Data? ??? ? / Train? ?? Word Representation Data? ?? / Data Augmentation(Intent, NER) 4. ????? ?? AI ?? ?? Intent (Char-CNN) / QnA (Seq2Seq) Named Entity Recognition (Bi-LSTM CRF) / Ontology (Graph DB)
  • 5. Contents 6. Chatbot Service? ?? Architecture ?? Chatbot Architecture NLP Architecture Web Service Architecture Bot builder / Chatbot API Test Codes for Chatbot 7. ???? ???? ??? ?? Tips Ensemble and voting / Trigger / Synonym(N-Gram) Tone Generator / Parallel processing / Response Speed 8. ??? [?? ??] Text Augmentation / Char-CNN / NER /Slot Bot / QA Bot / Graph DB / Response Generator
  • 6. ??
  • 7. Facebook AI shut down after creating their own language ?? https://arxiv.org/abs/1706.05125
  • 8. Chatbot? ????? ??? ? ?? ??? ?? (NLP, AI, F/W, Text Mining and ??? ?? skill) ? Deep Learning? ???? ???? ?? ??? ?? - ?? Computing?? ?? ???? ?? (Text ??) ? ??? ??(Micro Data??? ?? Biz dependency? ???) - ???(CNN)?? ??Data(DNN)??? Data??? ?? ???? ?? (??? ?????? ?? ??? ??? ????) ? ????? ?? (API??? ?????? ?? Smart Management) - Intent? Slot? ???? ?? ???? ???? ? ?? ????? ?? ???? (??? ??? ???????) - ???? ??? ??? ????? Text algorithm? ?? ???? ?? ?? ? Bot Service? ??? ????, ???? ???, Customize ??
  • 11. [?? Deview 2016 - https://deview.kr/2016/schedule#session/176] ?? ? Chatbot? ????? ???? UX ??? ?? ?? ??? ?? ?? ?? App ?? ?? ?? ??? ???? ?? ?? ?? Feedback ???? ??
  • 12. Chatbot Ecosystem [?? Digital Landscapes - https://www.linkedin.com/pulse/curated-list-digital-landscapes-marek-rakowski] Chatbot ???? ?? ???? ?? ??? ????? ??? ???NLP? Bot Builder (Slot?? ?? / QA? ??)
  • 13. ??? Chatbot Platform? ????? ?? API.AI? ???? ?? ??? https://calyfactory.github.io/api.ai-chatbot/ ?? ???? ??? ????? ?? ?? ? ?? ???? Data? ???!!! api.ai? ???? ??? ?????? ??? ????? ??? ?
  • 14. Closed Domain vs Open Domain Rule Based General (abstract) Open Closed Retrieval (accuracy) Impossible Strong AI Weak AI level of difficulty ?? Biz ????? ???? ???? ???? ?? Biz? ???? ??
  • 15. Rule Based vs AI Computer Input Program Output Rule ??, ??, ?? ???? ??? rule? ?????? - ???? ???? ?? ??? ? ???? Computer Input Output Program AI ???? Data??? ??? ?? ? ?? ??? ?? ? ?? - ??? Data?? ????(W2V,GloVe) intent = ??? ???? ??? ??? => Intent : ?? ?? ?? ??? NER = ??? ???? ??? ??? => B-Loc O O B-Name O ??? ??? ?? ? ??? ?? ??? ?? ??? ??? ??? ??? ? ??? Data? ?? ?? ??? ??(????) If (loc = ?? and comp = ???ICT) person = ??? elif (loc = ?? and comp = SK Planet) person = ??? else person = ???
  • 16. Chatbot Interface Flow NLP Context Analyzer Decision Maker ??? ???ICT? ???? Intent : ???? Entity : ?? = ?? ???ICT Service Manager Response Generator ??=null ??=null ???? Slot ??(Knowlodge Base) Entity : ??:Null, ??:null ???? ??? ???????. ???? Slot ?? ?? ??? ?????? ?? ??? ??? (Tone Gen) Slot OK
  • 17. Story slot? ?? (Frame-based DM) ?? ???? ?? Pizza Slot Size Type Side menu ?? ?? ?? ?? ?? Bot? ??? ?? 1) ?? ???? ?????? 2) ?? ??? ?????? 3) ??? ??? ??????? ??? ?? - ???? ??? ?? ???? ???????? NER?? ? Slot ?? Pizza Slot Size Large Type Pepperoni Side menu cola ??? ?? (Slot API Call) ??? ?? Slot? ???? ?? ??
  • 18. 1. ?? ?? ???? 2. ??? -> ?? ?? NER 3. ???? -> ?? Entity?? -> MacBook Pro API Call 4. ???? ?? 5. ?? ??? ??? ?? Slot ?? 6. ??? ?? ?? ??? ?? Slot? ??? ? ?? ??? ?????? ??? ???? ?? ?? ?? ? ?? (?? Frame???? ??? ? ???¡­) ex) ¡°?? ???¡± ??? Slot? ?? ??? http://www.11st.co.kr/toc/bridge.tmall?method=chatPage Slot Trigger API
  • 20. ????? Biz? ?? Text? ???? Deep Learning? ???? ???? ??? Text Tagging? ??? ?? ?? Data? ???? ??? Corpus? ????? ?? ???? ???? ???? Biz ??? ?? ????(???) - Corpus (annotation) ?????(2007 ) https://ithub.korean.go.kr/user/main.do - ??21 (2001~2014) ????? http://corpus.korea.ac.kr/ - Web Crawling or down (Wiki, Namu Wiki) - Domain Specific? ??? Text Data? ????? ??? ??? ?? ?? ?????? (??? , ????) ¡ù ????? ??? ??? ???? ?? ??? ????? Data? ??? ????
  • 21. Word Represention? ?? (???? ? ???? ??) - One Hot? ??? ?? ??? ???? Train ? ??? (Scope? ????-Sparse) - Word ?? Embedding ? ??? ? ??? (But Sparse) / W2V (???) - GloVe? ??? ?? ????? ?? (Global Vector : ???-???) - Char ?? Embedding ? ??? ?? ??? ?? (Vector? ????? ????) - ??? ??? ?? Char ?? Embedding? ?? ?? ???? ?? ??? ?? Train? ?? Word Representation 15 ???? ??? ?? ??? ?? ? ???? ??? ?? ??.pdf
  • 22. Train Vector? ?? ? Feature? ???? Cleansing -> Feature Engineering -> Train (??? ???? ??, ?? ?? ?? ?? - Tagging) ??? ??? ???? ??? ????? ??? ????Train Cost? ??? ??? ??? ??) ??? ??? ??? ?? (Dense Respresention-SVD) abcd~z, 0~9, ?, !, (,),¡¯,¡¯,??? ? 70?? ?????? ??? ????? ??? .lower()? ?????? ?? ?? ??? ???? Data? ??
  • 23. Data Augmentation for Deep Learning (Intent - tag) ??? ?? ?? ???? Story Definition Intent Mapping?? ?? Entity Mapping ?? : ??, ?? : ??, ?? : ?? Pattern Generation 30% of Train Data ?? : ?? ?? (??) Preprocessing?? ?? ?? ?? Story key value (??) tagloc tagdate tagmenu ?? Model Train Evaluation tagloc tagdate tagmenu ?? tagloc tagdate ?? tagdate tagmenu ?? tagloc tagmenu ?? Predictiontagloc tagdate ?? tagmenu Hyper parameter Selection ?? = ??
  • 24. Data flow for Model in Deep Learning (NER - BIO) ??? ?? ?? ???? Story Definition tagloc tagdate tagmenu ?? BIO-Mapping Preprocessing?? ?? ?? ?? B_Loc / B_Date / B_menu Model Train(Bi-LSTM) B-loc B-date B-menu O B-loc B-date O B-date B-menu O B-loc B-menu O Text Generator Pattern Matching tagloc tagdate tagmenu ?? tagloc tagdate ?? tagdate tagmenu ?? tagloc tagmenu ?? W2V 30% of Train Data Evaluation Prediction?? ?? ?? ?? Hyper parameter Selection ?? : 0.12 ?? : 0.7 ?? : 0.3 ???? B_loc B_Date B_menu O
  • 26. Intent? ????? (Text Classification) ???? ?? ?? / ?? ?? ??? / ?? ???? ??, ??, ??? 3?? ?? ?? ? Word???? ??? ??? ?? ??? ??? ?? ex) ?? ???? ?? / ?? ??? ???¡­. Deeplearning? ???? ?? ???? ?? ? ? ?? Char + CNN?? ????? (CNN - Feature ??, ??, ??) (Word Similarity ??, ?? / ??, ????)
  • 27. Intent? ????? (Text Classification - Data ??) Word ?? ?? ?? ?? Vector? ??? ?????? PIJA JUMUN HAGO SIPO ??, ????,??? ?? ????? W2V(Pretrained) ?? (0.12, 0.54, 0.72) ??(0.56, 0.65, 0.64) ??(0.67, 0.91, 0.13) ??(0.89, 0.14, 0.11) Ont Hotencoding (Word?? or ????) (0100000000) (0000010000) (0010000000) (0000000100) Ont Hotencoding (A~Z Vector) (0100000000) (0000010000) (0010000000) (0000000100)
  • 28. Char CNN? CNN? ????? ???? ??? ???? ????? ?? ??? ???? ??? Vector?? ???? Vector? ???? ???? Feature? ??? ? ??
  • 29. Text Classification - Char CNN ?? ?? ?? ?? ?? [?? Convolutional Neural Networks for Sentence Classification - Yoon Kim - https://arxiv.org/abs/1408.5882] ?? ?? ?? Feature ?????? [3,4,5 filter] Vector (W2V) ??/??/??? Static / Non Static / Random pooling ??? classification ?? Char-CNN? ???? ??? ?????
  • 30. Why Char-CNN?? Char-CNN? ???? ?? ????? ???? ??? ??? ?? ?? Convolutional Neural Networks for Sentence Classification - Yoon Kim - https://arxiv.org/abs/1408.5882
  • 31. RNN? ?? ?? ??? Data? ?? ???? ?? ???? ???? ?? ??? Backpropagation? ??? ???? ??(BPTT) http://aikorea.org/blog/rnn-tutorial-3/
  • 32. http://cs231n.stanford.edu/slides/2016/winter1516_lecture10.pdf Seq2Seq (RNN+RNN) ?? Chatbot??? Generator? ?? Sentence Generator ?? ???? ???? ???? ???? ? ?? (??? ???? input/output??)
  • 34. http://cs231n.stanford.edu/slides/2016/winter1516_lecture10.pdf LSTM? ?? ?? Cell State https://brunch.co.kr/@chris-song/9 updateforget out cell state
  • 36. Named Entity Recognition ???? Bidirectional LSTM (??? Layer) - RNN??? ?? - ????? ?? ??? ??? ?? ??? ?? ??? ?? ?? ???? ???? ?? [ ??? ???? ???? - https://sites.google.com/site/2016hclt/jalyosil]
  • 37. Why Bi-LSTM CRF ? [ Bidirectional LSTM-CRF Models for Sequence Tagging - https://arxiv.org/pdf/1508.01991.pdf ]
  • 38. ?? ???? ?? B-Pizza B-Order O O ?? ?? ??? B-Travel B-Information O ?? ???? B-Hotel B-Reserve O Named Entity Recognition ???? brat? ?? BIO Tagging B-???? I-???? ?? O-????, ??(OUT) U-Unknown (Word Embedding? ???) ¡ùNew York?,????? Brat - http://brat.nlplab.org/examples.html / https://wapiti.limsi.fr/
  • 39. Bi-LSTM?? ?? ?? -> ?? ?? ?? ???? ?? B-Pizza B-Order O O ?? ?? ??? B-Travel B-Info O ?? ???? B-Hotel B-Reserve O ??? ???? ?? ??? ?? ??? ?? ???? ?? ?? ?? Bi-LSTM? ??? ?? ??? ???? ??Data? ???? ??? ??? ????? ?? ??
  • 40. ????? with Ontology ??? ?? ?? ??? ???? ???? ??? ? ?????? ??? ??DB? ??? ?? ??? ????? ??? ?? ?? ???? ? ??? ?????? ??...
  • 41. Ontology with syntaxnet Neo4j - https://github.com/TensorMSA/tensormsa_jupyter/blob/master/chap13_chatbot_lecture/6.Ontology(Neo4J).ipynb Ontology??? ?? Neo4j DB? ?? ?????? ?? ???? ??? Syntaxnet? ?? A?B?? B? C?? ?? : A? C??
  • 43. ChatBot Layer Log File Chatbot Architecture Deep Learning Layer ?? ChatBot Layer ? ?? Application Layer ? ???? ? Application Layer ? ??? ??? DL Layer ? ??. DeepLearning Layer Bi-LSTM Char-CNN Seq2Seq NAS File Model Bot DB Residual NLP Context Analyzer Decision Maker Response Genertor ¡ù ?????? ?? Residual?? ?? ?? ?? Bot Builder GPU Deeplearning Predict Dict File Bot config Train Train Intent / NER
  • 44. NLP Architecture Preprocessing Python 3.5 Konlpy Mecab (Sejong Corpus) Tensorflow 1.1 AutoEncoder Char-CNN Bi-LSTM CRF Gensim FastText User-Dic Synonym Voting Python 3.5 API Service (Swagger) ?? ???? ???ICT? ????? ?? ???? ??... [Intent ??] ?? ?? [NER ??] ?? - Loc ???ICT - Loc ??? - Name ???? ('???'ICT'', 'NNP'), ('???', 'NNP'), ¡ùMecab ?????? ?? ?????? , ????(...) ?? ?? ?? ?? ?? [('??', 'NNG'), ('??', 'NNG'), ('?', 'XSV'), ('?', 'ETM'), ('???'ICT'', 'NNP'), ('?', 'JKB'), ('???', 'NNP'), ('??', 'JKB'), ('??', 'NNG'), ('??', 'NNG'), ('?', 'XSV'), ('?', 'EC'), ('?', 'VX'), ('?', 'EC')] Intent Slot ? ?? ?? ???? Slot? Entity? NER ??? Input Data=¡¯¡¯ ?? ???? ???ICT? ????? ?? ???? ??¡­¡± Intent=¡¯????¡¯ Intent_History=[¡®????¡¯,¡¯¡¯] story_slot_entity { ¡®??¡¯:??¡¯¡¯, ¡®??¡¯ : ¡¯?? ???ICT¡¯, ¡®??¡¯ : ¡®???¡¯} request_type=¡¯text¡¯ service_type=¡¯pizza order¡¯ output_data=¡¯¡¯ } Meta
  • 45. Docker (Ubuntu) in AWS EC2 (c4.8xlarge / p2.xlarge GPU) NAS DB Server Bot Builder (analysis) React Chatbot Server (Django) Python 3.5 Tensorflow Postgres SQL Bootstrap Web Service Architecture D3 SCSS Konlpy Nginx Celery Log File Model File Rabbit MQ Service Java Node Python Rest Gensim Front-End Java (Trigger) Rest LB Rest AP2 GPU Server (HDF5) GPU Server (HDF5) Dict File Hbase
  • 48. Rest API Client Input Data=???? ?? ???? Intent=¡¯¡¯ Intent_History=[¡® ¡¯,¡¯¡¯] story_slot_entity { ??:¡¯¡¯, ???:¡¯¡¯, ???:¡¯¡¯ } request_type=text service_type=¡¯¡¯ output_data=¡¯¡¯ Server Input Data= ???? ?? ???? Intent=???? Intent_History=[¡®????¡¯,¡¯¡¯] story_slot_entity { ??:??, ???:??, ???: ?? } request_type=text service_type=¡¯¡¯ output_data=???? Chatbot API ¡ù json? ??? ???? log??? ??
  • 49. Case? Test Coverage ?? ?? 1. ?? ?? (?????) 2. Model ?? (Hyper Parameter) 3. Data ??(Slot, Dict, Entity,???) 4. ?? ? ?? (Threshold, Rule??) ?? ?? ???? ??? Data? Model? ????? ??? ?? ? ? ?? ?? ?? ?????? ???? ??? ??? Continous Integration? ?? (Jenkins / Travis CI?) Test Codes for Chatbot ???? ???? ????->NER??-> Slot?? ???? input ??? ?????? -> intent : ???? slot : {??,??,???-extra}
  • 51. ??? ???? ??? ?? ???? ??? ???? ?? (Scoring / Voting) ??? ?? ?? ????? ???? ?? ??? ?? ??? Textming? ???? ???? ???? ??? ???ICT? ?? ?? ???? Char-CNN VotingSVM(Radial Basis/linear) Result naive_bayes.MultinomialNB (TF-IDF) ? ??? Slot ?? ??? ??? ??,??? ?? ???? ???? ???? ?? ?? Ensemble and Voting ??? ??? Slot ?? ?? ??
  • 52. Trigger ?? (??, ??? ??) 1. ????? ??? ?? <?? ?? ??> ?? : XXX ???? ????? ??? ??? ?? : ?? ?? ???? ???. ?? : ?? ?? ? ??? ?? ?? : ????? ?? ? ???? ???. ?? : ???? ?? : ??? [??, ??, ???] ?? Trigger? ???? ?? ??? Data? Seq2Seq??? ???? NLP??? ??? ?? 2.??? ?? ?(ResNet Model Call)
  • 53. ??? Tone Generater? ?? ??? ????? (???, ??? , ???) ??? ??????? (??) ??? ?????? (??) ??? ?????? (??) ??? ??????? (??) Seq2Seq Model?? - Encoder? ??? ?? Decoder? ??+?? ?? Response Generator? ?? ??? ???? ?? https://github.com/TensorMSA/tensormsa_jupyter/blob/master/chap13_chatbot_lecture/4.Chatb ot(Response-seq2seq).ipynb
  • 54. ??? ??(N-Gram) ???? - Pepperoni, ????, ????..... / Mac Book Pro - ???, ????... ???? ??? ??? ???? API????? ?? ??? ?? ? N-Gram? ???? ???? ??? ??? Dict? ?? ?? (??? trigram) ?? https://www.simplicity.be/article/throwing-dices-recognizing-west-flemish-and-other-languages/ ? Entity? N? Threshold ?? ???? ?? ¡ù threshold : ???? ???? ??
  • 55. Response Speed LB ?? Nginx ?? ??? ?? Thread? AP Caching of Data (Memory - API??) Chatbot?? ???? ?? MAX Time??
  • 56. ??? ?? ??? ?? Coding tf.device? ?? ??? Device? ?? CPU? GPU? ??? ?? GPU? ??? ??? ????...
  • 57. ??? ¡ñ ??? ??? ??? Hot? ??? ??? ????? ???? Domain? Data? ??? ?? ???? ? ??? ? ?? ??? ¡ñ ??? Data? ?? Data? ??? ????? ?? ??(???) ¡ñ ???? ??? ??? Data? ??? ??? ¡ñ ???? ????? ?? ??? ?? ??? ? ? ??
  • 59. Reference ??? ?? ??? http://hunkim.github.io/ml/ ?28? ?? ? ??? ???? ?? ?? https://sites.google.com/site/2016hclt/jalyosil Stanford University CS231n http://cs231n.stanford.edu/ Creating AI chat bot with Python 3 and Tensorflow[???] https://speakerdeck.com/inureyes/building-ai-chat-bot-using-python-3-and-tensorflow ????? ??_??? [???] /KimSungdong1/20170227-72644192?next_slideshow=1 ???? ??? ?? ???? ?? [???] http://www.slideshare.net/deview/221-67605830 Developing Korean Chatbot 101 [???] /JaeminCho6/developing-korean-chatbot-101-71013451 Tensorflow-Tutorials https://github.com/golbin/TensorFlow-Tutorials
AboutCopyright
? 2025 ºÝºÝߣ