??? ?? ??? ??
- 1. ??? ?? ??? ?? ???? ??? ??
- 2. pooling ReLU Momentum dropout ??? ?? ??? Tanh Softmax ??? Learning rate VGG Sigmoid backpropagation
- 4. ??? ?? ?? ???? ???? ?? ? ?? ??/?? ???? ?? Backpropagation 3?? ??? CNN, RNN
- 5. ?????? ｀?? ???? ??? ???? ??¨ Data? Model? ??? ????
- 6. ???? ?? ?? 1957 ?? ??? ?? (Perceptron) ??
- 7. ???? ?? ?? 1957 1986 ?? ??? ?? (Perceptron) ?? ? ???(30?)
- 8. ???? ?? ?? 1957 1986 ?? ??? ?? (Perceptron) ?? ? ???(30?) 1969 ?? ?? ???? ?? Perceptron? ???? ??? ? ??? ??? ? ??? ?? ??? ???? ??????, ??? ????? ????? parameter ?? ??? ?? ??? 10? 20?
- 9. ???? ?? ?? 1986 ??? ?? ?? ?? Data? Model? ??? ???? Backpropagation ?? ? ?? ?????? ???? ?? ??? ??? ?? ?? ??? weight? bias? ???? ?? ??
- 10. ???? ?? ?? 1986 ??? ?? ?? ?? 1990s BOOM ?????2 ??? ?(1991)
- 11. ???? ?? ?? 1990s ?? ?? ??? ???(10?) ???? ????? ??? ??? ?? ? ??. ??? ??? ???? ?? ?? ??? ??? ?? ??? ??? 10?? ?? 1. Overfitting 2. Vanishing gradient 3. Too slow SVM, Random Forest ??
- 12. ???? ?? ?? 1990s ?? ?? ?? ?? ??? 2000s 3?? ??? ???? ?? 1.Overfitting 2.Vanishing Gradient 3.Too slow GPU? ??? ?? ?? ????? ?? BOOM
- 13. ??? ???? ?? 1957 ?? ??? ?? (Perceptron) ?? 2000s1969 ?? ?? ???? ?? 1986 ??? ???? (Backpropagation) 3?? ??? ???? ??? ??? 1 ?????? N´
- 14. ??? ???? ?? ??? ??? 1 ?????? N W a y a y W W Backpropagation ???? (Activation Function) f(a) ????, tanh, Sigmoid, ReLU, ELU ? Softmax(a) Drop out Weight Update ??? (Optimization) SGD, AdaGrad, Momentum, Adam ? ??? (Normalization) ???? (Loss Function) Batch Size Learning rate epoch? ?? (Layer size) ?? ?? (Unit size)
- 15. ??/?? ???? ??
- 16. ??/?? ???? ?? ???? ?? ???? ???? Output = x1*w1 + x2*w2 + bias ??? #input #weight #bias #output #activation_function x1 Output bias x2 w1 w2
- 17. Weight? bias? ???? ?? ???? ?? ??! x1 Output bias x2 w1 w2 ??/?? ???? ?? ???? ?? ???? ???? ??? #input #weight #bias #output #activation_function
- 18. Weight? bias? ???? ?? ???? ?? ??! x1 Output bias x2 w1 w2 ??/?? ???? ?? ???? ?? ???? ???? ??? #input #weight #bias #output #activation_function
- 19. ??? #hidden_layer h(f(x)) bias yx1 x2 ??? ??? ??? h(x1*w1 + x2*w2 + bias*1) #activation_function ??/?? ???? ?? ???? ??, Deep Learning ????? (Activation Function) ????, tanh, Sigmoid, ReLU, ELU ? ????? (Activation Function) softmax ???? 2? ?? = Deep Learning
- 20. ??? #input #weight #bias #output #activation_function Output = h(x1*w1 + x2*w2 + x3*w3 + bias) x1 bias a x2 w1 w2 Activation Function (??? ??) y h( ) ? ??? ??? ???? ??/?? ???? ?? ???? ?? ???? ????
- 21. ??? #input #weight #bias #output #activation_function Output = h(x1*w1 + x2*w2 + x3*w3 + bias) x1 bias a x2 w1 w2 Activation Function (??? ??) y h( ) ??? ??? ?? ???, Linear regression? ???. (??? y = A?X + b? ??) ??/?? ???? ?? ???? ?? ???? ????
- 22. ??? #hidden_layer Output ??? ??? ??? x1*w11 + x2*w12 + bias1 #activation_function ??/?? ???? ?? ???? ??, Deep Learning x1*w21 + x2*w22 + bias2 x1*(w11 +w21) + x2*(w12 + w22 ) + (bias1 +bias2)
- 23. ??? #input #weight #bias #output #activation_function Output = h(x1*w1 + x2*w2 + x3*w3 + bias) x1 bias a x2 w1 w2 Activation Function (??? ??) y h( ) ??? ??? ?? ??? ????, Linear regression? ???. (??? y = A?X + b? ??) ??/?? ???? ?? ???? ?? ???? ????
- 24. ??? #input #weight #bias #output #activation_function Output = h(x1*w1 + x2*w2 + x3*w3 + bias) x1 bias a x2 w1 w2 Activation Function (??? ??) y h( )??? ??(non-linear function) ???? ?? ??? ?? ??? ???? ??? ??? ??? ?? ??/?? ???? ?? ???? ?? ???? ????
- 25. ??? #input #weight #bias #output #activation_function ylinear linear linear sigmoidx2 x1 x3 Q. ???? ???? ??, ???? ????? ?? ??/?? ???? ?? ???? ?? ???? ????
- 26. ??? #input #weight #bias #output #activation_function ylinear linear linear sigmoidx2 x1 x3 Q. ???? ???? ??, ???? ????? ?? ??/?? ???? ?? ???? ?? ???? ???? ???? ?? ??
- 27. Non-linear ??? ??? ???? ?? ???? ??? ??? ??? ??? ??
- 28. ???? ???? ??? ??? ?? ??? ??? ?? ? ?? ????? ??? ???? ???? NN(???3?) NN(???50?)
- 29. 3?? ???
- 30. 1. Overfitting 2. Vanishing Gradient 3. Too slow ???? ???? ????
- 31. 1. Overfitting 2. Vanishing Gradient 3. Too slow ???? ???? ????
- 32. ??? #overfitting #vanishing_gradient 3?? ??? Overfitting ???
- 33. ??? #overfitting #vanishing_gradient ?? ????? ? ?? ?? ? ???? CASE 1 CASE 2 3?? ??? Overfitting ???
- 34. ???? ??? ?????, ??? ???? ??? ??? #dropout 3?? ??? Overfitting ?? ??
- 35. Output ??? #dropout 3?? ??? Overfitting ?? ??
- 36. Output ??? #dropout 3?? ??? Overfitting ?? ??
- 37. Output ??? #dropout 3?? ??? Overfitting ?? ??
- 38. 1. Overfitting 2. Vanishing Gradient 3. Too slow ???? ???? ????
- 39. 1. Overfitting 2. Vanishing Gradient 3. Too slow ???? ???? ???? ???? ???
- 40. ???? ?? ???? ????
- 41. Backpropagation
- 42. ??? #backpropagation input Output input Forward Propagation ∠ Backpropagation (｀?? ??¨? ???? ??) ?? ?? (?? ??) ⊥ weight, bias ?? Backpropagation ???? ?? ??
- 43. ??? #backpropagation input Output input Forward Propagation ∠ Backpropagation (｀?? ??¨? ???? ??) ?? ?? (?? ??) ⊥ weight, bias ?? Backpropagation ???? ?? ??
- 44. ??? ?? -???? ???? ?? ???? ????? ??? = 0? ???? ?? ??? ?? -???? ???? ?? ????? ???? ??? #backpropagation Backpropagation ｀?? ??? ???¨?
- 45. ｀?? ??? ???¨ = ??? ?????? = f(?????, ???) 1 0 0.5 Vanishing gradient? ?? ???? ?? 0? ????, ??? ??? ???? ??? ? ??? ??? ? ??? ???? ? 3?? ??? Vanishing Gradient?? 0.25 0 sigmoid Sigmoid? ?? ??
- 46. 1 0 0.5 Sigmoid 1 0 - 1 Tanh ReLU(Rectified Linear Unit) h(x) = x (x ? 0) 0 (x ＋ 0) ?? ????? ???? 1 3?? ??? Vanishing Gradient ???? ?dead neuron? ????? ?????? ?? ?? 6?? ???!
- 47. 1. Overfitting 2. Vanishing Gradient 3. Too slow ???? ???? ????
- 48. ??? #cost_function #gradient_descent #learning_rate ??? ??? ???? bias, weight? ?? ? ???? inp ut Outpu t inp ut ∠ Backpropagation (｀?? ??¨? ???? ??) ?? ?? ⊥ weight, bias ?? ? ???? ??? cost function? ???(gradient)? ??? cost? ??? ??(descent)?? ?????? ?? ?? ??(learning rate)? ???? ??. W = W - α?J(W,b) 3?? ??? ?? ???
- 49. ??? #cost_function #gradient_descent #learning_rate 3?? ??? ?? ??? ??? ?? -???? ???? ?? ???? ????? ??? = 0? ???? ?? ??? ?? -???? ???? ?? ????? ???? ?? ??? ?? ???? ??? ? ??
- 50. ?? ´ ??! Step 1. ?? parameters? train set ?? ??? ?? ?? ?? Step 2. ?? ??? ???? ??? ?? ? ?? ?? ?? Step 3. ?? ?? ???? ???? Step 4. ??? ??? ??? ? ?? parameter? ?? Step 5. ??? parameter? ??? ?? train set ?? ???? ?? ?? ?? Step 6. ?? ´ ?? ???? bias? weight? ?? ??? ??? #cost_function #gradient_descent #learning_rate 3?? ??? ?? ???
- 51. SGD(Stochastic Gradient Descent) -??? ?? ??? ｀????? ???? ??? ???¨? ?? ???? ?? ??? https://www.coursera.org/learn/machine-learning/lecture/DoRHJ/stochastic-gradient-descent
- 52. ??? #SGD Gradient Descent Stochastic Gradient Descent Mini-batch Gradient Descent W = W-α?J(W,b) W = W-α?J(W,b,x(z),y(z))W = W-α?J(W,b,x(z:z+bs),y(z:z+bs)) 1?? 1?? 1?? 1?? 1?? 1?? 1?? ´ ´ 3?? ??? Stochastic Gradient Descent
- 53. ??? #SGD 3?? ??? SGD? ??? SGD?? ??? ??? ??? ??? ??? ?? ??? ??
- 54. ????? ????? ?? ?? ???? ???? ???? ?? ???? ????? Local minimum? ??? ?? Local minima Global minimum ??? #learning_rate 3?? ??? SGD? ?? #momentum ????? ???? ??? Optimizer ?? ????,
- 55. SGD ?? ??? Momentum NAG Adagrad Adadelta RMSProp Adam ???? ? ????, ? ?? ?? ?? ?? ?? ?? ???? ???? ??, ?? ??? ???? ???? ? ????, ??? ? ??? ???? ?? ???? ?? ?? ? ?? ?? ?? ?? ? ?? ??? Momentum?? ?? ? ?? gradient ?? ?? ???? ?? ????? ??? ?? 3?? ??? Optimizer ??? ??
- 56. ??????
- 57. ReLUinput Softmax input ReLU ReLU ReLU ReLU ReLU ReLU ReLU ???? ?? ??: Backpropagation cross entropy 1. ?? ?? ??: Dropout 2. ??? ??: ReLU, softmax dropout 3. ?? ?? ??: ReLU4. ?? ?? ??: Adam
- 58. Coming Soon
- 59. ?????.