Forward-Forward Algorithm

Jan 18, 20231 like704 views

Dong Heon Cho

Forward Forward by Hinton

What is wrong with backpropagation
The Forward-Forward Algorithm: Some Preliminary Investigations
Neurips 2022

What is wrong with backpropagation
> Biological Perspective
- Error derivative propagation? ???, neural activity?? ???? ??? cortex?? ??? ? ??
=> ??? “later time step”?? ????? ?? ???, real-time? ???? ????? ?? ??
> Computational Perspective
- backpropagation? “?? forward pass”? derivative? ??? ? ??? ? (ex. rnn ???? ? ??
sequence? recurring?? ??? ????? ? ??)
=> ??? ??? reinforcement learning? ? ? random weight?? ???? ??? high variance
??? ????

forward-forward algorithm
- Blotzmann machine? ??? greedy multi-layer learning procedure
Forward-Forward algorithm ?? ??
1. Goodness function for one layer: Weight? ????? ? ??? objective function
2. Forward (Positive), Forward (Negative): Network procedure

forward-forward algorithm
Goodness function
constant threshold
feature
logistic function

forward-forward algorithm
Forward (positive) - Forward (negative)
1 layer
Expected Output
: Positive Input => Output > threshold
: Negative Input => Output < threshold
Input

forward-forward algorithm
Sum-up!
- “Positive Sample”? forward?? network signal? “threshold” ??? response,
“Negative Sample”? forward?? signal? “threshold” ??? response? ?????

Negative data for Forward-Forward
Unsupervised Task
1. Create “Random” Binary Mask
2. ?? ???? ??? ??? Mask? ?? ?? sum
=> Negative Data? ???? “characterize shape”? ???? ??
??

Negative data for Forward-Forward
Unsupervised Task
=> ??? ???? ???, Positive Data? MNIST? ???? Negative Data? hybrid? ???
4 layer (for each 2000 feature) ????? ?? ?? 2,3,4 layer? feature? Linear Classifier
??? ???? 1.37% error rate in MNIST? ???
- (??? backprop FC ????? 1.4%, dropout/label smoothing ?? ???? 1.1%)
- (First layer? feature? linear classifier? ???? ?? ??)
=> ????? convolution kernel? ???? Linear Classifier ?? ? 1.16% error rate in MNIST
*?? following ???? negative data? “? ???
??”?? ?? ?
Forward Positive
Forward Negative Linear
Classifier
Sample/
Hybrid

Negative data for Forward-Forward
Supervised Task
Unsupervised?? ? ??? ??? sample? ??? ??, ???? label? ?? ??? ???
+ Label ??? “Input”? ??? Forward? ?!
=> ????? Label? Image ??? Correlation? ??? ??

Negative data for Forward-Forward
Supervised Task
- Supervised Task? Forward Forward? ??? ??
for i in range(num_class):
input = i Class Vector? ??? + sample
network? ?? ????? output? ?? accumulate
=> accumulated? ?? ?? “?” class? ??? class
Forward Positive
Forward Negative
Class Vector
+ Sample
=> Class? 10???,
10? Inference
???!
~ 1.36% error rate in FC with FF
~ 0.64% error rate in CNN with FF

Exp in CIFAR 10
*Network: 3 layer with 3072 ReLU each
- Compute goodness for every label: ? ????? Input
vector? ???? inference (supervised task)
- one-pass softmax: (unsupervised task)
=> min/max ssq? goodness function? minimize, maximize? ??
== network output? “threshold” ??? ?? ???, ?? ??
???
=> BP? “overfitting”?? ???? ??

Pros & Cons
- Pros
~ Backprop? ???? Full derivatives? ???? ???? ?????, forward-forward? ??? ??
??? ?? ??? ???? ???
~ Trillion? ????? ???? ???? ??? watts? ????, forward-forward? “mortal
computation”??? ???? ???? ?? (* hardware efficiency? ???? ?)
- Cons
~ ?? backpropagation?? ??? ???, generalize ??? ??? ???? ?? (backprop ???
???)
~ Big Model? Big Data? backpropagation? ?? ? (??? ?? ?????? ???? ? ??
???…?)

Future Works
- Negative Forward? Positive Forward?? ??
- Negative forward ?? Positive Forward? ????
- Goodness function? ?? ??? ??? ???
- ReLU ?? activation function? ??????? ???? (t-distribution ?)
- Forward-Forward? ?? ????
- …

ref
https://medium.com/mlearning-ai/pytorch-implementation-of-forward-forward-algorithm-by-geoffre
y-hinton-and-analysis-of-performance-7e4f1a26d70f
https://www.quantamagazine.org/artificial-neural-nets-finally-yield-clues-to-how-brains-learn-2021
0218/
https://bdtechtalks.com/2022/12/19/forward-forward-algorithm-geoffrey-hinton/
https://github.com/mohammadpz/pytorch_forward_forward
https://www.cs.toronto.edu/~hinton/

? ??? ??? ??? ?? ??? ??? ?????, ? ??? ??? 'Forward-Forward ????'???. ??? ??? ??? ???? ??? ??? ??? ?? ?(backward)? ??? '?-?' ????, ? ??? ??? '?-?' ??? ????? 'Forward-Forward'?? ????. ? ??????? '?? ???'? '???? ???' ? ??? ?????. '?? ???'? ??? ??? ?? ??? ????, '???? ???'? ???? ??? ???? ??? ????. ? ? ??? ???? ?? ??? ???, ? ??? '?? ???'? ??? ?? ???, '???? ???'? ??? ?? ??? ??? ?????. ??? ???? ?? ??? ??? ??, ? ????? ???? ??? ??? ? ??? ??? ????. ? ??? ???? ??? ?? ???? ???? ?? ????? ??? ??? ????? ???? ??? ?? ?? ??? ? ??? ?? ? ?????.

1???? GAN(Generative Adversarial Network) ?? ????NAVER Engineering

???: ???(??? ????) ??? (Yunjey Choi)? ??????? ?????? ??????, ??? ?????? Machine Learning? ???? ?? ????. ??? ???? ??? ?? ?? ????? ???? ?? ????. 1? ? TensorFlow? ???? Deep Learning? ????? ??? PyTorch? ???? Generative Adversarial Network? ???? ??. TensorFlow? ?? ???? ??, PyTorch Tutorial? ??? Github? ??? ??? ?? ??. ??: Generative Adversarial Network(GAN)? 2014? Ian Goodfellow? ?? ???? ??????, ??? ??? ?? ?? ???? ??? ???? ?? ?????. ?? ?? GAN? ?? ???? ?? ??? ???? ?? ???? ? ?? ?? ???? ??? ??? ????. ? ?? ??? ??? ?? GAN ???? ? ??? ?????? ?????. ???? GAN? ???? ????? ?? ??? ???? ?? ??? ? ????. ?? ??? ?? ?? GAN? ?? ?? ?? ?? ??? ??????? ???. GAN? ?? ???? ??, GAN? ?? ???? ??? ????? ??, GAN? ??? ??? ? ??? ????? ??? ??? ??? ?? ? ????. ????: https://youtu.be/odpjk7_tGY0

?????? ?? ?NAVER Engineering

???: ???(NAVER) ???: 2017.11. ?? ??? ??? ?????? ??????? ??? ?? ??? ?? ?? ????. ? ????? ?????? ?? ???? ??? ?????? ?? ?? ??? ????? ???. ?? ?????? ?? ?? ????Autoencoder? (AE) ? ?? ?? Denoising AE, Contractive AE? ??? ??? ???, ??? ?? ???? ?? ?? ?? Variational AE? (VAE) ? ?? ?? Conditional VAE, Adversarial AE? ??? ??? ????. ??, ?????? ??? ?? ??? ?????? ???? ??? ????? ??? ????. 1. Revisit Deep Neural Networks 2. Manifold Learning 3. Autoencoders 4. Variational Autoencoders 5. Applications

敌対的生成ネットワーク（骋础狈）cvpaper. challenge

cvpaper.challenge の Meta Study Group 発表スライド cvpaper.challenge はコンピュータビジョン分野の今を映し、トレンドを創り出す挑戦です。論文サマリ?アイディア考案?議論?実装?論文投稿に取り組み、凡ゆる知識を共有します。2019の目標「トップ会議30+本投稿」「2回以上のトップ会議網羅的サーベイ」 http://xpaperchallenge.org/cv/

[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB ImagesDeep Learning JP

【論文紹介】How Powerful are Graph Neural Networks?Masanao Ochi

Wasserstein GAN ?? ???? ISungbin Lim

[DL輪読会]Model-Based Reinforcement Learning via Meta-Policy OptimizationDeep Learning JP

贬测辫别谤辞辫迟とその周辺についてKeisuke Hosaka

【論文読み会】Self-Attention Generative Adversarial NetworksARISE analytics

【LT資料】 Neural Network 素人なんだけど何とかご機嫌取りをしたいTakuji Tahara

[Kaggle Tokyo Meetup #6](https://connpass.com/event/132935/) での LT 資料です。短いのでタイトル詐欺感ありますがお許しください。内容的には学習率って大事だよねと言うお気持ちが全て。【追記】 p.16 の「Adam 固定でうまく行く人々もいる」について、流石に scheduling はしてるのではないかと(現地で)コメントを頂きました。"固定"はあくまで optimizer の話であって、scheduling なしの Adam はそこまで有能ではないとのこと。これは僕の経験とも一致しています。

Generative Models（メタサーベイ）cvpaper. challenge

cvpaper.challenge のメタサーベイ発表スライドです。 cvpaper.challengeはコンピュータビジョン分野の今を映し、トレンドを創り出す挑戦です。論文サマリ作成?アイディア考案?議論?実装?論文投稿に取り組み、凡ゆる知識を共有します。2020の目標は「トップ会議30+本投稿」することです。 http://xpaperchallenge.org/cv/

教師なし画像特徴表現学習の動向 {Un, Self} supervised representation learning (CVPR 2018 完全読破...cvpaper. challenge

CVPR 2018 完全読破チャレンジ報告会 cvpaper.challenge 勉強会@Wantedly白金台オフィス cvpaper.challenge はコンピュータビジョン分野の今を映し、創り出す挑戦です。論文読破?まとめ?アイディア考案?議論?実装?論文執筆（?社会実装）に至るまで広く取り組み、あらゆる知識を共有しています。 http://xpaperchallenge.org/cv/

【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)Deep Learning JP

This document summarizes recent research on applying self-attention mechanisms from Transformers to domains other than language, such as computer vision. It discusses models that use self-attention for images, including ViT, DeiT, and T2T, which apply Transformers to divided image patches. It also covers more general attention modules like the Perceiver that aims to be domain-agnostic. Finally, it discusses work on transferring pretrained language Transformers to other modalities through frozen weights, showing they can function as universal computation engines.

グラフニューラルネットワーク入门ryosuke-kojima

[DL輪読会]SOLAR: Deep Structured Representations for Model-Based Reinforcement L...Deep Learning JP

Positive-Unlabeled Learning with Non-Negative Risk EstimatorKiryo Ryuichi

[DL輪読会]Reinforcement Learning with Deep Energy-Based PoliciesDeep Learning JP

SSII2020SS: グラフデータでも深層学習 ? Graph Neural Networks 入門 ?SSII

SSII2020 技術動向解説セッション SS1 6/11 (木) 14:00～14:30　メイン会場 (vimeo + sli.do) グラフ構造をもつデータに対する DNN、すなわち Graph Neural Networks (GNNs) の研究はこの２、３年で参加する研究者が急増している。現状、様々なアーキテクチャの GNN が様々なドメインや様々なタスクで個別に提案され、概観を捉えるのも簡単ではない状態になっている。本チュートリアルは、広範に散らばった GNN 研究の現状についての概観と基盤技術を紹介するとともに、時間が許す範囲でコンピュータビジョン領域における応用例の紹介にも取り組みたい。

畳み込みニューラルネットワークの高精度化と高速化Yusuke Uchida

2012年の画像認識コンペティションILSVRCにおけるAlexNetの登場以降，画像認識においては畳み込みニューラルネットワーク (CNN) を用いることがデファクトスタンダードとなった．CNNは画像分類だけではなく，セグメンテーションや物体検出など様々なタスクを解くためのベースネットワークとしても広く利用されてきている．本講演では，AlexNet以降の代表的なCNNの変遷を振り返るとともに，近年提案されている様々なCNNの改良手法についてサーベイを行い，それらを幾つかのアプローチに分類し，解説する．更に，実用上重要な高速化手法について、畳み込みの分解や枝刈り等の分類を行い，それぞれ解説を行う． Recent Advances in Convolutional Neural Networks and Accelerating DNNs 第21回ステアラボ人工知能セミナー講演資料 https://stair.connpass.com/event/126556/

Point netFujimoto Keisuke

Batch normalization effectiveness_20190206Masakazu Shinoda

【DL輪読会】GPT-4Technical ReportDeep Learning JP

[DL輪読会]Few-Shot Unsupervised Image-to-Image TranslationDeep Learning JP

グラフニューラルネットワークとグラフ组合せ问题joisino

以下の二つの論文の紹介を中心に、グラフニューラルネットワークとグラフ组合せ问题の交わりについて解説しました。 SIG-FPAI での招待講演の内容に少し修正を加えたものです。 * Learning Combinatorial Optimization Algorithm over Graphs (NIPS 2017) * Approximation Ratios of Graph Neural Networks for Combinatorial Problems (NeurIPS 2019)

[DL輪読会] Spectral Norm Regularization for Improving the Generalizability of De...Deep Learning JP

翱辫迟颈尘颈锄别谤入门＆最新动向Motokawa Tetsuya

[DL輪読会] Residual Attention Network for Image ClassificationDeep Learning JP

1???? - ???? ?????????_ Hidden layer?? Backpropagation.pdfjyyyukk

??? ?? ??? ??Hee Won Park

More Related Content

What's hot (20)

贬测辫别谤辞辫迟とその周辺についてKeisuke Hosaka

【論文読み会】Self-Attention Generative Adversarial NetworksARISE analytics

【LT資料】 Neural Network 素人なんだけど何とかご機嫌取りをしたいTakuji Tahara

Generative Models（メタサーベイ）cvpaper. challenge

教師なし画像特徴表現学習の動向 {Un, Self} supervised representation learning (CVPR 2018 完全読破...cvpaper. challenge

【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)Deep Learning JP

グラフニューラルネットワーク入门ryosuke-kojima

[DL輪読会]SOLAR: Deep Structured Representations for Model-Based Reinforcement L...Deep Learning JP

Positive-Unlabeled Learning with Non-Negative Risk EstimatorKiryo Ryuichi

[DL輪読会]Reinforcement Learning with Deep Energy-Based PoliciesDeep Learning JP

SSII2020SS: グラフデータでも深層学習 ? Graph Neural Networks 入門 ?SSII

畳み込みニューラルネットワークの高精度化と高速化Yusuke Uchida

Point netFujimoto Keisuke

Batch normalization effectiveness_20190206Masakazu Shinoda

【DL輪読会】GPT-4Technical ReportDeep Learning JP

[DL輪読会]Few-Shot Unsupervised Image-to-Image TranslationDeep Learning JP

グラフニューラルネットワークとグラフ组合せ问题joisino

[DL輪読会] Spectral Norm Regularization for Improving the Generalizability of De...Deep Learning JP

翱辫迟颈尘颈锄别谤入门＆最新动向Motokawa Tetsuya

[DL輪読会] Residual Attention Network for Image ClassificationDeep Learning JP

贬测辫别谤辞辫迟とその周辺についてKeisuke Hosaka

【論文読み会】Self-Attention Generative Adversarial NetworksARISE analytics

【LT資料】 Neural Network 素人なんだけど何とかご機嫌取りをしたいTakuji Tahara

Generative Models（メタサーベイ）cvpaper. challenge

教師なし画像特徴表現学習の動向 {Un, Self} supervised representation learning (CVPR 2018 完全読破...cvpaper. challenge

【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)Deep Learning JP

グラフニューラルネットワーク入门ryosuke-kojima

[DL輪読会]SOLAR: Deep Structured Representations for Model-Based Reinforcement L...Deep Learning JP

Positive-Unlabeled Learning with Non-Negative Risk EstimatorKiryo Ryuichi

[DL輪読会]Reinforcement Learning with Deep Energy-Based PoliciesDeep Learning JP

SSII2020SS: グラフデータでも深層学習 ? Graph Neural Networks 入門 ?SSII

畳み込みニューラルネットワークの高精度化と高速化Yusuke Uchida

Point netFujimoto Keisuke

Batch normalization effectiveness_20190206Masakazu Shinoda

【DL輪読会】GPT-4Technical ReportDeep Learning JP

[DL輪読会]Few-Shot Unsupervised Image-to-Image TranslationDeep Learning JP

グラフニューラルネットワークとグラフ组合せ问题joisino

[DL輪読会] Spectral Norm Regularization for Improving the Generalizability of De...Deep Learning JP

翱辫迟颈尘颈锄别谤入门＆最新动向Motokawa Tetsuya

[DL輪読会] Residual Attention Network for Image ClassificationDeep Learning JP

Similar to Forward-Forward Algorithm (20)

1???? - ???? ?????????_ Hidden layer?? Backpropagation.pdfjyyyukk

??? ?? ??? ??Hee Won Park

ML + ?? phase 2HoChul Shin

Imagination-Augmented Agents for Deep Reinforcement Learning?? ?

13-DfdasdfsafdsafdasfdasfdsadfasfdsafNN.pptxHinPhmXun1

[Tf2017] day4 jwkang_pubJaewook. Kang

Nationality recognition?? ?

Convolutional rnnLee Gyeong Hoon

Coursera Machine Learning (by Andrew Ng)_????SANG WON PARK

??? ???? ???? ??, ?? ?? ? ?????? ???? ??? ??? ??? ????? ???? ???? ??. ?? week1 ~ week4 ??? ??? ???? "??? ?? ???"?? ?? ???? ???? ? ?? ????, ???? ??? ??? ???? ?? ??? ? ?? ????? ???? ??. ?? ??? ??? ???? ????? ????? ????, ? ???? ??? ???? ???? ??. ?? Octave code? ??? ????? ???? ??? ????? ? ??. Week1 Linear Regression with One Variable Linear Algebra - review Week2 Linear Regression with Multiple Variables Octave[incomplete] Week3 Logistic Regression Regularization Week4 Neural Networks - Representation Week5 Neural Networks - Learning Week6 Advice for applying machine learning techniques Machine Learning System Design Week7 Support Vector Machines Week8 Unsupervised Learning(Clustering) Dimensionality Reduction Week9 Anomaly Detection Recommender Systems Week10 Large Scale Machine Learning Week11 Application Example - Photo OCR

Deep learning overview??? ???

Attention is all you need ??Junho Lee

Rnn kerasPark Seong Hyeon

[paper review] ??? - Eye in the sky & 3D human pose estimation in video with ...Gyubin Son

[Paper review] neural production systemSeonghoon Jung

Densely Connected Convolutional NetworksOh Yoojin

Workshop 210417 dhleeDongheon Lee

???? ??? ??: Rainbow ???? ???? (2nd dlcat in Daejeon)Kyunghwan Kim

Survey of activation functions?? ?

AUTOML?? ?

Automl?? ?

1???? - ???? ?????????_ Hidden layer?? Backpropagation.pdfjyyyukk

??? ?? ??? ??Hee Won Park

ML + ?? phase 2HoChul Shin

Imagination-Augmented Agents for Deep Reinforcement Learning?? ?

13-DfdasdfsafdsafdasfdasfdsadfasfdsafNN.pptxHinPhmXun1

[Tf2017] day4 jwkang_pubJaewook. Kang

Nationality recognition?? ?

Convolutional rnnLee Gyeong Hoon

Coursera Machine Learning (by Andrew Ng)_????SANG WON PARK

Deep learning overview??? ???

Attention is all you need ??Junho Lee

Rnn kerasPark Seong Hyeon

[paper review] ??? - Eye in the sky & 3D human pose estimation in video with ...Gyubin Son

[Paper review] neural production systemSeonghoon Jung

Densely Connected Convolutional NetworksOh Yoojin

Workshop 210417 dhleeDongheon Lee

???? ??? ??: Rainbow ???? ???? (2nd dlcat in Daejeon)Kyunghwan Kim

Survey of activation functions?? ?

AUTOML?? ?

Automl?? ?

More from Dong Heon Cho (20)

What is Texture.pdfDong Heon Cho

The document discusses texture analysis in computer vision. It begins by asking what texture is and whether objects themselves can be considered textures. It then outlines several statistical and Fourier approaches to texture analysis, citing specific papers on texture energy measures, texton theory, and using textons to model materials. Deep convolutional neural networks are also discussed as being able to recognize and describe texture through learned filter banks. The concept of texels is introduced as low-level features that make up texture at different scales from edges to shapes. The document hypothesizes that CNNs are sensitive to texture because texture repeats across images while object shapes do not, and that CNNs act as texture mappers rather than template matchers. It also questions whether primary visual cortex

BADGEDong Heon Cho

This document discusses active learning techniques called Deep Badge Active Learning. It proposes using gradient embeddings to represent samples and k-means++ initialization for sample selection. Specifically, it uses the gradient embedding for feature representation, then performs k-means++ initialization to select samples by finding those with the maximum 2-norm and those farthest from existing samples, adding them to the set iteratively. This aims to select a diverse set of samples, similar to how binary search works. The technique could improve over entropy-based and core-set selection approaches for active learning with convolutional neural networks.

Neural Radiance FieldDong Heon Cho

Neural Radiance Fields (NeRF) represent scenes as neural networks that map 5D input (3D position and 2D viewing direction) to a 4D output (RGB color and opacity). NeRF uses an MLP that is trained to predict volumetric density and color for a scene from many camera views. Key aspects of NeRF include using positional encodings as inputs to help model view-dependent effects, and training to optimize for integrated color and density values along camera rays. NeRF has enabled novel applications beyond novel view synthesis, including pose estimation, dense descriptors, and self-supervised segmentation.

2020 > Self supervised learningDong Heon Cho

All about that poolingDong Heon Cho

The document discusses various pooling operations used in image processing and convolutional neural networks (CNNs). It provides an overview of common pooling methods like max pooling, average pooling, and spatial pyramid pooling. It also discusses more advanced and trainable pooling techniques like stochastic pooling, mixed/gated pooling, fractional pooling, local importance pooling, and global feature guided local pooling. The document analyzes the tradeoffs of different pooling methods and how they can balance preserving details versus achieving invariance to changes in position or lighting. It references several influential papers that analyzed properties of pooling operations.

Background elimination reviewDong Heon Cho

This document discusses background elimination techniques which involve three main steps: object detection to select the target, segmentation to isolate the target from the background, and refinement to improve the quality of the segmented mask. It provides an overview of approaches that have been used for each step, including early methods based on SVM and more recent deep learning-based techniques like Mask R-CNN that integrate detection and segmentation. The document also notes that segmentation is challenging without object detection cues and discusses types of segmentation as well as refinement methods that use transformations, dimension reduction, and graph-based modeling.

Transparent Latent GANDong Heon Cho

1. TL-GAN matches feature axes in the latent space to generate images without fine-tuning the neural network. 2. It discovers correlations between the latent vector Z and image labels by applying multivariate linear regression and normalizing the coefficients. 3. The vectors are then adjusted to be orthogonal, allowing different properties to be matched while labeling unlabeled data to add descriptions.

Image matting atocDong Heon Cho

Image matting is the process of separating the foreground and background of an image by assigning each pixel an alpha value between 0 and 1 indicating its transparency. Traditionally, matting uses a trimap to classify pixels as foreground, background, or uncertain. Early sampling-based methods calculated alpha values based on feature distances of closest foreground and background pixels. More recent approaches use deep learning, where the first deep learning matting method in 2016 took local and non-local information as input, and the 2017 Deep Image Matting method used an RGB image and trimap as input in a fully deep learning framework.

Multi object Deep reinforcement learningDong Heon Cho

This document discusses multi-objective reinforcement learning and introduces Deep OLS Learning, which combines multi-objective learning with deep Q-networks. It presents Deep OLS Learning with Partial Reuse and Full Reuse to handle multi-objective Markov decision processes by finding a convergence set of policies that optimize multiple conflicting objectives, such as maximizing server performance while minimizing power consumption. The approach is evaluated on multi-objective versions of mountain car and deep sea treasure problems.

Multi agent reinforcement learning for sequential social dilemmasDong Heon Cho

This document summarizes research on multi-agent reinforcement learning in sequential social dilemmas. It discusses how sequential social dilemmas extend traditional matrix games by adding temporal aspects like partial observability. Simulation experiments are described where agents learn cooperative or defective policies for tasks like fruit gathering and wolfpack hunting in a partially observable environment. The agents' learned policies are then used to construct an empirical payoff matrix to analyze whether cooperation or defection is rewarded more, relating the multi-agent reinforcement learning results back to classic social dilemmas.

Multi agent SystemDong Heon Cho

This document discusses multi-agent systems and their applications. It provides examples of multi-agent systems for spacecraft control, manufacturing scheduling, and more. Key points: - Multi-agent systems consist of interacting intelligent agents that can cooperate, coordinate, and negotiate to achieve goals. They offer benefits like robustness, scalability, and reusability. - Challenges include defining global goals from local actions and incentivizing cooperation. Games like the prisoner's dilemma model social dilemmas around cooperation versus defection. - The document outlines architectures like the blackboard model and BDI (belief-desire-intention) model. It also provides a manufacturing example using the JADE platform.

Hybrid reward architectureDong Heon Cho

The document discusses Hybrid Reward Architecture (HRA), a reinforcement learning method that decomposes the reward function of an environment into multiple sub-reward functions. In HRA, each sub-reward function is learned by a separate agent using DQN. This allows HRA to learn complex reward functions more quickly and stably compared to using a single reward signal. An experiment is described where HRA learns to eat 5 randomly placed fruits in an environment over 300 steps more effectively than a standard DQN agent.

Use Jupyter notebook guide in 5 minutesDong Heon Cho

AlexNet and so on...Dong Heon Cho

Deep Learning AtoC with Image PerspectiveDong Heon Cho

Deep learning models like CNNs, RNNs, and GANs are widely used for image classification and computer vision tasks. CNNs are commonly used for tasks like classification, detection, segmentation through learning hierarchical image features. Fully convolutional networks with encoder-decoder architectures like SegNet and Mask R-CNN can perform pixel-level semantic segmentation and instance segmentation by combining classification and bounding box detection. Deep learning has achieved state-of-the-art performance on many image applications due to its ability to learn powerful visual representations from large datasets.

LOL win predictionDong Heon Cho

How can we train with few dataDong Heon Cho

The document discusses approaches for using deep learning with small datasets, including transfer learning techniques like fine-tuning pre-trained models, multi-task learning, and metric learning approaches for few-shot and zero-shot learning problems. It also covers domain adaptation techniques when labels are not available, as well as anomaly detection for skewed label distributions. Traditional models like SVM are suggested as initial approaches, with deep learning techniques applied if those are not satisfactory.

Domain adaptation ganDong Heon Cho

The document discusses domain adaptation and transfer learning techniques in deep learning such as feature extraction, fine tuning, and parameter sharing. It specifically describes domain-adversarial neural networks which aim to make the source and target feature distributions indistinguishable and domain separation networks which extract domain-invariant and private features to model each domain separately.

Dense sparse-dense training for dnn and Other ModelsDong Heon Cho

Squeeeze modelsDong Heon Cho

This document discusses various techniques for compressing and speeding up deep neural networks, including singular value decomposition, pruning, and SqueezeNet. Singular value decomposition can be used to compress fully connected layers by minimizing the difference between the original weight matrix and its low-rank approximation. Pruning techniques remove unimportant weights below a threshold. SqueezeNet is highlighted as designing a small CNN architecture from the start that achieves AlexNet-level accuracy with 50x fewer parameters and less than 0.5MB in size.

What is Texture.pdfDong Heon Cho

BADGEDong Heon Cho

Neural Radiance FieldDong Heon Cho

2020 > Self supervised learningDong Heon Cho

All about that poolingDong Heon Cho

Background elimination reviewDong Heon Cho

Transparent Latent GANDong Heon Cho

Image matting atocDong Heon Cho

Multi object Deep reinforcement learningDong Heon Cho

Multi agent reinforcement learning for sequential social dilemmasDong Heon Cho

Multi agent SystemDong Heon Cho

Hybrid reward architectureDong Heon Cho

Use Jupyter notebook guide in 5 minutesDong Heon Cho

AlexNet and so on...Dong Heon Cho

Deep Learning AtoC with Image PerspectiveDong Heon Cho

LOL win predictionDong Heon Cho

How can we train with few dataDong Heon Cho

Domain adaptation ganDong Heon Cho

Dense sparse-dense training for dnn and Other ModelsDong Heon Cho

Squeeeze modelsDong Heon Cho

Forward-Forward Algorithm

1. What is wrong with backpropagation The Forward-Forward Algorithm: Some Preliminary Investigations Neurips 2022

2. What is wrong with backpropagation > Biological Perspective - Error derivative propagation? ???, neural activity?? ???? ??? cortex?? ??? ? ?? => ??? “later time step”?? ????? ?? ???, real-time? ???? ????? ?? ?? > Computational Perspective - backpropagation? “?? forward pass”? derivative? ??? ? ??? ? (ex. rnn ???? ? ?? sequence? recurring?? ??? ????? ? ??) => ??? ??? reinforcement learning? ? ? random weight?? ???? ??? high variance ??? ????

3. forward-forward algorithm - Blotzmann machine? ??? greedy multi-layer learning procedure Forward-Forward algorithm ?? ?? 1. Goodness function for one layer: Weight? ????? ? ??? objective function 2. Forward (Positive), Forward (Negative): Network procedure

4. forward-forward algorithm Goodness function constant threshold feature logistic function

5. forward-forward algorithm Forward (positive) - Forward (negative) 1 layer Expected Output : Positive Input => Output > threshold : Negative Input => Output < threshold Input

6. forward-forward algorithm Sum-up! - “Positive Sample”? forward?? network signal? “threshold” ??? response, “Negative Sample”? forward?? signal? “threshold” ??? response? ?????

7. Negative data for Forward-Forward Unsupervised Task 1. Create “Random” Binary Mask 2. ?? ???? ??? ??? Mask? ?? ?? sum => Negative Data? ???? “characterize shape”? ???? ?? ??

8. Negative data for Forward-Forward Unsupervised Task => ??? ???? ???, Positive Data? MNIST? ???? Negative Data? hybrid? ??? 4 layer (for each 2000 feature) ????? ?? ?? 2,3,4 layer? feature? Linear Classifier ??? ???? 1.37% error rate in MNIST? ??? - (??? backprop FC ????? 1.4%, dropout/label smoothing ?? ???? 1.1%) - (First layer? feature? linear classifier? ???? ?? ??) => ????? convolution kernel? ???? Linear Classifier ?? ? 1.16% error rate in MNIST *?? following ???? negative data? “? ??? ??”?? ?? ? Forward Positive Forward Negative Linear Classifier Sample/ Hybrid

9. Negative data for Forward-Forward Supervised Task Unsupervised?? ? ??? ??? sample? ??? ??, ???? label? ?? ??? ??? + Label ??? “Input”? ??? Forward? ?! => ????? Label? Image ??? Correlation? ??? ??

10. Negative data for Forward-Forward Supervised Task - Supervised Task? Forward Forward? ??? ?? for i in range(num_class): input = i Class Vector? ??? + sample network? ?? ????? output? ?? accumulate => accumulated? ?? ?? “?” class? ??? class Forward Positive Forward Negative Class Vector + Sample => Class? 10???, 10? Inference ???! ~ 1.36% error rate in FC with FF ~ 0.64% error rate in CNN with FF

11. Exp in CIFAR 10 *Network: 3 layer with 3072 ReLU each - Compute goodness for every label: ? ????? Input vector? ???? inference (supervised task) - one-pass softmax: (unsupervised task) => min/max ssq? goodness function? minimize, maximize? ?? == network output? “threshold” ??? ?? ???, ?? ?? ??? => BP? “overfitting”?? ???? ??

12. Pros & Cons - Pros ~ Backprop? ???? Full derivatives? ???? ???? ?????, forward-forward? ??? ?? ??? ?? ??? ???? ??? ~ Trillion? ????? ???? ???? ??? watts? ????, forward-forward? “mortal computation”??? ???? ???? ?? (* hardware efficiency? ???? ?) - Cons ~ ?? backpropagation?? ??? ???, generalize ??? ??? ???? ?? (backprop ??? ???) ~ Big Model? Big Data? backpropagation? ?? ? (??? ?? ?????? ???? ? ?? ???…?)

13. Future Works - Negative Forward? Positive Forward?? ?? - Negative forward ?? Positive Forward? ???? - Goodness function? ?? ??? ??? ??? - ReLU ?? activation function? ??????? ???? (t-distribution ?) - Forward-Forward? ?? ???? - …

14. ref https://medium.com/mlearning-ai/pytorch-implementation-of-forward-forward-algorithm-by-geoffre y-hinton-and-analysis-of-performance-7e4f1a26d70f https://www.quantamagazine.org/artificial-neural-nets-finally-yield-clues-to-how-brains-learn-2021 0218/ https://bdtechtalks.com/2022/12/19/forward-forward-algorithm-geoffrey-hinton/ https://github.com/mohammadpz/pytorch_forward_forward https://www.cs.toronto.edu/~hinton/

狠狠撸

Forward-Forward Algorithm

Recommended

More Related Content

What's hot (20)

Similar to Forward-Forward Algorithm (20)

More from Dong Heon Cho (20)

Forward-Forward Algorithm