PFN福田圭祐による東大大学院「融合情報学特別講義Ⅲ」(2022年10月19日)の講義資料です。
?Introduction to Preferred Networks
?Our developments to date
?Our research & platform
?Simulation ? AI
The document discusses recent advances in generative adversarial networks (GANs) for image generation. It summarizes two influential GAN models: ProgressiveGAN (Karras et al., 2018) and BigGAN (Brock et al., 2019). ProgressiveGAN introduced progressive growing of GANs to produce high resolution images. BigGAN scaled up GAN training through techniques like large batch sizes and regularization methods to generate high fidelity natural images. The document also discusses using GANs to generate full-body, high-resolution anime characters and adding motion through structure-conditional GANs.
This document summarizes a research paper on scaling laws for neural language models. Some key findings of the paper include:
- Language model performance depends strongly on model scale and weakly on model shape. With enough compute and data, performance scales as a power law of parameters, compute, and data.
- Overfitting is universal, with penalties depending on the ratio of parameters to data.
- Large models have higher sample efficiency and can reach the same performance levels with less optimization steps and data points.
- The paper motivated subsequent work by OpenAI on applying scaling laws to other domains like computer vision and developing increasingly large language models like GPT-3.
PFN福田圭祐による東大大学院「融合情報学特別講義Ⅲ」(2022年10月19日)の講義資料です。
?Introduction to Preferred Networks
?Our developments to date
?Our research & platform
?Simulation ? AI
The document discusses recent advances in generative adversarial networks (GANs) for image generation. It summarizes two influential GAN models: ProgressiveGAN (Karras et al., 2018) and BigGAN (Brock et al., 2019). ProgressiveGAN introduced progressive growing of GANs to produce high resolution images. BigGAN scaled up GAN training through techniques like large batch sizes and regularization methods to generate high fidelity natural images. The document also discusses using GANs to generate full-body, high-resolution anime characters and adding motion through structure-conditional GANs.
This document summarizes a research paper on scaling laws for neural language models. Some key findings of the paper include:
- Language model performance depends strongly on model scale and weakly on model shape. With enough compute and data, performance scales as a power law of parameters, compute, and data.
- Overfitting is universal, with penalties depending on the ratio of parameters to data.
- Large models have higher sample efficiency and can reach the same performance levels with less optimization steps and data points.
- The paper motivated subsequent work by OpenAI on applying scaling laws to other domains like computer vision and developing increasingly large language models like GPT-3.