北村大地, 小野順貴, "独立性基準を用いた非負値行列因子分解の効果的な初期値決定法," 日本音響学会 2016年春季研究発表会, 3-3-5, pp. 619-622, Kanagawa, March 2016.
Daichi Kitamura, Nobutaka Ono, "Statistical-independence-based effective initialization for nonnegative matrix factorization," Proceedings of 2016 Spring Meeting of Acoustical Society of Japan, 3-3-5, pp. 619-622, Kanagawa, March 2016 (in Japanese).
Several recent papers have explored self-supervised learning methods for vision transformers (ViT). Key approaches include:
1. Masked prediction tasks that predict masked patches of the input image.
2. Contrastive learning using techniques like MoCo to learn representations by contrasting augmented views of the same image.
3. Self-distillation methods like DINO that distill a teacher ViT into a student ViT using different views of the same image.
4. Hybrid approaches that combine masked prediction with self-distillation, such as iBOT.
This document summarizes recent research on applying self-attention mechanisms from Transformers to domains other than language, such as computer vision. It discusses models that use self-attention for images, including ViT, DeiT, and T2T, which apply Transformers to divided image patches. It also covers more general attention modules like the Perceiver that aims to be domain-agnostic. Finally, it discusses work on transferring pretrained language Transformers to other modalities through frozen weights, showing they can function as universal computation engines.
Effective Optimization Algorithms for Blind and Supervised Music Source Separation with Nonnegative Matrix Factorization
長倉研究奨励賞第三次審査,20分間の研究概要説明
内容は自身の学位論文の一部に相当
AAAI2023「Are Transformers Effective for Time Series Forecasting?」と、HuggingFace「Yes, Transformers are Effective for Time Series Forecasting (+ Autoformer)」の紹介です。
Efficient initialization for nonnegative matrix factorization based on nonneg...Daichi Kitamura
?
Daichi Kitamura, Nobutaka Ono, "Efficient initialization for nonnegative matrix factorization based on nonnegative independent component analysis," The 15th International Workshop on Acoustic Signal Enhancement (IWAENC 2016), Xi'an, China, September 2016.
This document summarizes recent research on applying self-attention mechanisms from Transformers to domains other than language, such as computer vision. It discusses models that use self-attention for images, including ViT, DeiT, and T2T, which apply Transformers to divided image patches. It also covers more general attention modules like the Perceiver that aims to be domain-agnostic. Finally, it discusses work on transferring pretrained language Transformers to other modalities through frozen weights, showing they can function as universal computation engines.
Effective Optimization Algorithms for Blind and Supervised Music Source Separation with Nonnegative Matrix Factorization
長倉研究奨励賞第三次審査,20分間の研究概要説明
内容は自身の学位論文の一部に相当
AAAI2023「Are Transformers Effective for Time Series Forecasting?」と、HuggingFace「Yes, Transformers are Effective for Time Series Forecasting (+ Autoformer)」の紹介です。
Efficient initialization for nonnegative matrix factorization based on nonneg...Daichi Kitamura
?
Daichi Kitamura, Nobutaka Ono, "Efficient initialization for nonnegative matrix factorization based on nonnegative independent component analysis," The 15th International Workshop on Acoustic Signal Enhancement (IWAENC 2016), Xi'an, China, September 2016.
Relaxation of rank-1 spatial constraint in overdetermined blind source separa...Daichi Kitamura
?
Presented at The 2015 European Signal Processing Conference (EUSIPCO 2015, international conference)
Daichi Kitamura, Nobutaka Ono, Hiroshi Sawada, Hirokazu Kameoka, Hiroshi Saruwatari, "Relaxation of rank-1 spatial constraint in overdetermined blind source separation," Proceedings of The 2015 European Signal Processing Conference (EUSIPCO 2015), pp.1271-1275, Nice, France, September 2015 (Invited Special Session).
音源分離における音響モデリング(Acoustic modeling in audio source separation)Daichi Kitamura
?
北村大地, "音源分離における音響モデリング," 日本音響学会 サマーセミナー 招待講演, September 11th, 2017.
Daichi Kitamura, "Acoustic modeling in audio source separation," The Acoustical Society of Japan, Summer Seminar Invited Talk, September 11th, 2017.
ランク1空間近似を用いたBSSにおける音源及び空間モデルの考察 Study on Source and Spatial Models for BSS wi...Daichi Kitamura
?
Presented at 2015 Autumn Meeting of Acoustical Society of Japan (domestic conference)
北村大地, 猿渡洋, 小野順貴, 澤田宏, 亀岡弘和, "ランク1空間近似を用いたBSSにおける音源及び空間モデルの考察," 日本音響学会 2015年秋季研究発表会, 3-6-10, pp.583-586, Fukushima, September 2015.
Daichi Kitamura, Hiroshi Saruwatari, Nobutaka Ono, Hiroshi Sawada, Hirokazu Kameoka, "Study on source and spatial models for BSS with rank-1 spatial approximation," Proceedings of 2015 Autumn Meeting of Acoustical Society of Japan, 3-6-10, pp.583-586, Fukushima, September 2015 (in Japanese).
Blind source separation based on independent low-rank matrix analysis and its...Daichi Kitamura
?
Daichi Kitamura, "Blind source separation based on independent low-rank matrix analysis and its extension to Student's t-distribution," Télécom ParisTech, Invited Lecture, September 4th, 2017.
非負値行列分解の確率的生成モデルと多チャネル音源分離への応用 (Generative model in nonnegative matrix facto...Daichi Kitamura
?
北村大地, "非負値行列分解の確率的生成モデルと多チャネル音源分離への応用," 慶應義塾大学理工学部電子工学科湯川研究室 招待講演, Kanagawa, November, 2015.
Daichi Kitamura, "Generative model in nonnegative matrix factorization and its application to multichannel sound source separation," Keio University, Science and Technology, Department of Electronics and Electrical Engineeing, Yukawa Laboratory, Invited Talk, Kanagawa, November, 2015.
2017年6月24日,ICASSP2017読み会(関東編)@東京大学
AASP-L3: Deep Learning for Source Separation and Enhancement I
東京大学特任助教 北村大地担当分のスライド
私が著者ではないペーパーの紹介スライドですので,再配布等はご遠慮ください.また,このスライドで取り扱っていない詳細な情報に関しては対象となる論文をご参照ください.
Experimental analysis of optimal window length for independent low-rank matri...Daichi Kitamura
?
Daichi Kitamura, Nobutaka Ono, and Hiroshi Saruwatari, "Experimental analysis of optimal window length for independent low-rank matrix analysis," Proceedings of The 2017 European Signal Processing Conference (EUSIPCO 2017), pp. 1210–1214, Kos, Greece, August 2017 (Invited Special Session).
Presented at 25th European Signal Processing Conference (EUSIPCO) 2017, "SS14: Multivariate Analysis for Audio Signal Source Enhancement," 14:30-16:10, August 30, 2017.
Audio Source Separation Based on Low-Rank Structure and Statistical IndependenceDaichi Kitamura
?
Daichi Kitamura presented his research on audio source separation. He discussed using low-rank modeling of spectrograms and non-negative matrix factorization to separate sources based on their structural properties in supervised settings. He also discussed using statistical independence between sources and the central limit theorem as the basis for blind source separation via independent component analysis. The talk covered applications of source separation, demonstrations of techniques, and challenges like basis mismatch for supervised methods and permutation problems for blind separation.
統計的独立性と低ランク行列分解理論に基づくブラインド音源分離 –独立低ランク行列分析– Blind source separation based on...Daichi Kitamura
?
北村大地, "統計的独立性と低ランク行列分解理論に基づくブラインド音源分離 –独立低ランク行列分析–," 筑波大学システム情報工学研究科マルチメディア研究室 招待講演, Ibaraki, September 26th, 2016.
Daichi Kitamura, "Blind source separation based on statistical independence and low-rank matrix decomposition –Independent low-rank matrix analysis–," University of Tsukuba, Graduate School of Systems and Information Engineering, Multimedia Laboratory, Invited Talk, Ibaraki, September 26th, 2016.
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...Daichi Kitamura
?
東京大学 システム情報学専攻 談話会
2017年2月27日(月)15時~16時30分
北村大地, "独立性に基づくブラインド音源分離の発展と独立低ランク行列分析," 東京大学 システム情報学専攻 談話会, 2月27日, 2017年.
Daichi Kitamura, "History of independence-based blind source separation and independent low-rank matrix analysis," The University of Tokyo, Department of Information Physics and Computing, Seminar, 27th Feb., 2017.
独立低ランク行列分析に基づく音源分離とその発展(Audio source separation based on independent low-rank...Daichi Kitamura
?
北村大地, "独立低ランク行列分析に基づく音源分離とその発展," IEICE信号処理研究会, 2021年8月24日.
Daichi Kitamura, "Audio source separation based on independent low-rank matrix analysis and its extensions," IEICE Technical Group on Signal Processing, Aug. 24th, 2021.
http://d-kitamura.net
日本音響学会2021春季研究発表会1-1-2
北村大地, 矢田部浩平, "スペクトログラム無矛盾性を用いた独立低ランク行列分析の実験的評価," 日本音響学会 2021年春季研究発表会講演論文集, 1-1-2, pp. 121–124, Tokyo, March 2021.
Daichi Kitamura and Kohei Yatabe, "Experimental evaluation of consistent independent low-rank matrix analysis," Proceedings of 2021 Spring Meeting of Acoustical Society of Japan, 1-1-2, pp. 121–124, Tokyo, March 2021 (in Japanese).
独立深層学習行列分析に基づく多チャネル音源分離(Multichannel audio source separation based on indepen...Daichi Kitamura
?
角野隼斗, 北村大地, 高宗典玄, 高道慎之介, 猿渡洋, 小野順貴, "独立深層学習行列分析に基づく多チャネル音源分離," 日本音響学会 2018年春季研究発表会講演論文集, 1-4-16, pp. 449–452, Saitama, March 2018.
Hayato Sumino, Daichi Kitamura, Norihiro Takamune, Shinnosuke Takamichi, Hiroshi Saruwatari, Nobutaka Ono, "Multichannel audio source separation based on independent deeply learned matrix analysis," Proceedings of 2018 Spring Meeting of Acoustical Society of Japan, 1-4-16, pp. 449–452, Saitama, March 2018 (in Japanese).
近接分離最適化によるブラインド?源分離(Blind source separation via proximal splitting algorithm)Daichi Kitamura
?
矢田部浩平, 北村大地, "近接分離最適化によるブラインド?源分離," 日本音響学会 2018年春季研究発表会講演論文集, 1-4-10, pp. 431–434, Saitama, March 2018.
Kohei Yatabe, Daichi Kitamura, "Blind source separation via proximal splitting algorithm," Proceedings of 2018 Spring Meeting of Acoustical Society of Japan, 1-4-10, pp. 431–434, Saitama, March 2018 (in Japanese).
Blind source separation based on independent low-rank matrix analysis and its...Daichi Kitamura
?
Daichi Kitamura, "Blind source separation based on independent low-rank matrix analysis and its extensions," Ohio State University, Invited Lecture, December 15th, 2017.
Evaluation of separation accuracy for various real instruments based on super...Daichi Kitamura
?
Presented at 2013 Spring Meeting of Acoustical Society of Japan (domestic conference)
Daichi Kitamura, Hiroshi Saruwatari, Kiyohiro Shikano, Kazunobu Kondo, Yu Takahashi, "Evaluation of separation accuracy for various real instruments based on supervised NMF with basis deformation," Proceedings of 2013 Spring Meeting of Acoustical Society of Japan, 3-1-11, pp.1057-1060, Tokyo, March 2013.
Divergence optimization based on trade-off between separation and extrapolati...Daichi Kitamura
?
Presented at 2013 Autumn Meeting of Acoustical Society of Japan (domestic conference)
Daichi Kitamura, Hiroshi Saruwatari, Satoshi Nakamura, Kazunobu Kondo, Yu Takahashi, "Divergence optimization based on trade-off between separation and extrapolation abilities in superresolution-based nonnegative matrix factorization," Proceedings of 2013 Autumn Meeting of Acoustical Society of Japan, 1-1-6, pp.583-586, Aichi, September 2013 (学生優秀発表賞受賞).
Depth estimation of sound images using directional clustering and activation-...Daichi Kitamura
?
Presented at 2014 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP 2014) (international conference)
Tomo Miyauchi, Daichi Kitamura, Hiroshi Saruwatari, Satoshi Nakamura, "Depth estimation of sound images using directional clustering and activation-shared nonnegative matrix factorization," Proceedings of 2014 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP 2014), pp.437-440, Hawaii, USA, March 2014 (Student Paper Award).