【摘要】 通过功能磁共振成像(fMRI)进行大脑编码和解码是视觉神经科学的两个重要方面。尽管以前的研究人员在大脑编码和解码模型方面取得了显著进步,但是现有方法仍需要使用先进的机器学习技术进行改进。例如,传统方法通常会分别构建编码和解码模型,并且容易对小型数据集过度拟合。实际上,有效地统一编码和解码过程可以进行更准确的预测。在本文中,我们首先回顾了现有的编码和解码方法,并讨论了"双向"建模策略的潜在优势。接下来,在体系结构和计算规则方面,我们证明了深度神经网络和人类视觉通路之间存在的对应关系。此外,深度生成模型[如变分自编码器(VAE)和生成对抗网络(GAN)]在大脑编码和解码研究中产生了可喜的成果。最后,我们提出了最初为机器翻译任务设计的对偶学习方法,该方法通过利用大规模未配对数据提高了编码和解码模型的效果。更多还原

【Abstract】 Brain encoding and decoding via functional magnetic resonance imaging(fMRI) are two important aspects of visual perception neuroscience.Although previous researchers have made significant advances in brain encoding and decoding models,existing methods still require improvement using advanced machine learning techniques.For example,traditional methods usually build the encoding and decoding models separately,and are prone to overfitting on a small dataset.In fact,effectively unifying the encoding and decoding procedures may allow for more accurate predictions.In this paper,we first review the existing encoding and decoding methods and discuss the potential advantages of a "bidirectional" modeling strategy.Next,we show that there are correspondences between deep neural networks and human visual streams in terms of the architecture and computational rules.Furthermore,deep generative models(e.g.,variational autoencoders(VAEs) and generative adversarial networks(GANs)) have produced promising results in studies on brain encoding and decoding.Finally,we propose that the dual learning method,which was originally designed for machine translation tasks,could help to improve the performance of encoding and decoding models by leveraging large-scale unpaired data.更多还原