【摘要】 本文从几何角度来理解深度学习,特别是提出了生成对抗网络(GAN)的最优传输(OT)观点。自然数据集具有内在的模式,该模式可被概括为流形分布原理,即同一类高维数据分布于低维流形附近。GAN主要完成流形学习和概率分布变换两项任务。其中,后者可以用经典的OT方法来实现。从OT的角度来看,生成器用于计算OT映射,而判别器用于计算生成数据分布与真实数据分布之间的Wasserstein距离;两者都可以归结为一个凸优化过程。此外,OT理论揭示了生成器与判别器之间的内在关系是协作的而不是竞争的,并且解释了模式崩溃的根本原因。在此基础上,我们提出了一种新的生成模型,该模型利用自编码器(AE)进行流形学习,并利用OT映射进行概率分布变换。这个AE-OT模型提升了深度学习理论的严谨性和透明性、提高了计算的稳定性和效率,尤其是避免了模式崩溃问题。实验结果验证了我们的假设,并充分展示了我们提出的AE-OT模型的优点。更多还原

【Abstract】 This work introduces an optimal transportation(OT) view of generative adversarial networks(GANs).Natural datasets have intrinsic patterns, which can be summarized as the manifold distribution principle:the distribution of a class of data is close to a low-dimensional manifold. GANs mainly accomplish two tasks: manifold learning and probability distribution transformation. The latter can be carried out using the classical OT method. From the OT perspective, the generator computes the OT map, while the discriminator computes the Wasserstein distance between the generated data distribution and the real data distribution; both can be reduced to a convex geometric optimization process. Furthermore, OT theory discovers the intrinsic collaborative—instead of competitive—relation between the generator and the discriminator, and the fundamental reason for mode collapse. We also propose a novel generative model, which uses an autoencoder(AE) for manifold learning and OT map for probability distribution transformation. This AE–OT model improves the theoretical rigor and transparency, as well as the computational stability and efficiency; in particular, it eliminates the mode collapse. The experimental results validate our hypothesis, and demonstrate the advantages of our proposed model.更多还原