【摘要】 为了探究神经网络权重对网络产生的影响,通过选取多个典型的卷积神经网络模型,采用统计分析及最大似然拟合等方式进行分布拟合,对网络权重进行可视化的对比研究。研究结果表明,神经网络的权重值在训练过程中会向负方向偏移,均值更趋向于0。同时,预训练权重的分布也不再服从初始化时的正态分布,而是向趋于0的方向收缩,呈现出峰值更高的特征,且出现了长尾的现象,局部表现出了幂律分布的特性。利用这些分布特征对神经网络初始化参数做调整,可在一定程度上提高训练效率。更多还原

【Abstract】 In order to explore the influence of neural network weights on the network,a number of typical convolutional neural network models,were selected to conduct distribution fitting by means of statistical analysis and maximum likelihood fitting,so as to conduct a visual comparative study on network weights.The results show that the weight value of neural network will shift to the negative direction during training,and the mean value tends to 0.At the same time,the distribution of pre-training weight is no longer subject to the normal distribution at the initial stage,but shrinks towards 0,showing a higher peak value and a long tail,which shows the characteristics of power law distribution locally.Using these distribution characteristics to adjust neural network initialization parameters can improve training efficiency to a certain extent.更多还原