【作者】 刘庆平；

【导师】 姜万录；

【作者基本信息】 燕山大学 ， 机械电子工程， 2003， 硕士

【摘要】 传统的神经网络(BP网络)在网络训练和网络设计上长期受困于三个难以克服的缺陷，即网络训练速度慢、训练易陷入局部极小点和网络学习的推广性能差。本文从算法层和计算理论层两个层次对造成这些缺陷的原因和克服这些缺陷的方法进行了系统的研究。在算法层，本文对目前用于神经网络训练的各种算法，包括梯度算法、智能学习算法和混合学习算法进行了比较研究；对用于神经网络训练的BP算法的优化原理进行了详细的理论分析，找到了BP算法存在严重缺陷的原因，并对其两类改进算法-启发式算法和二次梯度算法―的优化原理，在统一的框架之下进行了详尽的理论描述；对神经网络全局优化算法主要是遗传算法进行了详细的阐述，并在此基础上，设计了一种性能改进的遗传算法；最后基于神经网络学习的benchmark问题对各种算法在网络训练中的应用性能进行了仿真研究，并提出了遗传算法受困于“维数灾难”的观点。这一层次的研究表明，算法层只是在原有神经网络的框架下利用高性能的优化算法克服网络学习的前两个缺陷，由于受目前优化理论的限制，很难有巨大的突破。在计算理论层，从机器学习的角度分析了造成神经网络设计困难的原因；对指导神经网络设计的统计学习理论和正规化方法给以了系统的阐述；并重点对由统计学习理论直接导出的先进的学习机器—支持向量机—的理论进行了比较全面的阐述；通过函数逼近和系统建模等学习任务对神经网络和支持向量机学习的推广性能进行了仿真研究。这一层次的研究表明，支持向量机可以很好地克服神经网络学习的三个缺陷。因此，从计算理论层出发，对网络学习的本质进行研究，并设计新的高性能的学习机器，从而避开传统神经网络学习机器存在的难以克服的困难，是从根本上解决神经网络学习问题的可行方法。本文最后对神经网络学习和支持向量机学习的研究领域仍需进一步研究的课题提出了自己的见解。更多还原

【Abstract】 The traditional neural networks, BP networks, are subject to three hardly conquerable drawbacks in network training and network design a long time, including slow training speed, the training tending to sinking into local minimum and the trained networks having poor generalization capability. In this paper the reasons for these drawbacks and the methods for overcoming these drawbacks are systemically studied from two levels, algorithm level and computing theory level. In the algorithm level, currently various training algorithms of neural networks, including gradient algorithms, intelligent learning algorithms and hybrid algorithms, are comparatively studied; the optimization principle of BP algorithm for neural networks training is analyzed in detail, and the reasons for serious disadvantages of BP algorithms are found out, moreover, the optimization principle of two kinds of improved BP algorithms is described in a uniform theoretic framework; and the global optimization algorithms of neural networks, mainly genetic algorithm are expounded in detail, it follows that a improved genetic algorithm is proposed; finally the training performances of various algorithms are compared based on a simulation experiment on a benchmark problem of neural network learning, furthermore, a viewpoint that genetic algorithm is subject to "curse of dimension" is proposed. The studies indicate that the algorithm level only deals with getting over the former two drawbacks of neural network learning using advanced optimization algorithms in the intrinsic framework of neural network, and great breakthrough is hard to made because of the limit of current optimization theory. In the computing theory level, the reasons resulting in difficulty in neural network design are analyzed from the point of machine learning; statistical learning theory and regularization approach directing neural network design are systemically expounded; the most important is the theory of support vector machine, which is directly induced from statistical learning theory, is comprehensively expounded; at last the generalization capability of neural networks and support vector machines is studied through a simulation experiment<WP=6>on such learning tasks as function regression and system modeling. The studies of this level indicate that support vector machine can excellently overcome the overall three drawbacks of neural network learning. In conclusion, studying on the essence of machine learning from the point of computing theory and working out novel good-character learning machine, in order to avoiding the hardly surmountable handicaps of neural network learning, is a applicable way solving radically the problems of neural network learning. At last, a personal preview of further tasks in the research realms of neural network and support vector machine is presented.更多还原