【作者】 张颖；

【导师】 阮敬；

【作者基本信息】 首都经济贸易大学 ， 应用统计（专业学位）， 2020， 硕士

【摘要】 现今深度学习模型在各领域应用的非常广泛,而算法模型的性能取决于它的结构。寻找一组最优的超参数来确定网络结构非常困难,这成为深度学习模型广泛应用的一大阻碍。决定模型结构的超参数无法从训练中获得,需要在模型运行之前设定。而现有的方法多依赖于以往经验和人工选择,其效率低下,且当数据量巨大,模型结构复杂时手动调参就困难重重。因此从模型的角度来说,如何实现高效率地自动调参是超参数优化问题的关键,而超参数的优化又是模型算法得以有效应用的关键。因此,本文主要研究深度学习模型的超参数优化问题。首先研究六种不同优化算法在BP神经网络与LSTM上的应用效果。使用的六种超参优化方法为:网格搜索;随机搜索;贝叶斯优化;Talos优化;Hyperband和Optuna框架。其次从调优的时间效率与最终模型的准确率上进行对比,分别找出适用于BP与LSTM的最佳超参数优化方法。具体为适用于BP神经网络的超参数优化方法为Hyperband,适用于LSTM的超参数优化方法为Optuna。为模型优化的广泛使用提供了建议。最后本文提出新的猜想:对于不同结构类型的神经网络(前馈神经网络与循环神经网络)其适用的优化算法是否不同。并证实了对于不同的深度学习问题,适用的超参数优化方法不同。具体为解决简单问题的BP神经网络,最佳的超参数优化方法为Hyperband。解决较难问题的结构复杂网络(LSTM),最佳的超参数优化方法为Optuna。更多还原

【Abstract】 Deep learning models have been extensively applied in various fields,and the performance of algorithm model is decided by its own structure.It can be rather difficult to find a set of optimum hyperparameters to determine the network structure,which presents as a major obstacle for the application of deep learning model.While the hyperparameters cannot be obtained through the model training,it must be set no later than the model runs.At present stage,most methods are operated mostly rely on previous experiences and manual selection,which is inefficient.When encountered with huge volume of data or complex structures,it can be hard to manually adjust parameters.Therefore,the way of realizing efficiently automatic parameter adjustment,from the perspective of model itself,serves as the key of hyperparameter optimization.Conversely,the optimization of hyperparameter also means significantly for the effective application of model algorithm.Therefore,this paper lays its focus on the hyperparameter optimization of deep learning model.Firstly,the application effect of six optimization algorithms on BP neural network and LSTM is respectively studied.These methods specifically involve grid search,random search,Bayesian optimization,Talos optimization,hyperband and Optuna framework.Secondly,the finest hyperparameter optimization method for BP and LSTM is respectively illustrated by comparing the time efficiency of optimization with the accuracy of the final model.More specifically,the hyperparameter optimization method for BP neural network is hyperband,and Optuna is prepared for LSTM,which provides optimization experiences for more widely used models.A hypothesis is hereby proposed in the end of this paper: whether the varied optimal algorithm is also corresponded to varied types of neural networks(feedforward neural network and cyclic neural network).Additionally,the author has also demonstrated that different hyperparameter optimization methods is corresponded to various deep learning issues.To cope with the simple problem encountered in BP neural network,the optimum hyperparameter optimization method is hyperband,and Optuna serves as the optimum one for LSTM.更多还原