【摘要】 在使用分布式内存计算的机器学习算法进行情感分析时,其训练效率还亟需提升,同时使用单个机器学习算法的准确率也不是很高,因此提出了一种基于Spark的集成算法来进行情感分析。在进行情感分析过程中通过Spark分布式内存计算,来实现算法的并行操作,有利于机器学习算法处理大数据集。Spark自带的机器学习库,使开发过程变得更加简单和快速。采用改进后的TF-IDF特征提取算法,以AdaBoost算法集成决策树和SVM,降低了单个算法的偶然性,提高了整个结果的准确性。实验结果表明:Spark分布式计算提高了效率,集成算法的情感识别更准确。更多还原

【Abstract】 When the machine learning algorithm for emotional analysis is calculated with distributed memory, its training efficiency needs to be improved urgently, and the accuracy of using single machine learning algorithm is not very high. To solve these problems, an integrated algorithm based on Spark is proposed for emotional analysis. In the process of emotional analysis, Spark distributed memory computing is used to realize parallel operation of the algorithm. It facilitates machine learning algorithm to process large data sets. The machine learning library that comes with Spark makes the development process easier and faster. The improved TF-IDF feature extraction algorithm is used to integrate decision tree and SVM with AdaBoost algorithm, which reduces the contingency of single algorithm and improves the accuracy of the entire result. The experimental results show that Spark distributed computing improves the efficiency and the emotion recognition with integrated algorithm is more accurate.更多还原