【作者】 薛冰；

【导师】 吕钊；

【作者基本信息】 安徽大学 ， 计算机科学与技术， 2021， 硕士

【摘要】 随着人工智能技术的快速发展,智能化的人-机交互设备已逐渐走进人们的生活。虽然它们能够较好地按照人们的意愿完成相应的功能,但几乎不能进行情感交流,无法根据使用者的心理感受调整交互方式,这极大制约了其功能和应用范围。情感作为一种主要的信息交流方式,在人们的日常沟通中发挥着重要的作用。开发具有情感自主感知的人机交互系统,已成为人工智能与人机交互领域中一个重要的研究方向。目前,情感计算的主要数据来源大致可分为三类:面部表情、语音以及生理信号。其中,脑电信号(Electroencephalogram,EEG)由于其具有实时差异性与不易伪装性等特点,因此,使用EEG进行情感识别已成为一个新的研究热点。现阶段,从识别模型上来看,基于EEG的情感分类方法大致可分为传统机器学习和深度学习两种。其中,传统机器学习方法主要包括支持向量机、K近邻分类器、逻辑回归等。相比较传统的机器学习方法,深度学习方法由于其具有较强的特征表达能力、复杂任务的建模能力和抽象认知识别能力,因此,能够较好地模拟人脑情感认知和思维,具备处理更为复杂情感EEG信号能力。另外,由于不同被试者在数据采集过程、生活环境及身心状态等方面的差异性,将导致所采集的情感EEG信号个体差异较大。这种差异将会降低识别模型的准确性,限制了情感识别的应用场景。基于此,论文围绕基于EEG的情感识别及迁移性展开研究。首先,研究并实现了使用深度学习方法对情感EEG信号进行分类识别;在此基础上,利用迁移特征算法迁移成分分析(Transfer Component Analysis,TCA)对多受试者情感脑电的联合特征进行迁移性学习。主要研究内容如下:(1)使用卷积神经网络(Convolutional Neural Network,CNN)对脑电数据进行情感识别。对原始情感62导EEG信号进行预处理后,分别提取了微分熵、功率谱密度、左右半脑不对称性等频域特征及不同导联的空间位置信息特征;在此基础上使用卷积神经网络,实现对积极、中性、消极3种情感脑电的识别。在SEED数据库上对所提特征使用分类识别网络进行了实验验证,其综合平均识别正确率为88.01%。相比较支持向量机、深度神经网络、极限学习机等方法,识别结果分别提升了14.56%、13.53%和15.72%。实验结果验证了所提方法在情感识别中的有效性。(2)开展了基于混合微分熵(Differential Entropy,DE)的情感脑电特征迁移性研究。首先,将SEED数据库中的15个受试者的情感脑电数据分别提取微分熵特征;接着,使用TCA算法对于DE特征进行迁移降维处理;在此基础上,随机选取其中的14个受试者的TCA后的特征作为训练数据,预留的1个受试者的数据作为测试数据进行深度学习模型的训练与情感状态的识别。为了保证实验结果的准确性,上述过程执行了15次重复后得到最优平均识别率为58.49%,相比较原始的混合DE特征平均识别率52.26%,其正确率提升了6.23%,实验结果验证了所提算法在跨被试者识别时的有效性。(3)提出联合不同特征对多个受试者的混合EEG进行情感迁移学习。首先,将14名被试微分熵、空域特征及其联合特征(微分熵+空域)分别进行组合后,进行深度学习模型的训练;接着,使用该模型对剩余的1名被试数据进行测试,其平均识别率分别为52.26%,46.77%和53.88%,结果表明,微分熵与空域特征相结合能够更为细致地描述不同被试间的共性情感信息。进一步,使用特征迁移方法对以上混合特征进行数据迁移降维,联合特征最优平均识别率达85.73%,结果表明通过联合特征迁移减少了特征域之间的差异性,提高了模型的适用性和识别性。更多还原

【Abstract】 With the rapid development of Artificial intelligence(AI),intelligent Human-Computer Interaction(HCI)equipment has been gradually applied to people’s daily lives.Although the HCI equipment can satisfied parts of human’s willingness,they hardly interact with human emotionally,that is to say,they are poor at adjusting the interaction mode according to user’s psychological feelings.Hence,the function and application of HCI equipment are greatly restricted.As one of the main methods of information exchange,emotion plays an important role in people’s daily communication.Developing the HCI equipment which can autonomously perceive person’s emotion has become an important research direction in the fields of AI and HCI.At present,the main data sources of affective computing can be roughly categorized as three types: facial expressions,speech signals and physiological signals.While,due to the properties of the real-time difference and the difficult camouflage,the electrocardiogram(EEG)signals,which is one of physiological signals,has attract more and more researchers attention.In the view of recognition models,EEG-based emotion classification methods can be roughly categorized as traditional machine learning and deep learning,where traditional machine learning methods mainly include Support Vector Machines(SVM),K-nearest Neighbor Classifiers(KNN),Logistic Regression(LR)and so on.Compared to traditional machine learning,deep learning methods perform better performance on feature representation,complex task modeling,and abstract cognitive recognition.Therefore,deep learning can simulate human’s emotion cognition and thinking modes,which ensure that deep learning could deal with the more complexly emotion-related EEG signals.Because of the difference caused by data collection process,living environment,and physical and mental state,the emotion-related EEG signal could be a great difference between different subjects,which could be one of reasons that may highly reduce the accuracy of the deep learning model and limit the application scenarios of the emotion recognition.Therefore,the EEG-based emotion recognition and corresponding transfer learning is researched in this thesis.First of all,deep learning methods are used to classify the emotion-related EEG signals.Then,aimed at the joint features of multi-subject’s emotional EEG signals,the Transfer Component Analysis(TCA)is employed.The specific works are as follows:(1)Convolutional Neural Network(CNN)is employed to achieve the emotion recognition task based on EEG databases.Firstly,the frequency domain and spatial location information features,such as differential entropy(DE),power spectral density(PSD),hemisphere asymmetry,are extracted after preprocessing the original emotional 62-channel EEG signals.Then,CNN is used to recognize the three different emotion modes,i.e.,positive,neural and negative,based on the above extracted features.Finally,the SJTU Emotion EEG Dataset(SEED)database is employed to verify the extracted features combined with CNN,in which the comprehensive average recognition accuracy reach to 88.01%.Compared with the SVM,Deep Neural Network(DNN)and Extreme Learning Machine(ELM),the average accuracy of the proposed method is increased by 14.56%,13.53%,15.72%,respectively.The effectiveness of the proposed method is demonstrated by the outstanding experimental results.(2)Mixed-DE feature which is extracted from emotional EEG is used to carry out the transfer research.Firstly,DE features are extracted from the emotional EEG data of 15 subjects in SEED database respectively.Then,the TCA algorithm is used to transfer the features and reduce the dimension.Next,after utilize the TCA algorithm,select the features of14 of all subjects as training set,and the features of another subject as test set.Finally,the deep learning method is employed to recognize the different emotion state.In order to ensure the accuracy of the experimental results,the experiment is repetitively carried out by 15-hold cross validation,and the best optimal accuracy is 58.49%.Compared with the original DE features(52.26%),the optimal average accuracy is increased by 6.23%.The effectiveness of the proposed method is illustrated by the prominent experimental results.(3)The method combining different features is proposed to make a transfer learning based on the mixed emotion-related EEG signals of multi-subjects.Firstly,the DE features,the spatial features,and their jointed feature(DE-Spatial)extracted from the 14 of all subjects are combined to feed the deep learning model.Then,the same features extracted from another one subject is used to test the proposed model,where the average classification rate are52.26%,46.77% and 53.88% respectively.Finally,TCA is used to transfer the above mixed features and reduce their dimension,and the corresponding optimal average recognition rate reaches to 85.73%.The remarkable classification accuracy demonstrates that the transferred joint features have capability to reduce the difference between different feature domains,and improve the performance and applicability of the proposed model.更多还原