【作者】 温万惠；

【导师】 邱玉辉； 刘光远；

【作者基本信息】 西南大学 ， 基础心理学， 2010， 博士

【摘要】 心理生理学和智能人机交互领域对情感的生理可分性研究主要采用方差分析和基于模式分类的方法。这些方法通常直接从生理信号的统计特征中寻找情感特异性生理反应模式,而忽略了证明生理信号中是否存在情感生理反应。本文基于国家重点学科基础心理学基金项目(No. NKSF07003)和国家自然科学基金项目(No.60873143)的工作,以发现情感生理信号重要特征及其组合,并建立情感的生理信息计算模型为研究目的,首次应用随机矩阵理论对情感生理信号进行信号序列相关性分析,以揭示信号序列中确实存在的情感生理反应和特定的情感生理反应时间模式。按信号种类构造信号序列相关矩阵,分析相关矩阵特征值的最近邻间距分布和特征值谱刚度,检验相关矩阵的特征值和特征向量的分布,以探讨情感生理信号是否表现出相关的情感生理反应模式,从而找出包含可靠情感生理反应的信号,避免非情感特异性生理反应模式对研究结论的干扰。以随机矩阵理论的情感生理反应模式分析结果为依据,提出了高兴、惊奇、厌恶、悲伤、愤怒和恐惧6种基本情感基于特征解空间搜索和Fisher映射分类的二分类情感识别模型。将特征选择问题抽取为一个组合优化模型,使用优化问题中常用的遗传算法、蚁群算法、粒子群算法、后向选择算法和前向漂移选择算法进行特征选择,比较了上述算法在预测识别能力、避免过拟合能力、计算代价和特征维度压缩能力等方面各自的优缺点,找出了最适合于基于生理信号的情感识别特征选择问题的解空间搜索算法,并获得了每一种基本情感区别于其它情感的关键生理信号特征组合。通过上述研究工作得到了如下结果：(1)通过恰当的情感生理信号采集方案设计,获得了300名大学生的情感生理反应样本,建立了规模较大的情感生理反应样本库,其中包含了皮肤电导、心率、心电、呼吸、脉搏、肌电和额叶的两路脑电共8路生理信号,唤起了高兴、惊奇、厌恶、悲伤、愤怒和恐惧6种基本情感。与已有研究相比,该情感生理反应样本库的被试数量大,有利于发现用户非依赖的情感生理反应模式,并且生理信号种类较多,有利于从多种信号中综合提取情感特异性生理特征,以及发现对情感识别而言可靠的生理信号特征提取源。(2)通过随机矩阵理论分析发现,信号序列相关矩阵的特征值最近邻间距分布和特征值谱刚度具有高斯正交系综的一般特性,遵从随机矩阵理论的预测；信号序列相关矩阵的特征值大部分落入随机矩阵理论的预测范围之内,对应的特征向量的分布也服从随机矩阵理论预测的Porter-Thomas分布。然而在考察信号的缓变规律时,皮肤电导、心率、心电和呼吸信号序列相关矩阵的最大特征值远远超过随机矩阵理论预测的最大特征值,其对应的特征向量也背离了随机矩阵理论的预测分布,显示出信号序列之间由共同的情感唤起素材所激发的相关情感生理反应模式;在考虑信号的瞬变规律时,皮肤电导和心率仍然保持上述背离特性,表明情感生理信号采集所测试的8种生理信号中,只有皮肤电导和心率的瞬变和缓变规律中均包含了可靠的情感生理反应模式,可作为情感生理特征提取的可靠信号源。对信号的缓变和瞬变规律进行基于随机矩阵理论的数据分析,同时也揭示了情感生理反应与时间的关系。分析结果表明8路信号中只有皮肤电导和心率的变化可以快速响应情感心理体验的变化,并且心率的变化具有较强的情感时间累积效应；情感对心电和呼吸变化的影响则需要一段累积时间,而不会从信号波形变化中快速响应情感心理体验的变化；而脉搏、肌电和额叶的两路脑电中没有体现出可靠的情感生理反应。基于随机矩阵理论的分析方法不仅发现了包含可靠情感生理反应模式的生理信号,排除了未产生可靠情感生理反应的生理信号,同时也揭示了使用方差分析和模式分类方法所不能获得的情感生理反应时间模式。(3)在皮肤电导和心率两种生理信号中综合提取信号特征,从组合优化的角度进行特征选择,对各种解空间搜索算法的比较发现,后向选择算法在模型预测识别能力、计算代价、避免数据过拟合能力和特征降维能力等各方面的综合性能上是几种特征选择算法中最好的。算法比较结果揭示了基于生理信号的情感识别系统构建中,要获得具有良好泛化能力和预测识别能力的情感识别系统,特征选择过程对解空间的搜索应该是粗粒度的,这和旅行商(TSP)问题等一般优化问题有显著区别。(4)基于皮肤电导和心率生理信号特征、后向特征选择算法和Fisher映射分类器的6个二分类情感识别系统都具有比虚报率高出超过20%的击中率,并且6种目标情感各自对应的最优特征子集包含不超过10个特征,与初始的110个特征相比,特征维数显著压缩,各基本情感区别于其它情感的关键特征得以体现。上述的研究结论通过引入基于随机矩阵理论的情感生理信号分析方法,取得了情感生理反应样本分析方法上的创新,发现了视听情感诱发刺激下皮肤电导和心率信号包含可靠的情感生理反应,对情感生理信号进行了有效的筛选,并且发现了情感生理反应的时间模式。这种有效情感生理信号的筛选以及情感生理反应的时间模式未见文献报道。在此基础上构建的二分类情感识别系统具有良好的预测识别能力,各离散情感状态区别于其它情感的重要情感生理特征得以体现,情感的生理信息计算模型得以建立。更多还原

【Abstract】 Analysis of variance (ANOVA) and the pattern-classifier based methods are usually adopted to examine the physiological differentiability of the discrete emotions in the fields of psychophysiology and intelligent human-computer interaction. These methods directly depend on the statistic features of the physiological signals to find out whether there are emotion-specific physiological response patterns. However, no confirmation about the existence of affective physiological response in the physiological signals was given. For the first time, this thesis applied Random Matrix Theory (RMT) to study whether a certain kind of physiological signal contained correlated affective physiological response, and what kind of reaction time patterns the affective physiological signals had. To demonstrate the validity of the universal prediction of RMT for the eigenvalue statistics of the signal sequence empirical cross-correlation matrix, we first calculated the distribution of the nearest-neighbor spacings and the spectral rigidity of their eigenvalues, and then compared them with the RMT prediction for real symmetric random matrices. To compare the experimental data with the mutually independent time-series, a null hypothesis R which was the cross-correlation matrix of uncorrelated time-series having the same length and the same amount to the empirical data was constructed. And then the eigenvalue probability density and the eigenvector component distribution of the empirical cross-correlation matrix were computed and compared with the analytical results of the eigenvalue distribution and the eigenvector component distribution of the null hypothesis R, giving information that enabled us to identify which signals were reliable for emotion recognition in the current work. According to the analysis results of RMT, physiological features were extracted from the reliable signals, and the affective physiological signal samples each of which was depicted as a vector of the features were classified by using a Fisher classifier. In order to find out which feature subset could well distinguish the target emotion from the interference emotions, and which feature selection method was the best for the two-class emotion recognition problem, the Genetic Algorithm (GA), the Sequential Backward Selection (SBS), the Ant Colony (AC) algorithm, the Swarm Particle Optimization (SPO) algorithm and the Forward Floating Selection (FFS) were applied to the feature selection process, and the prediction performance, the computational complexity, the ability to avoid over-fitting and the feature reduction ability of these algorithms were compared. Feature combinations having the best recognition performance to the discrete emotions were given by the best feature selection method during the comparison of the above feature-space search strategies.The research and results of this thesis are as follows(1) An affective physiological signal database of 300 subjects was established, including six kinds of emotions (i.e. joy, surprise, disgust, grief, anger and fear) and eight kinds of physiological signals (i.e. GSR, HR, BVP, ECG, Rsp., facial EMG and two EEGs form the frontal lobe). The user-independent property given by the large number of subjects enabled the research results to be generalized well. Some of the eight kinds of physiological signals could be the reliable affective physiological feature extraction source.(2) By means of the analysis of RMT, it was found that the above 8 kinds of signals had the universal properties predicted by RMT. However, when the slow variation of the above signals was explored, marked deviations of the largest eigenvalue and the corresponding eigenvector component distribution from the RMT prediction were shown in the empirical data of GSR, HR, ECG and Rsp.; when the fast variation of the signals was explored, the above-mentioned marked deviations were only displayed in the empirical data of GSR and HR. Two conclusions can be drawn from the above two results:the signals of GSR and HR can be the reliable sources for feature extraction of emotion recognition; only the fast variation of GSR and HR can quickly respond to the emotional feeling, revealing the relationship between the affective physiological response and the time.(3) By comparing five kinds of feature selection methods, the Sequential Backward Selection (SBS) was found the best for the feature selection of two-class emotion recognition problems. And such comparison revealed the difference of the feature selection problem in emotion recognition from the other combinatorial optimization problem such as Traveling Salesman Problem (TSP).(4) The two-class emotion recognition systems established on the basis of RMT data analysis and the SBS feature selection method had good prediction performance, and the true positive rates of these systems were all 20% larger than their false positive rates. At the same time, the number of the selected features of each emotion recognition systems was less than 10, revealing the key features to distinguish a certain discrete emotion from the others. The database established on the affective physiological responses of 300 subjects ensured the user-independent property of the emotion recognition systems; the data analysis of RMT eliminated the disturbance of non-affective specific physiological responses to the affective specific physiological responses and increased the generalization performance of the research results; by comparing several searching algorithms, the best feature selection method was found; the best physiological feature subset of each emotion recognition system revealed the affective information encoding in the physiological signals.更多还原