多模态情感数据标注方法与实现

【作者】 夏凡； 王宏；

【机构】 清华大学计算机科学与技术系智能技术与系统国家重点实验室；

【摘要】 随着近年来计算机技术在图像及语音处理方面的快速发展,情感计算的多模态融合已成为人机交互方面的一个研究热点。由于大规模情感数据库的缺乏,情感计算的研究被限制在一些具体而零散的领域。传统的情感标注大多基于情感范畴模型或二维情感空间模型,本文分析了在建立情感数据库的过程中采用的基于三维情感模型的PAD量表,同时详细介绍了多模态标注工具的开发。更多还原

【Abstract】 The purpose of Affective Computing is to give computers an ability to observe,understand, and express different human emotions. Successful implementation of AffectiveComputing allows computers to interact with humans in a natural and amiable mannersimilarly to humans. Due to significant lack of affective data resources, Affective Computingis limited to specific and scattered research fields such as language and body gesture. Isolatedresearches contribute little into estimating and forming human emotional stages. In order tosolve this problem, National Science Foundation of China has established an importantresearch project “Research on Affective Computing Theory and Approach”. And one majorbranch of this project concerns with building a gigantic, multi-dimensional database to accessand process data simultaneously and therefore, shaping it into an integrated and precisedatabase for scientific research. Our research concentrates on creating an affective annotationtool that allows users to interact easily and efficiently with the perspective database under thenational project. The traditional 2-dimensional—valence and arousal affective space is able to describemost human emotions satisfactorily. Unfortunately, this model proves to be inefficient inprocessing several affective types: for instance, it fails to demonstrate the difference betweenanger and fear. Thus the entire model suffers from inaccuracy and the two-dimensional scopealso restricts future development of more sophisticated human emotions. In our project ofbuilding affective database annotation tool, we applied a three-dimensional PAD affectivemodel, which is a combination of the traditional model and a dominance dimension Thisinnovative model allows us to distinguish those affective types that cannot be recognized intwo-dimensional affective space. When building specific annotations, we used PADsimplification scale. The scale is developed from the Mehrabian PAD affective model. Theunderlying foundation of the table lies in the usage of a pair of words: in order to test thedatabase accurately, we set one word correspond with two out of three-dimensional PADaffective space. This primary step enables us to keep the traditional two-dimensionaladvantage while working on additional settings. Moreover, the other word of the pair showsthe difference among the third dimension. Adding an extra dimension into database testingenriches the pool of emotions computing machines are able to recognize. Therefore, using apair of words to test database allows us to access and reflect upon multi-dimensional affectivevalues. Furthermore, in order to make up for the deficiency of PAD three-dimensional affectiveannotation tools, the author here has created a multi-dimensional affective annotation toolbased on the XML file, named MAAT (Multi-model Affective Annotation Tool.) This toolresults from careful analysis of numerous affective annotation methods and is used in offlineannotation. The base file of annotation utilizes XML form, defines the category of informationthroughout annotation process such as the affective categories, the value of PAD, expressionmood and other experiment settings. The user can freely add on visual, sonic and other specialannotation information. The only additional step is defining a self-describing indicator inschema file. The author’s method of implementing affective annotation is extremelyconvenient, since it has already defined a set of PAD values, affective span and outlet ofstimulating settings. After all, the user only needs to select simple options when annotating. Inconclusion, this particular affective annotation control not only simplifies the data-searchprocess, but also expands the horizon of Affective Computing to include more dimensions todifferentiate human emotions. With a larger database and an efficient annotation method, thescope of scientific research in related field will be greatly widened. We sincerely hope that ourfindings are able to make valuable contributions to the national project of “Research onAffective Computing Theory and Approach”.更多还原