节点文献
基于主体注意力与多空间域信息协同的多模态情感分析
Multimodal Sentiment Analysis Based on Dominant Attention and Multi-space Domain Information Collaboration
【摘要】 多模态情感分析在智慧教育中具有重要应用价值,例如通过分析学生的语言、表情和语调等多模态信息,来评估课堂参与度和情感状态,从而辅助教师实时调整教学策略。然而,现有多模态情感分析领域中,跨模态注意力机制对于异构模态间的关联捕捉不够充分,并且对共享空间与私有空间的信息协同并未进行深入探索,存在跨模态融合学习受限且多空间域信息协同不充分的问题。针对这些问题,文中提出了基于主体注意力融合多空间域异构模态的多模态情感分析模型,该模型通过主体注意力机制,对两个空间域中的异构模态分别进行充分融合,以解决跨模态融合学习受限的问题。然后,利用门控机制补充共享空间域异构模态融合向量的模态独立性,以实现私有空间与共享空间信息的协同,有效解决多空间域信息协同不充分的问题。实验结果表明,该模型在公共数据集MOSI和MOSEI上的得分整体都有提高,说明该方法可以充分捕捉多模态异构信息间的潜在关系并有效协同不同空间域的异构融合信息。
【Abstract】 Multimodal sentiment analysis has significant applications in smart education, such as assessing students’ engagement and emotional states through speech, facial expressions, and tone to help teachers adjust teaching strategies in real time.How-ever, existing cross-modal attention mechanisms struggle to capture associations between heterogeneous modalities effectively, and the collaboration between shared and private spaces remains underexplored, limiting multimodal fusion learning.To address these issues, this paper proposes a multimodal sentiment analysis model that integrates heterogeneous modalities across multiple space domains using dominant attention.This mechanism enables effective fusion of heterogeneous modalities in both domains, enhancing cross-modal learning.Additionally, a gating mechanism preserves the modality independence of shared-space fusion vectors, ensuring complementary interactions between private and shared spaces.Experimental results on the MOSI and MOSEI datasets demonstrate that the proposed model achieves overall performance improvements, validating its ability to capture and integrate heterogeneous multimodal information effectively.
【Key words】 Multimodal sentiment analysis; Dominant attention; Multi-space domain; Gating mechanism; Smart education;
- 【文献出处】 计算机科学 ,Computer Science , 编辑部邮箱 ,2025年S2期
- 【分类号】TP18
- 【下载频次】52