【摘要】 目的 对话情绪识别旨在准确捕捉对话中各话语的情绪状态，但面临两大挑战。其一，情绪状态随对话语境动态演变，且个体内部与说话者间存在结构异质的情绪依赖关系，增加了建模难度。其二，多模态信息融合时常出现语义错位和模态冗余，影响跨模态语义对齐与情绪线索的准确捕捉。方法 提出一种解耦情绪依赖关系的跨模态感知对话情绪识别模型（decoupled emotion dependencies and cross-modal awareness network,DECANet）。该模型通过结构解耦策略，将个体内部和说话者间的情绪依赖分别建模为两个独立子图，并设计启发式动态交互机制实现差异化建模与协同融合，精准捕捉多层次情绪演化特征。在多模态建模方面，设计跨模态上下文感知自注意力机制强化模态间深层语义关联，辅以语义一致性驱动的特征选择模块，有效筛选语义偏离和冗余信息，提升情绪表征的判别力。结果 在IEMOCAP(interactive emotional dyadic motion capture database)和MELD(multimodal emotion lines dataset)数据集上的实验结果表明，DECANet在准确率与加权F1值两个关键指标上均优于多种主流方法。在IEMOCAP上分别提升1.74%和1.77%；在MELD上亦取得了具有竞争力的性能增幅。消融实验进一步验证了情绪依赖解耦建模和跨模态交互机制的有效性。结论 DECANet能够更清晰地区分并建模对话中异构的情绪依赖结构，增强多模态语义对齐，减少信息冗余，在复杂交互场景中展现出较强的情绪识别能力和良好的泛化性能。更多还原

【Abstract】 Objective Emotion recognition in conversations(ERC) aims to identify the emotional state of each utterance within dialogues.Unlike traditional emotion recognition,which typically classify emotions in isolated utterances,ERC must account for the dynamic evolution of emotions shaped by contextual cues throughout the dialogue.Two key emotional dependencies are crucial:intra-speaker dependencies,representing emotional continuity or variation within the same speaker,and inter-speaker dependencies,reflecting emotional influence between speakers.Properly modeling both dependencies is essential for tracking emotional flow and transitions over time.However,existing methods often conflate these structures,failing to distinguish their unique characteristics,thereby limiting their performance.Concurrently,ERC relies heavily on multimodal fusion of textual,acoustic,and visual cues to capture the full emotional context.Yet,current fusion strategies frequently suffer from semantic misalignment,noisy or conflicting modality signals,and inadequate discrimination of relevant information.These shortcomings result in suboptimal multimodal representations and impair recognition accuracy.Together,these challenges point to the need for a unified framework that can simultaneously disentangle emotional dependencies and enable robust cross-modal semantic integration.Method A decoupled emotion dependencies and cross-modal awareness network,named DECANet,is proposed for ERC to address the aforementioned challenges.The core idea of DECANet lies in the structural disentanglement and dynamic integration of emotional dependencies.Specifically,two distinct subgraphs are constructed to separately model intra-speaker and inter-speaker emotional relationships.The intra-speaker subgraph tracks emotional continuity or fluctuation within the same speaker’ s dialogue turn,while the inter-speaker subgraph captures emotion shifts triggered by interaction among different speakers.Each subgraph is processed using graph attention networks augmented with learnable relation-type embeddings.These embeddings encode various temporal and emotional relations,enabling more precise and context-sensitive propagation of affective cues across the graph structure.A heuristic dynamic interaction strategy is introduced to bridge these two emotional structures while preserving their independence.Drawing inspiration from Siamese architecture,the model concatenates original node features with the outputs of both subgraphs to preserve foundational semantic information and ensure representation alignment.Element-wise difference operations are used to quantify semantic divergence between representations,whereas elementwise multiplication captures their semantic consistency.These composite features are then passed through a gating mechanism,which adaptively learns fusion weights on the basis of the current conversational context.This selective integration strategy enhances emotional discrimination by emphasizing the more informative dependency pathway under different conditions.A context-aware self-attention mechanism is developed to further improve cross-modal semantic alignment.This component performs iterative refinement of audio and visual modality representations through sequential integration of contextual cues from other modalities.For example,audio features are initially refined via context-aware interaction with textual representations,followed by a second-stage fusion with visual features,conditioned on the updated audio-text representation.This sequential alignment strengthens inter-modal cohesion and reduces modality gaps.Additionally,speaker embeddings and positional embeddings are incorporated to capture structural cues inherent in multi turn,multispeaker dialogues.These embeddings help encode speaker-specific emotional tendencies and temporal structure within the conversation,facilitating context-aware emotion recognition.A semantic consistency-driven feature selection mechanism is employed to address the issue of semantic noise and inconsistencies often introduced during multimodal fusion.This mechanism selectively preserves only those multimodal representations that maintain high semantic similarity with their corresponding unimodal features.By filtering out semantically deviating or redundant signals,this process helps maintain the integrity of emotion-related information and enhances the robustness of the final emotion representations.Finally,the entire model is implemented under the supervision of multimodal and unimodal objectives,ensuring that the learned features retain discriminative capacity while maintaining robustness across modality configurations.Result DECANet was evaluated on two widely-used ERC benchmarks,IEMOCAP and MELD,consistently demonstrating competitive performance across multiple evaluation metrics.On IEMOCAP,it achieved improvements of 1.74% in accuracy and 1.77% in weighted F1 score.On MELD,it attained gains of 0.63% in accuracy and 0.52% in weighted F1.These results demonstrate the effectiveness and generalizability of DECANet across diverse conversational scenarios.Comprehensive ablation studies further validate the functional contribution of each proposed component.The removal of either the intra-speaker or inter-speaker subgraph leads to notable performance degradation,underscoring the complementary roles of the two emotional dependency structures.Comparative experiments with single-graph modeling,parallel fusion,and hierarchical fusion strategies reveal that the proposed heuristic dynamic interaction strategy consistently achieves superior results by enabling adaptive and contextsensitive subgraph integration.Furthermore,the cross-modal context-aware self-attention mechanism is essential for effective multimodal fusion.Replacing it with conventional cross-attention or removing speaker and positional embeddings significantly reduces model performance,highlighting the importance of fine-grained inter-modal alignment and contextual sensitivity.In addition,the semantic consistency-based feature selection module effectively filters out semantically inconsistent signals,thereby enhancing the robustness and discriminative quality of the resulting multimodal representations.Conclusion DECANet effectively disentangles and models intra-and inter-speaker emotional dependencies in multispeaker conversations.It further achieves fine-grained semantic alignment across modalities through an iterative and selective fusion mechanism,enhancing the robustness and accuracy of multimodal emotion recognition.Extensive experiments verify that DECANet offers a generalizable and interpretable solution for understanding emotions in complex dialogue scenarios.更多还原