【摘要】 可解释性视觉问答(Explanatory Visual Question Answering, EVQA)在回答视觉问题的同时为推理过程生成用户友好的多模态解释，从而提高模型推理的可信度.然而，由于缺乏对视觉区域对象关系的有效建模，现有EVQA生成的解释文本存在视觉区域与语义不一致的问题.为此，文中提出粗细粒度因果关系协同驱动的可解释性视觉问答方法(Fine-to-Coarse Grained Causality Co-Driven Approach for Explanatory Visual Question Answering, FCGC-CoD).首先，建模视觉区域特征的因果关系，识别其中的主体对象和支撑对象，增强视觉与语言预训练模型的多模态表征能力.然后，设计联合变分推理网络，通过细粒度的多模态因果表征增强模型粗粒度的宏观因果推理过程，实现多模态解释和答案的生成.实验表明，FCGC-CoD在准确回答问题的同时，可提升解释的视觉推理一致性.更多还原

【Abstract】 Explanatory visual question answering(EVQA) generates user-friendly multimodal explanations for the reasoning process while answering visual questions. Thereby, the credibility of model inference is enhanced. However, due to the lack of effective modeling of visual regions object relations, the explanations generated by existing explanatory visual question answering(EVQA) models suffer from the problem of inconsistency between visual regions and semantics. To address this issue, a fine-to-coarse grained causality co-driven(FCGC-CoD) approach for explanatory visual question answering is proposed. First, the causal relationships of visual regions features are modeled, and the influential and supportive objects are identified to enhance the multimodal representation capability of the vision-and-language pretrained model. Then, a joint variational causal inference network is designed to strengthen the coarse-grained reasoning process through fine-grained multimodal causal representations, and thus the generation of multimodal explanations and answers is achieved. Experimental results demonstrate that FCGC-CoD enhances the visual reasoning consistency of explanations while answering questions accurately.更多还原