【摘要】 针对传统方法预测肝细胞癌（HCC）疗效通常只采用图像信息、临床信息或基因信息等单一模态信息的问题，提出一种基于多模态融合注意力的HCC疗效预测方法。首先，使用残差网络（ResNet）提取图像特征和多层感知机提取临床特征和常规放射学特征；其次，构建一个多模态融合注意力模块，通过计算不同模态特征之间的相关性有效地融合图像特征、临床特征和常规放射学特征；最后，通过一个分类网络实现对HCC患者疗效的准确分类预测。实验结果表明，与单一模态预测疗效的方法相比，所提方法的准确率在实验数据集上提升了5.95个百分点，验证了所提方法能显著改善HCC患者疗效预测的结果。更多还原

【Abstract】 A multimodal fusion attention-based HepatoCellular Carcinoma(HCC) efficacy prediction method was proposed to address the problem that traditional methods for predicting the efficacy of HCC usually use single modal information such as image information,clinical information,or genetic information.Firstly,Residual Network(ResNet) was used to extract image features,and a multi-layer perceptron was employed to extract clinical and conventional radiological features.Then,a multimodal fusion attention module was constructed to efficiently fuse image features,clinical features,and conventional radiological features by calculating the correlation between different modal features.Finally,a classification network was used to accurately predict and classify HCC patient outcomes.The experimental results show that compared to the single-modality method for predicting efficacy,the accuracy of the proposed method increases by 5.95 percentage points on experimental dataset,confirming that the proposed method can significantly improve the prediction results of HCC patient efficacy.更多还原