【作者】 庄艳；

【导师】 黄忠兵；

【作者基本信息】 四川大学 ， 生物医学工程， 2022， 博士

【摘要】 随着深度学习的快速发展,人工智能在医学影像、生物学领域、工业制造等领域已经取得了巨大的成就。由于深度学习端到端的数据驱动学习模式,缺乏特征提取和语义规则的逻辑关系,也缺乏预测结果的可解释性,从而也难以支持医疗诊断或医学研究中的因果推理。因此,人类专家仍然需要探索理解和追溯深度学习决策过程的方法,并且这已成为其在临床应用、生命科学研究中的关键性问题。因此,根据多元化的任务驱动型目标设计可解释性的深度学习模型,提升网络模型的准确性和可靠性,这是深度学习面临的最大挑战之一。本论文研究了针对不同结构差异度图像的识别模型,以及不同解释原理与复杂度的深度卷积网络模型在决策过程中的知识理解、再现和提取的过程与机制,旨在通过深度卷积网络可解释性研究,建立符合判别逻辑的生物医学图像识别方法,以进一步提升识别的准确率。研究主要基于两类问题:第一,深度卷积模型的因果可解释性,再现分类任务可理解的关注区域,从而提升模型准确性与可靠性,研究以基于细胞显微图像的哺乳动物结核分枝杆菌感染识别,以及基于胸部X射线图像的新型冠状病毒肺炎诊断为示范。第二,研究决策过程中的可理解的关键性特征,构建基于医学先验知识的多层级语义深度学习模型,从而提高医学诊断结果的准确性和可理解的诊断依据,研究以基于病理图像的膀胱癌风险分级为范例。本论文的具体研究内容如下。首先,以临床上急需发展的结核病(Tuberculosis,TB)检测技术为示范,研究基于相衬(Phase Contrast)显微图像的结核分枝杆菌(Mycobacterium tuberculosis,Mtb)感染细胞的识别方法。针对这类形态和结构差异较小的细胞显微图像,研究构建了基于迁移学习的深度卷积网络,实现了被Mtb感染的哺乳动物细胞的检测和肉眼无法区分的毒力特异性细胞(被不同毒力因子感染的细胞)的识别。然后,通过网络特征层激活图的可解释性研究,揭示了深度卷积网络模型识别过程中注意到的“关键性决策依据”是细胞边界,从而改进了细胞分类检测算法,使细胞保留更完整的边界形态,进一步提升了识别准确率。最后,联合生物实验验证,结果显示哺乳动物细胞系(人体巨噬细胞与小鼠巨噬细胞)中感染细胞在各时间点(2、12、24小时)的平均识别准确率从83.20%提高到了95.28%,敏感性从86.93%提高到94.16%,特异性从78.43%提高到96.10%。其次,针对模型决策过程中的“关注区域”,研究了基于胸部X射线(Chest X-ray)图像的新型冠状肺炎识别。这类医学图像的识别相比于上述类别具有病灶区域形态与分布多样性的特点。研究构建了基于特征的无监督目标检测算法和基于深度学习分类模型,实现了新型冠状肺炎、普通型肺炎和正常肺部的自动识别。最后,基于梯度加权类激活的可解释性分析方法,对决策过程中的关键区域实现了可视化,为放射科医生是否采纳AI识别结果提供了可靠的诊断依据,从而提升了诊断的透明性与可靠性。在这项研究中,共收集16,456例临床胸部X射线图像,其中标签完备的1000例测试数据集来自美国德克萨斯大学医学健康研究中心(University of Texas Medical Branch,UTMB)的临床数据。结果表明,所提出的研究方案对新冠阳性病例的敏感性为94.48%,对普通型肺炎的敏感性为84.88%,对正常肺部的敏感性为94.91%。最后,针对结构层次差异大的医学图像识别问题,研究了复杂度高的膀胱癌病理图像。由于这类图像往往无法准确地可视化图像中的关键决策区域,因此本文突出了关键识别特征在决策过程中的价值,提出了一种基于医学先验知识的多级语义特征深度学习可解释性模型。研究将深度卷积网络学习到的“代表性特征”(特征层特征)与基于医生经验的临床诊断特征相融合架构分类网络,实现了膀胱癌的风险分级(即:正常、低级别和高级别乳头状尿路上皮癌)。其研究结果表明,对于膀胱癌病理图像公开数据集(4253例),本论文提出的方法对正常、低级别、高级别乳头状尿路上皮癌的预测精确率分别为97.67%、78.22%、90.37%,总体预测准确率为85.21%,相比单纯使用深度卷积网络的识别准确率提高了8.45%。综上,本论文针对不同结构差异度图像的识别问题,对深度学习框架下的可解释性进行了深入研究,为结合不同结构层次的图像差异性分析可解释性提供了一种全新的角度,并在此基础上利用可解释性结果优化模型结构,有效地提高了深度卷积网络分类的准确率。这些研究结果为医学影像人工智能满足医学伦理要求,进一步提升临床运用的可靠性奠定了重要基础。更多还原

【Abstract】 Artificial Intelligence in general have demonstrated remarkable achievements in medical imaging,biology,industrial manufacturing and other fields,especially with the rapid development of deep learning.However,due to its end-to-end learning pattern,the data-driven learning model lacks the logical relationship between feature extraction,semantic rules and the interpretability for the prediction,which becomes a major obstacle to further explore the full potentials of causal reasoning in medical diagnosis or medical research.Therefore,human experts still need to investigate methods for understanding and tracing the transparent decision-making process of deep learning models,which has become a key issue in its clinical applications and life science research.Therefore,designing interpretable deep learning models according to diverse task-driven goals with superior performance and reliability is one of the biggest challenges in deep learning.In this thesis,the process and mechanism of knowledge understanding,reproduction and extraction of deep convolutional networks with different explanation principles and complexities for biomedical images in the decision-making process was studied.Therefore,we aim to establish interpretable deep convolutional networks that conform to discriminative logic,so as to further improve the accuracy and reliability of biomedical image recognition method.There are mainly two aspects included: on the one hand,we investigated the causal interpretability of deep convolutional models to reproduce understandable regions of interest for classification tasks,thereby improving accuracy and reliability for recognition model.Specifically,the classification methods of mammalian mycobacterial infections based on cell microscopic images were developed,and also for the diagnosis of COVID-19(Corona Virus Disease 2019)based on chest X-ray images.On the other hand,the comprehensible features in the decision-making process were studied by establishing a multi-level semantic deep learning model based on prior medical knowledge,and the model was applied in the risk grading of bladder cancer based on pathological images in order to improve the accuracy of prediction results and provide understandable diagnostic basis.The main research content of this thesis is as follows.Firstly,the identification method of Mycobacterium tuberculosis(Mtb)-infected cells was investigated based on phase contrast microscopic images,aiming at proposing a novel detection technology of Tuberculosis(TB),which needs to be developed urgently in clinical practice.For cell-level microscopic images with indistinguishable differences of morphological structure,a deep convolutional network was constructed based on transfer learning strategies to realize the Mtb-infected mammalian cells detection and the recognition of differentiated virulence-specific cells(cells infected with different virulence factors),which couldn’t be distinguished by naked eyes.Then,through the interpretability study of the activation maps of the network feature layers,the morphological changes of the cell boundary was revealed by the visualized "key decision basis",which was noticed in the recognition process of the deep convolutional network.Thereby,the detection algorithm was optimized to enhance boundary integrity of cell morphology to further improve the classification accuracy.Ultimately,the model performance was verified on the biological experiments,and the results have shown that the average accuracy of the recognition scheme for infected cells in mammalian cell lines(human macrophages and mouse macrophages)during the entire time points(2,12,and 24hours)increased from 83.20% to 95.28%,and its sensitivity was increased from86.93% to 94.16%,and its specificity from 78.43% to 96.10%.To handle the attention areas of interest in the model decision-making process,the identification of novel coronavirus pneumonia was studied based on chest X-ray images,which differed from the above category due to the characteristics of regional morphology and distribution diversity of lesion areas.Therefore,we introduced a novel diagnosis system based on chest X-ray images,which consisted of a feature-based unsupervised algorithm for target detection and a deep convolutional classification network to achieve the automatic recognition of COVID-19 pneumonia,common pneumonia and normal lungs.Finally,the interpretability analysis method was implemented based on gradient weighted class activation maps to visualize the attention lung areas in the diagnosis process,which provided a reliable diagnostic basis for whether radiologists adopt AI prediction results,thereby improving the model’s transparency and reliability.In this study,a total of 16,456 clinical chest X-ray images were collected,of which a fully-labeled test dataset of 1000 cases was obtained from clinical data from the University of Texas Medical Branch(UTMB).These results suggested that the proposed scheme achieved a classification sensitivity of 94.48%for COVID-19 positive cases,84.88%for the common pneumonia,and94.91%for the normal type.Finally,the medical interpretability across highly complex deep convolutional models was investigated in terms of pathological image recognition with large structural level differences.Since such situations usually have very limited performance on accurately visualizing the discriminative features in the images,a joint multi-level semantic interpretability model was presented based on medical prior knowledge to prominent the vital value of significant features in the decision-making process.The proposed method fused the "representative features"(feature layer features)learned by deep convolutional networks with clinical diagnostic features based on expert’s experience to construct a hierarchical classification network,which exhibited superior performance on risk grading of bladder cancer(namely,normal,low-grade and high-grade papillary urothelial carcinoma).These research results suggested that the prediction accuracy for normal,low-grade,and high-grade papillary urothelial carcinoma were 97.67%,78.22%,and90.37%,respectively,in the public data set of bladder cancer pathological images(4253 cases).The overall prediction accuracy was 85.21%,which was 8.45% higher than only using the deep convolutional networks.In summary,we make an extensive study for the interpretability under the deep learning frameworks for the recognition of images with diverse structural differences,which provides an novel perspective for analyzing interpretability in the issue of image discrepancy at multiple complexity levels.On this basis,the interpretable outcomes are used to optimize the model structure,which effectively improves the performance of deep convolutional network classification.These research results laid a pivotal foundation for upgrading the level of artificial intelligence in medical imaging that meets the requirements of medical ethics,and further promoting the reliability in the clinical applications.更多还原