【作者】 傅宇；

【导师】 董恩清；

【作者基本信息】 山东大学 ， 先进制造（专业学位）， 2023， 博士

【摘要】 医学影像辅助诊断是医学影像分析领域的关键问题之一,在现代临床疾病诊断中发挥着重要作用。近年来,深度学习技术凭借其优异的特征表达能力和强大的数据拟合性能,已经被广泛地应用于各种医学影像辅助诊断任务中。然而,作为一种数据驱动的建模方法,深度学习严重依赖大量样本训练模型,并且数据集中不同类别间样本数量不均衡的问题也会严重影响深度学习模型的性能表现。受限于样本获取途径少、标注成本高、数据隐私性强以及不同类型疾病发病率差异大等问题,医学影像数据集往往存在样本数量少和类间样本数量不均衡的问题,这给构建高性能的深度学习分类模型带来了极大挑战。为应对上述挑战,本文聚焦于医学影像辅助诊断中的小样本分类和类不均衡分类关键问题,从多任务学习、注意力机制、损失函数优化和半监督学习等关键技术入手,提出多种小样本分类方法与类不均衡分类方法,有效降低了深度学习模型对庞大数据量和不同类型间样本数量均衡性的要求。本文的主要研究内容和创新点概括如下:(1)提出一种基于多任务学习和孪生网络的小样本分类方法。针对深度学习模型在小样本医学影像辅助诊断任务中难以提取类间样本区分性特征的问题,基于多任务学习,通过设计能够获取同类样本相似性和不同类样本差异性的辅助分类任务,并将其与常见的分类问题相结合,构建了 一个深度孪生网络(Deep Model with Siamese Network,DS-Net),有效地提升了深度学习模型在小样本数据集上的分类性能。DS-Net模型由一个基于孪生网络的辅助监督网络(Auxiliary Supervision Network,ASN)和一个普通的分类网络(Classification Network,CN)组成。在DS-Net模型的构建过程中,为了增强模型的特征提取能力,提出膨胀残差模块(Dilated Residual Block,DRB)和DRB网络,并基于DRB网络构建ASN和CN的主干网络,使模型能够提取同一尺度不同感受野的特征。在模型训练时,DS-Net模型利用成对学习的思想,以成对的数据作为输入,ASN通过判断成对样本是否属于同一类来提取类间差异和类内相似特征;分类网络执行目标分类任务,并利用ASN提取的区分性特征提高自身分类性能。在测试阶段,DS-Net模型无需使用成对样本,仅利用单一输入数据即可得出预测结果。在基于病理图像的骨肉瘤坏死区域评估问题上的实验结果显示,DS-Net能够获得95.10%的平均准确率,取得了目前最好的诊断结果。(2)提出一种基于密集连接注意力机制的小样本分类方法。针对深度学习模型在小样本医学影像辅助诊断任务中对样本关键区域关注度不够的问题,从空间注意力机制的角度出发,提出一个密集连接注意力网络(Densely Connected Attention Network,DenseANet),并在基于胸部CT影像的新冠肺炎(新冠感染)智能诊断任务中进行验证。在模型构建过程中,为了充分利用深度学习模型中的自注意力特征,设计了一个用于生成强注意力特征的密集连接注意力模块(Densely Connected Attention Block,DAB),并通过密集连接DAB模块内部和DAB模块之间相同尺度的注意力特征,构建了密集连接的注意力子网络(Densely Connected Attention Sub-Network,DA-SNet);同时,为了进一步增强模型对高阶特征的表征能力,在模型末端设计了一个能够密集连接不同尺度注意力特征的注意力特征聚合模块(Densely Connected Attention Feature Aggregation Block,DA-FAB),进一步增强模型的特征表达。随着深度学习模型网络层数的增加,不同尺度和不同深度的空间注意力特征被密集连接而逐渐向模型后端传递,使DenseANet模型能够依据强注意力特征输出诊断结果。实验结果表明,DenseANet可以有效定位被SARS-CoV-2病毒感染的肺部病变区域,并且能够以平均95.69%的准确率将新冠肺炎、普通肺炎和健康人群进行区分,优于现有的注意力模型。(3)提出一种基于主动注意力机制的小样本分类方法。针对深度学习模型在小样本医学影像辅助诊断任务中难以精确定位病变区域的问题,设计了一个基于先验知识的主动注意力网络(Prior Knowledge-based Active Attention Network,PKA2-Net)模型。PKA2-Net 由残差块、主体增强和背景抑制(Subject Enhancement and Background Suppression,SEBS)模块以及候选模板生成器构成,其中,模板生成器用来生成候选模板,以描述特征图中不同空间位置的重要性;SEBS模块是PKA2-Net的核心,用于生成主动注意特征以增强模型对病变区域的定位能力。从结构上看,基于突出明显特征并抑制无关特征能够提高分类效果这一先验知识设计的SEBS模块能够主动生成监督信息,进而通过校准当前特征生成主动注意力特征。在PKA2-Net模型中,主动注意力特征的生成过程不需要高级特征对低级特征的校正过程,解决了某些不准确的高级特征可能引起注意力特征过度分散,进而导致病变定位过程出现偏差的问题。本文在基于胸部X射线影像的肺炎诊断任务上对所提PKA2-Net模型进行验证,实验结果表明,在公开的ChestXRay2017数据集上,PKA2-Net能够以97.28%的准确率和0.9846的敏感度识别肺炎患者,获得了目前最先进的分类结果。(4)提出一种基于调和损失函数的类不均衡预测方法。针对深度学习模型在不均衡医学影像预测问题中对少数类(样本数量较少的类别)样本识别能力退化的问题,依据Precision-Recall(PR)曲线的曲线下面积(Area Under the PR Curve,AUCPR)对每类样本都敏感的特性,设计了一个具有统计学意义的调和损失函数(Harmony Loss)。由于AUCPR是在离散域上进行计算的,为了保证Harmony Loss连续可微,且存在平滑的梯度,首先利用Logistic函数在连续域上初步近似计算了 AUCPR;而后,为了提升训练过程中模型的优化速度,采用人工设定一定数量阈值的方式进一步近似计算AUCPR。通过以上两个近似计算过程,构建了一个梯度稳定且计算效率高的损失函数Harmony Loss。在模型优化时,Harmony Loss通过调和不同阈值下每一类预测结果的召回率和精确度,提升模型对少数类样本的识别能力,并保持模型训练曲线的稳定。本文在缺损颅骨3D重建和脑肿瘤分割两个密集预测任务,以及在糖尿病性视网膜病变分级和肺结节多种病理类型诊断两个稀疏预测任务中综合评估了 Harmony Loss的效果,实验表明本文提出的Harmony Loss能够极大地提升深度学习模型对少数类样本的敏感度,在不均衡预测问题中优于现有的损失函数。(5)提出一种基于半监督对抗学习的不均衡小样本分类方法。针对深度学习模型在小样本且类不均衡医学影像辅助诊断问题中容易产生过拟合以及降低少数类样本敏感度的问题,提出一个基于半监督学习的反向对抗分类网络(Reverse Adversarial Classification network,RACN),并在肺结节多种病理类型(腺癌、鳞癌、炎症和其他良性)诊断任务中进行验证。RACN模型由一个执行无监督回归任务的反向生成对抗网络(Reverse Generative Adversarial Network,RGAN)和一个有监督分类网络(Classification Network,CN)组成。在RGAN的设计过程中,指定了四个具有不同均值和方差的特定正态分布来代表肺结节的四种病理类型,并通过将肺结节映射到对应正态分布的随机抽样设计了一个特殊的不确定性回归任务,用于提取有助于增强CN分类能力的特异性特征。由于模型在每次迭代优化时,不确定性回归任务的目标是随机生成的满足假设性分布的向量,所以模型在训练时会引入一定的不确定性,有利于降低深度学习模型在小样本数据集上过拟合的风险;同时,正态分布具备的方差属性能够控制对应假设性分布的回归难度,进而控制RACN模型对少数类样本和多数类(样本数量较多的类别)样本的关注度,解决了深度学习模型在不均衡数据集中容易降低少数类样本召回率的问题。在自构建的不均衡小样本肺部薄层CT影像数据集上的实验结果显示,RACN模型在肺结节多种病理类型分类任务中获得了平均88.62%的准确率,且在公开的LIDC-IDRI数据集上能够以93.21%的准确率识别恶性结节,取得了目前最好的分类结果。更多还原

【Abstract】 Medical image-aided diagnosis is one of the key issues in the field of medical image analysis and plays an important role in modern clinical disease diagnosis.In recent years,deep learning technology has been widely used in various medical image-aided diagnosis tasks due to its excellent feature expression ability and powerful data fitting performance.However,as a datadriven modeling method,deep learning relies heavily on a large number of samples to train the model,and the imbalance in the number of samples between different categories in the dataset will also seriously affect the performance of deep learning models.Due to existing problems such as few sample acquisition channels,high labeling costs,strong data privacy,and large differences in the incidence of different disease types,medical imaging datasets often suffer from small number of samples and an imbalance in the number of samples between categories,which will bring great challenges to the construction of high-performance deep learning classification models.In order to cope with the above challenges,this thesis focuses on the key issues of small sample classification and class-imbalance classification in medical image-aided diagnosis,starting with key technologies such as multi-task learning,attention mechanism,loss function optimization as well as semi-supervised learning,and proposes a variety of small sample classification methods and class-imbalance classification methods,which effectively reduce the requirements of deep learning modeis on massive data and the balance of sample numbers between different categories.The main research contents and innovations of this thesis are summarized as follows:(1)A small sample classification method based on multi-task learning and Siamese networks is proposed.Aiming at the problem that the deep learning model is difficult to extract the distinguishing features of samples between categories in the small-sample medical imaging aided diagnosis task,based on multi-task learning,by designing an auxiliary classification task that can capture the similarity of samples in same category and the differences of samples between different categories,and then combining it with a common classification task,a deep model with Siamese network(DS-Net)is constructed,which can effectively improve the classification performance of deep learning models on small sample datasets.The DS-Net model consists of an auxiliary supervised network(ASN)based on the Siamese network and a general classification network(CN).In the construction process of DS-Net,in order to enhance the feature extraction ability of the model,a dilated residual block(DRB)and DRB network are proposed,which will then be used to construct the backbone networks of ASN and CN,so that the model can extract features of the same scale with different receptive fields.During model training.DS-Net uses the idea of paired learning and takes paired data as input.ASN extracts inter-class differences and intra-class similarity features by judging whether the paired samples belong to the same category;the classification network performs target classification tasks,and uses the discriminative features extracted by ASN to improve its own classification performance.In the test phase,DS-Net does not need to use paired samples,and can obtain prediction results only using a single input data.The experimental results on the task of necrotic area assessment in osteosarcoma based on pathological image show that DS-Net can obtain an average accuracy of 95.10%,achieving the best diagnostic result so far.(2)A small sample classification method is proposed based on a densely connected attention mechanism.Aiming at the problem that the deep learning model does not pay enough attention to the key region of the sample in the small-sample medical image-aided diagnosis task,from the perspective of the spatial attention mechanism,a densely connected attention network(DenseANet)is proposed,which is verified on the intelligent diagnosis task of corona virus disease 2019(COVID-19)based on chest CT images.In the construction process of DenseANet,in order to make full use of the self-attention features in the deep learning model,a densely connected attention block(DAB)for generating strong attention features is designed,and a densely connected attention sub-network(DA-SNet)is constructed by densely connecting attention features of the same scale within DAB blocks and between DAB blocks.At the same time,in order to further enhance the model’s ability to represent high-order features,an attention feature aggregation block(DA-FAB)that can densely connect attention features at different scales is designed at the end of the model to further enhance the feature expression.With the increase of the number of deep learning model layers,the spatial attention features at different scales and depths can be densely connected and gradually transmitted to the back end of the model,so that the DenseANet model can output diagnostic results based on strong attention features.Experimental results show that DenseANet can effectively locate lung lesions infected by SARS-CoV-2 virus,and can distinguish COVID-19,common pneumonia and healthy people with an average accuracy of 95.69%,which is better than existing attention models.(3)A small sample classification method is proposed based on an active attention mechanism.Aiming at the problem that the deep learning model is difficult to accurately locate the lesion region in the small-sample medical imaging aided diagnosis task,a prior knowledge-based active attention network(PKA2-Net)is designed.PKA2-Net consists of residual blocks,subject enhancement and background suppression(SEBS)blocks,and candidate template generators,where template generators are used to generate candidate templates to describe the importance of different spatial positions in a feature map.The SEBS block is the core of PKA2-Net,which is used to generate active attention features to enhance the model’s ability to localize lesion regions.From a structural point of view,the SEBS block designed based on the prior knowledge that highlighting obvious features and suppressing irrelevant features will improve the classification effect can actively generate supervision information,and then generate active attention features by calibrating current features.In the PKA2-Net model,the generation process of active attention features does not require the correction of high-level features to low-level features,which solves the problem that some inaccurate high-level features may cause excessive dispersion of attention features,which in turn leads to deviations in the lesion localization process.The proposed PKA2-Net is verified on the pneumonia diagnosis task based on chest X-ray images,and experimental results show that PKA2-Net can identify pneumonia patients with an accuracy of 97.28%and a sensitivity of 0.9846 on the public ChestXRay2017 dataset,achieving the state-of-the-art classification results.(4)A class-imbalance prediction method based on a novel Harmony loss function is proposed.Aiming at the problem that the deep learning model degrades the recognition ability of minority(classes with a small number of samples)samples in the unbalanced medical image prediction problem,according to the characteristic that the area under Precision-Recall curve(AUCPR)is sensitive to each category of sample,a statistically significant Harmony loss function(Harmony Loss)is designed.Since AUCPR is ralculated on the discrete domain to ensure that Harmony Loss is continusly differentiable and has a smoth gradient,we first approximat AUCPR on the continuous domain by using the Logistic function.Then,in order to improve the optimization speed of the model during training process,the AUCPR is further approximated by manually setting a certain number of thresholds.Through the above two approximate calculation processes,a Harmony Loss with stable gradient and high computational efficiency is constructed.During model optimization,Harmony Loss can improve the model’s ability to identify minority samples and can keep the model training curve stable by reconciling recall and precision of each category in prediction results under different thresholds.We comprehensively evaluated the effect of Harmony Loss on the two dense prediction tasks of 3D defective skull reconstruction and brain tumor segmentation,as well as the two sparse prediction tasks of diabetic retinopathy grading and diagnosis of various pathological types of pulmonary nodules.Experiments show that the proposed Harmony Loss can greatly improve the sensitivity of deep learning models to minority samples,and outperform existing loss functions in the imbalance prediction problem.(5)An unbalanced small-sample classification method is proposed based on semi-supervised adversarial learning.Aiming at the problem that the deep learning model is prone to overfitting and easy to reduce the sensitivity of minority class samples in the small sample and class imbalance medical image aided diagnosis issue,a reverse adversarial classification network(RACN)is proposed based on semi-supervised learning,which is validated in the diagnosis task of multiple pathological types(adenocarcinoma,squamous cell carcinoma,inflammatory and other benign diseases)of pulmonary nodules.The RACN model consists of a reverse generative adversarial network(RGAN)for performing unsupervised regression task and a supervised classification network(CN).In the design process of RGAN,four specific normal distributions with different means and variances are specified to represent four pathological types of pulmonary nodules.And then a special uncertainty regression task is designed by mapping pulmonary nodules to random samplings from the corresponding normal distribution to extract specific features that can help enhance the classification ability of CN.Since the goal of uncertainty regression task is a randomly generated vector satisfying the hypothetical distribution during each iteration of the model optimization,the model will introduce certain uncertainties during model training,which is beneficial to reduce the overfitting risk of deep learning models on small sample datasets.At the same time,the variance attribute of normal distribution can control the regression difficulty of corresponding hypothetical distribution,and then control the attention of RACN model to minority samples and majority(classes with a large number of samples)samples,which can solve the problem that the deep learning model is easy to reduce the recall of minority samples in unbalanced datasets.The experimental results on the self-constructed unbalanced small-sample lung thin-section CT image dataset show that the RACN model has achieved an average accuracy of 88.62%in the classification task of multiple pathological types of pulmonary nodules,and can identify malignant nodules with an accuracy of 93.21%on the public LIDC-IDRI dataset,obtaining the state-of-the-art results.更多还原