【摘要】 提出一种基于生成对抗网络的多模态磁共振（MR）图像缺失模态合成方法，在不同缺失情况下均能预测出完整的模态序列。该模型将生成对抗网络作为模态合成的基本框架，生成对抗网络由生成器和判别器2个部分组成。其中生成器有3个阶段：特征提取、特征融合和图像生成。在特征提取过程中，使用U-Net网络进行编码。然后将4个模态的特征进行融合，并将融合结果输入混合注意力模块。这样使网络能够自适应地调整不同模态的权重，最后进行图像生成。判别器部分由4个相同的判别器组成，对4种不同的模态进行判别，其网络结构基于PatchGan进行改进。通过与其他具有代表性的模态合成算法比较，此方法无论在视觉效果还是客观评价指标上都取得了更好的效果，其中峰值信噪比均在23 dB以上，结构相似性均在0.99以上。更多还原

【Abstract】 An incomplete modality synthesis method for multimodality magnetic resonance(MR) images based on generative adversarial network(GAN) is proposed to predict the complete modal sequence in different scenarios. The proposed model uses GAN which consists of generator and discriminator as the backbone of the network. The generator contains feature extraction, feature fusion and image generation. U-Net is used for encoding during feature extraction, and the features of the 4modalities are fused before being input into the hybrid attention module, so that the network can adaptively adjust the weights for different modalities and finally generate images. The discriminator consists of 4 same discriminators which network structure is improved from PatchGan to discriminate different modalities. Compared with other state-of-the-art modality synthesis algorithms, the proposed method achieves better results in both visual effects and objective criteria, with peak signal-to-noise ratios higher than 23 dB and structural similarity index measurements above 0.99.更多还原