【作者】 李斌；

【导师】 周世杰；

【作者基本信息】 电子科技大学 ， 工程博士（专业学位）， 2025， 博士

【摘要】 网络媒体在给人们带来便利的同时,也成为虚假内容恣意传播的渠道。因此,探索准确高效的虚假内容检测技术具有极高理论价值和现实意义。本论文通过分析互联网网络媒体的特点,研究多模态特征融合的虚假媒体内容监测技术,解决虚假媒体内容检测识别问题,主要贡献包括如下四部分。(1)提出了一种时空融合特征的深度伪造人脸视频鉴别方法。该方法针对人脸面部特征区和面部边缘区的深度伪造视频未知篡改检测通用性不足问题,突破了采集混合训练样本、深度卷积泛化处理、训练时空融合特征等技术,提高了伪造人脸视频的鉴别成功率。选取Face Forencies++数据集进行验证,实验结果表明,Face Swap和Deepfakes两种真伪视频识别具有较高的准确率和置信度,实现了对人脸深度伪造视频的准确鉴别。(2)提出了一种基于深度离散哈希的虚假图文跨模态检索方法。该方法针对文本图像跨模态异构数据检索累计误差失效问题,突破图文倒排索引构建、高维语义特征分析、相似度跨模态鉴别等技术,生成检索区分度更高的哈希编码。选取Wikipedia、Flickr30K、NUS-WIDE三个数据集实验验证,实验结果表明,在图像检索文本和文本检索图像两项任务中,较常用最好方法的平均检索精度有小幅提升,证明该方法在跨模态图文检索时具有优越性。(3)提出了一种基于语义特征融合的多模态虚假新闻检测方法。该方法针对多模态语义特征表达不一致和识别不准确问题,突破图文特征向量化、语义一致性映射、多模态特征融合等技术,采用BERT和VGG19模型,进行特征向量化表示,最后加权联合得到虚假新闻检测结果。选取Fake News Net中的Politi Fact和Gossip Cop数据集进行验证,实验结果表明,准确率和召回率较常用最好方法有小幅提升,证明该方法能够提升虚假社交媒体新闻内容检测性能。(4)搭建了多模态虚假媒体内容监测平台进行应用验证。该平台针对多渠道海量虚假媒体应用监管能力不足问题,突破了多通道媒体数据采集、媒体内容检索与识别、人机共判的检测评估、基于密码的安全共享等关键技术。研发的虚假媒体内容监测识别引擎和分析工具,针对微信、微博、客户端等互联网平台的谣言、虚假图片、伪造视频等虚假内容的识别速度和准确率指标满足实际应用要求,并在网信办、公安、运营商等行业监管中得到了应用。更多还原

【Abstract】 While online media brings convenience to people,it has also become a channel for the rampant dissemination of false content.Therefore,exploring accurate and efficient false content detection techniques has high theoretical value and practical significance.By analyzing the characteristics of internet network media,dissertation studies the false media content monitoring technology based on multi-modal feature fusion to solve the problem of false media content detection and identification.The main contributions include the following four parts.(1)A deepfake face video authentication method based on spatiotemporal fusion features is proposed.This method addresses the issue of insufficient universality in detecting unknown tampering in deepfake videos of facial features and facial edges.It breaks through techniques such as collecting mixed training samples,deep convolution generalization processing,and training spatiotemporal fusion features,improving the discriminative power of deepfake facial videos.The Face Forencies++dataset was selected for validation,and the experimental results showed that Face Swap and Deepfakes have high accuracy and confidence in recognizing genuine and fake videos,achieving accurate identification of deepfake facial videos.(2)A cross modal retrieval method for fake images and text based on deep discrete hashing has been proposed.This method addresses the problem of cumulative error failure in cross-modal heterogeneous data retrieval of text images.It breaks through techniques such as constructing inverted indexes for text and images,analyzing high-dimensional semantic features,and identifying similarity across modalities to generate hash codes with higher retrieval discrimination.Three datasets,Wikipedia,Flickr30K,and NUS-WIDE,were selected for experimental verification.The experimental results showed that the average retrieval accuracy of the commonly used best method was slightly improved in both text retrieval and image retrieval tasks,proving the superiority of this method in cross modal image and text retrieval.(3)A multi-modal false news detection method based on semantic feature fusion is proposed.This method addresses the issues of inconsistent expression and inaccurate recognition of multi-modal semantic features.It breaks through techniques such as image text feature vectorization,semantic consistency mapping,and multi-modal feature fusion,and uses BERT and VGG19 models for feature vectorization representation.Finally,a weighted joint approach is used to obtain false news detection results.The Politi Fact and Gossip Cop datasets from Fake News Net were selected for experimental verification.The experimental results showed that the accuracy and recall were slightly improved compared to the commonly used best methods,proving that this method can improve the performance of detecting fake social media news content.(4)Built a multi-modal fake media content monitoring platform for application verification.This platform addresses the issue of insufficient regulatory capabilities for massive fake media applications across multiple channels,and has broken through key technologies such as multi-channel media data collection,media content retrieval and recognition,human-machine co judgment detection and evaluation,and password based secure sharing.The false media content monitoring and identification engine and analysis tool developed meet the actual application requirements for the identification speed and accuracy indicators of rumors,false pictures,forged videos and other false content on We Chat,Weibo,clients and other internet platforms,and have been applied in the supervision such as the Cyberspace Administration of China,public security,and telecom operators.更多还原