节点文献

准确性提示对虚假信息传播的干预作用——基于GPT模型的实验

The Intervention Effect of Accuracy Prompt on Misinformation Sharing——Experiment Based on GPT

  • 推荐 CAJ下载
  • PDF下载
  • 不支持迅雷等下载工具,请取消加速工具后下载。

【作者】 江浩扬; 彭小刚; 董艺涵; 朱晓隆; 彭晓哲;

【Author】 Jiang Haoyang;Peng Xiaogang;Dong Yihan;Zhu Xiaolong;Peng Xiaozhe;School of Psychology,Shenzhen University;National Engineering Laboratory for Big Data System Computing Technology;

【通讯作者】 彭晓哲;

【机构】 深圳大学心理学院; 深圳大学大数据系统计算技术国家工程实验室;

【摘要】 大语言模型(LLM)已广泛用于内容生成与审核,但LLM对虚假信息的辨别并非总是可靠,并且自身也时常表现出“幻觉”。探索准确性是否提示以及如何有效提高LLM信息辨别力,可以为理解LLM的类人行为及机制提供依据,也关系到如何以最简单有效的方式提升模型在真实世界中的信息审核与内容呈现。本研究系统探讨了准确性提示能否有效提升GPT对虚假信息的辨别能力。三个研究依次使用经典新闻材料、高推理负荷的TruthfulQA材料和未学习的新材料,均发现准确性提示显著提高了LLM的分享辨别力。结果表明,提示策略可能将LLM的注意转移到真实性维度,并激发模型内部认知机制切换至慎思模式、提高模型表现。研究结果为心理学干预方法与LLM认知机制的整合提供了理论与实证基础。

【Abstract】 Large language models(LLMs) are increasingly used for content generation and verification. However, their capability to accurately discern misinformation remains imperfect, frequently exhibiting confident yet incorrect outputs, known as "hallucinations". Investigating whether and how accuracy prompts—brief reminders prompting attention toward accuracy—can effectively enhance LLMs’ misinformation discernment abilities is crucial. Such inquiry not only advances our understanding of LLMs’ human-like cognitive processes and underlying mechanisms but also provides practical guidance for implementing simple yet effective interventions to improve content moderation in real-world applications.In this study, we systematically examined whether accuracy prompts could enhance misinformation discernment abilities of GPT models. Three sequential studies were conducted, each targeting a distinct cognitive and material context. In Study 1, we employed classical news headline materials previously used in human misinformation intervention studies to test the robustness of accuracy prompts. The results indicated that both GPT-3.5 and GPT-4o models demonstrated significantly improved sharing discernment after receiving accuracy prompts, characterized by reduced intentions to share false news and increased intentions to share true news. Notably, GPT-4o showed stronger improvements compared to GPT-3.5, suggesting that advanced LLMs may be better able to use such prompts to realign their internal cognitive focus toward accuracy considerations.Study 2 further tested the robustness and generalizability of accuracy prompts by employing Truthful QA, a dataset that is specifically designed to probe reasoning and common misconceptions. These materials required the model to engage in deeper reasoning processes and cross-domain knowledge. Consistent with Study 1, accuracy prompts robustly improved GPT models’ sharing discernment performance even in this cognitively demanding context. This suggests that the effect of accuracy prompt can generalize across different types of information and varying cognitive demands.To further clarify whether the observed improvements resulted from genuine reasoning or simple retrieval of training data, Study 3 utilized recently emerging news materials published after GPT models’ training cutoff date. Thus, the models could not rely on previously learned information. The results showed that accuracy prompts continued to significantly improve sharing discernment in GPT-4o, whereas GPT-3.5 showed limited improvement. These findings indicate that accuracy prompts effectively activate deeper cognitive processes, such as increased attention allocation towards assessing veracity and analytical reasoning, in advanced LLMs, thereby enhancing their capacity to evaluate novel misinformation.Collectively, these three studies provide robust empirical evidence that simple accuracy prompts effectively enhance misinformation discernment capacities in GPT models by shifting their internal attentional focus toward assessing informational accuracy and triggering deeper analytical processes. Crucially, the observed effectiveness across classical, high-reasoning, and novel materials underscores the robustness and practical applicability of accuracy prompts as cognitive interventions within LLMs.This research contributes theoretically and practically to the integration of psychological intervention strategies with artificial intelligence cognitive mechanisms. Specifically, it offers foundational insights for implementing psychologically informed interventions( "psychology for AI") that not only clarify cognitive analogies between human cognition and LLMs but also guide practical methodologies for enhancing LLMs’ information discernment capabilities, ultimately benefiting real-world misinformation management and digital content verification.

【基金】 国家自然科学基金(72371167);广东省哲学社会科学“十四五”规划(GD23CXL03);深圳市科技计划(KCXFZ20230731093600002)的资助
  • 【文献出处】 心理科学 ,Journal of Psychological Science , 编辑部邮箱 ,2025年04期
  • 【分类号】B842;TP18
  • 【下载频次】53
节点文献中: 

本文链接的文献网络图示:

本文的引文网络