【摘要】 近年来，在线人工智能系统在众多领域展现出强大的推理能力，对社会产生了广泛的影响。在使用此类模型服务时，用户通常需将相关查询数据上传至云端平台以提供明确的任务指令。然而，这些查询数据可能包含隐私敏感或者机密信息，直接与云端平台共享会存在隐私泄露风险。此外，人工智能平台通常也会收集并利用用户数据进一步训练模型，可能导致用户的私有信息被生成式大模型记忆，并在后续公共服务中被生成并传播，从而加剧隐私泄露的可能性。现有生成式人工智能应用的隐私保护机制普遍依赖于针对提示词的脱敏技术，其安全性高度依赖敏感信息识别的准确性，通常需依赖大量标注数据进行隐私识别模型训练，不仅在实施成本上存在挑战，在训练过程中还极有可能引入新的隐私漏洞。为应对这一问题，本文提出一种新型隐私保护协同学习框架PrivateAI，该框架的核心思想是在严格保障隐私安全的前提下，充分利用分散在不同终端设备中的敏感数据，以训练本地隐私识别模型。同时，PrivateAI通过提取云端大模型推理过程中隐含的知识，并将其压缩为轻量级知识蒸馏数据集，实现对本地模型的高效性能增强。此外，针对标注数据和大模型蒸馏数据的异构性挑战，本框架引入了异构知识融合机制，用于对齐并整合来自基础模型与分布式标注数据的多源知识，从而显著提升隐私识别模型的泛化能力与隐私风险预警性能。为验证PrivateAI的实际效果，本文在两个真实医疗数据集上进行了系统评估。该框架能够在满足隐私约束的前提下，有效训练隐私识别模型，并对潜在隐私风险进行预警。在两个公开医疗数据集上的实验结果表明，PrivateAI训练得到的模型可最高提升53.7个百分点的隐私保护成功率。上述验证展现出PrivateAI在缓解隐私泄露风险方面的潜力，可作为在线智能应用中预防隐私泄露的有效工具。更多还原

【Abstract】 In recent years, state-of-the-art online artificial intelligence systems demonstrate remarkable capabilities in various fields, exerting broad social impacts. In order to access these model services, users are typically required to upload their personal data to the cloud platform. However, these queries may contain sensitive or confidential information, and directly sharing them with cloud platforms introduces potential privacy leakage risks. Moreover, platforms may exploit user data for further model training, causing private information to be memorized by the model and later regenerated in public services, thereby aggravating the risk of privacy breaches. Existing privacy-preserving mechanisms in generative AI applications predominantly rely on prompt sanitization techniques, whose security critically depends on the accuracy of sensitive information identification. These approaches usually require large amounts of annotated data for model training, which not only raises implementation costs but may also introduce new privacy vulnerabilities in specific scenarios. To address this issue, this paper proposes a novel privacy-preserving collaborative learning framework named PrivateAI. The core idea of this framework is to fully exploit sensitive data distributed across different devices to train local privacy identification models, while strictly ensuring data privacy. Meanwhile, PrivateAI extracts the implicit knowledge embedded in the large foundation models and compresses it into a lightweight distilled dataset, thereby achieving effective privacy detection performance enhancement of local models. In addition, to tackle the heterogeneity challenge between the knowledge extracted from labeled data and foundation models, the framework introduces a heterogeneous knowledge fusion mechanism that aligns and integrates multi-source knowledge from both the foundational models and distributed labeled datasets. We evaluate PrivateAI on two datasets, and the results demonstrate that models learned by PrivateAI can maximally improve the privacy protection success rate by 53.7 percentage points. PrivateAI holds significant potential in mitigating privacy breaches, acting as a sentinel against severe privacy leakage incidents within online AI applications.更多还原