【摘要】 大语言模型已广泛应用于日常学习、工作和生活中，但由于其参数规模庞大、资源消耗高，且推理高度依赖GPU，这严重制约其推广。针对上述问题，论文在CPU环境下提出基于离群特征优化的混合INT8量化方法，充分发挥其在模型压缩中的优势；同时，基于注意力机制在文本首尾集中分布的规律，设计高效的参数快速读取机制。两种方法的有机结合显著减少模型内存消耗和提升推理效率，为解决大语言模型在边缘计算环境中的应用瓶颈提供新的技术方案。在I7-13700 CPU环境下，基于LLaMA2、GPT-J和FSEQ大模型，使用C4、Wikitext和PG19数据集进行全面验证，结果充分验证了所提方法的优越性与实用价值。更多还原

【Abstract】 Large language models(LLMs) have been widely adopted in daily learning, work, and life. However, their enormous parameter sizes, high resource consumption, and reliance on GPUs for inference severely limit their widespread adoption. To address these challenges, a hybrid INT8(HI8) quantization method optimized using outlier parameters in a CPU environment was proposed to leverage its advantages in model compression. Simultaneously, a high-speed parameter rapid(HSR) retrieval mechanism was designed based on the observed pattern that attention mechanisms tend to concentrate at the beginning and end of texts. Memory consumption was substantially reduced, and inference efficiency was enhanced by the combination of these two methods, offering a novel technical solution to overcome the deployment bottlenecks of LLMs in edge computing scenarios. Comprehensive evaluations on the I7-13700 CPU environment using LLaMA2, GPT-J, and FSEQ models, with datasets such as C4, Wikitext, and PG19, demonstrate the superiority and practical value of the proposed approach.更多还原