【作者】 张蕾；

【导师】 李华旸；

【作者基本信息】 江西财经大学 ， 软件工程（专业学位）， 2020， 硕士

【摘要】 当今世界,中国经济蓬勃发展、国际竞争力不断提高,汉语的语言魅力吸引了越来越多的外国学习者进行学习,本文针对外国学习者学习汉语这一主题展开中文文本纠错的探究。中文文本的词语纠错方法研究是确保学习者在学习和交流时准确性的重要探究,是预估文本中错误存在与否并选取正确纠错字的关键技术手段,是中文自然语言处理研究领域中的重要课题。本文旨在辅助学习者纠正其在学习过程中产生的错误,同时也为减轻汉语教师的辅导压力。中文文本纠错研究从利于模型构建的角度出发,将纠错任务分成两大类:中文拼写纠错和中文语法纠错,分别构建模型进行纠错。在经由一系列详实的调查,概括了文本拼写问题的形成因素和类别,在N-gram语言模型基础上采用基于字的N元切分文本并统计其概率,引入混淆集和动态规划提高模型纠错效率,并使用平滑技术应对数据稀疏问题,提出了结合中文分词、二元模型和三元模型的算法,构建了基于N-gram的中文拼写纠错组合模型。针对中文语法纠错,本文将语法问题分为四大类:用词冗余、缺少词语、用词错误、词序错误。针对传统的基于统计的N元语言模型无法应对邻接词未登录以及长距离语法错误的问题,本文采用神经语言模型双向长短期记忆网络(BiLSTM)来建模,通过双向上下文信息来评测句子正误,此外,又引入了条件随机场(CRF)进行序列标注,并在嵌入层加入词性特征与词向量相结合作为BiLSTM输入,构建了基于BiLSTM-CRF的中文语法纠错模型,进一步优化了模型的纠错性能。利用开发评测数据集进行的实验结果表明,本文提出的基于N-gram的中文拼写纠错组合模型和基于BiLSTM-CRF的中文语法纠错模型在各自针对的错误领域内都取得了良好的效果。更多还原

【Abstract】 In today’s world,China’s economy is booming and international competitiveness is constantly improving.The charm of Chinese language has attracted more and more foreign learners to learn.This article explores the correction of Chinese text for the topic of foreign learners learning Chinese.The study of word correction methods in Chinese texts is an important inquiry to ensure the accuracy of learners in the process of learning and communication.It is a key technical means to predict the existence of errors in texts and select accurate correction words.It is the field of Chinese natural language processing research.Important topics in this article aims to assist learners to correct their mistakes in the learning process,and at the same time to reduce the pressure of Chinese teachers’ guidance.From the perspective of facilitating model construction,Chinese text error correction research divides error correction tasks into two categories: Chinese spelling error correction and Chinese grammar error correction,and constructs models for error correction.After a series of detailed investigations,the formation elements and categories of text spelling problems were summarized.Based on the N-gram language model,word-based N-ary text was used to segment the text and its probability was counted.Confusion sets and dynamic programming were introduced to improve model correction.Error efficiency,and use smoothing technology to deal with data sparse problems,proposed an algorithm combining Chinese word segmentation,binary model and ternary model,and constructed a Chinese spelling error correction combined model based on N-gram.In view of Chinese grammatical error correction,this article divides grammatical problems into four categories: word redundancy,missing words,word errors,and word order errors.Aiming at the problem that the traditional statistical-based N-gram language model cannot cope with the problem of unregistered adjacent words and long-distance grammatical errors,the paper applies the neural language model Bidirectional Long Short-Term Memory Network(Bi LSTM)to model and evaluate the sentence correctness through bidirectional context information.In addition,a conditional random field(CRF)was introduced for sequence labeling,and the combination of part-of-speech features and word vectorswas added to the embedding layer as Bi LSTM input.A Chinese grammatical error correction model based on Bi LSTM-CRF was constructed to further optimize the model correction.Wrong performance.The experimental results using the development evaluation data set display that the Chinese spelling correction model based on N-gram and the Chinese grammar correction model based on Bi LSTM-CRF proposed in this paper have achieved good results in their respective error fields.更多还原