【摘要】 在自然语言处理领域,全局注意力机制通过考虑编码器的所有隐藏状态来捕获信息,从而帮助预测结果。然而在理解汉语成语这种复杂的语言现象时,模型往往会因特定语境的干扰而产生错误的决定和认知。因此,为了使模型更好地感知成语在不同语境下的语法功能,该文提出了一种增强型的全局注意力机制,通过对每个位置空间产生额外的注意因子来调整原始的全局注意力,最终提高了模型对特定语义的学习能力。该文将增强型全局注意力机制与BERT语言模型相结合,设计了一个用于完形填空任务的模型,并在最近发布的中文成语完形填空数据集ChID上进行了实验。结果表明,相比于传统的BERT模型和全局注意模型,该模型取得的效果更优。更多还原

【Abstract】 To address the machine reading comprehension of some complex linguistic phenomena, such as Chinese idioms, we propose an enhanced global attention module to better perceive the grammatical functions of idioms in different contexts. We adjust original global attention by generating an extra attention factor for each spatial position, so as to enhance the recognition of specific word senses. We integrate this module with the popular BERT language model for Chinese cloze task. Results on a recently released cloze-test dataset ChID show that our method achieves significant improvements, compared with the fine-tuned BERT model and global attention model.更多还原