【作者】 刘江；

【导师】 姬东鸿；

【作者基本信息】 武汉大学 ， 网络空间安全， 2023， 硕士

【摘要】 自然语言处理可以用于网络空间安全的工作流程,以帮助进行漏洞保护,识别,规模和范围分析。其中,命名实体识别是自然语言处理中的一项基本任务,它为其他任务提取主要信息奠定了一定的根基,例如,事件分析,图谱构建等。该任务的目的是从文本中抽取出预先定义好的类型的实体,例如人名、地名、组织名以及网络安全相关的专有实体等。虽然研究人员已经对命名实体识别展开了众多的研究,但是,在一些场景下,现有的模型并不能完全胜任。如:(1)在医学领域中,大多数实体是不连续的,这样的实体很大程度上增加了现有模型识别出正确实体的难度。近年来,基于网格标记的方法在不连续命名实体识别中取得了很好的性能。但是,最先进的网格标记模型仍然存在一些问题。(2)在新兴领域中,相应的语料库缺乏,而构建充足的语料库是耗时且复杂的,那么在这样的少样本场景下,现有的模型也很难识别出实体。其中基于提示调优的方法在小样本命名实体识别中有着显著的性能,但它们仍然不能充分利用知识。本文针对在这两种复杂场景下存在的问题,通过标签和知识增强构建实体识别模型。本文的主要研究内容有如下三个方面:1.基于标签增强的网格标记模型用以不连续实体识别。本文设计了两种面向标签的增强方法来优化最先进的网格标记模型,称作TOE。首先,本文设计了一个标签表示嵌入模块,以迫使本文的模型不仅考虑词-词关系,还考虑词-标签和标签-标签关系。另一方面,在最先进模型中的下邻词和尾头词标签的激励下,本文添加了两个新的对称标签,即前邻词和头尾词,以建模更细粒度的词字关系,并减轻标签预测的错误传播。本文在CADEC,Sh ARe13和Sh ARe14三个基准数据集上进行广泛实验,本文的TOE模型在F1中将最先进模型的结果提高了约0.83%、0.05%和0.66%,证明了其有效性。2.基于三重知识增强的深度提示调优模型用以小样本实体识别。本文设计了一种基于三重知识增强的深度提示调优模型,称为TKDP。本文建议将深度提示调优框架与三重知识相结合,包括上下文知识、标签知识和义素知识。TKDP对三个特征源进行编码,并将它们合并到软提示嵌入中,软提示嵌入被进一步注入到现有的预训练语言模型中,以促进预测。本文在五个基准数据集上的5-/10-/20-shot的设置下进行了实验,大部分情况下本文的方法是优于8个性能强大的基线系统,在小样本命名实体识别中显示出巨大潜力。3.基于标签增强和知识增强的小样本不连续实体识别。本文将标签增强的不连续实体识别模型和知识增强的小样本实体识别模型以及它们的结合用以解决小样本的不连续实体识别任务,本文在小样本不连续实体识别数据集的20-shot设置下进行了实验,并与基线模型对比和自我对比,实验结果表明本文所提出的两个模型是可以很好的结合的,并可以用于处理更复杂的实体识别任务。本文对不连续实体识别,小样本实体识别和小样本不连续实体识别等复杂场景下的任务进行了研究并设计了相应的解决方案,主要创新在于设计了标签表示嵌入模块并扩充了标签体系,设计了义素知识融合模块和知识增强提示构建模块将丰富知识融入提示中,并且在多个数据集上进行了实验,其效果都优于所对比的基线模型,证明本文所设计的模型是有效的。另外,本文还进行了更广泛的实验来分析和理解本文的模型。希望本文的工作能够对命名实体识别的研究提供有价值的参考。更多还原

【Abstract】 Natural language processing(NLP)can be used in cyber security workflows to aid in vul-nerability protection,identification,size and scope analysis.Among them,Named entity recog-nition(NER)is a basic task in NLP,which lays a foundation for other downstream tasks to extract main information,such as event analysis,graph construction,etc.The purpose of this task is to extract predefined types of entities from text,such as person names,place names,organization names,and proprietary entities related to cyber security.Although researchers have done a lot of research on named entity recognition,in some complex scenarios,existing models are not fully competent.For example:(1)In the medical field,most entities are discontinuous.Such entities It greatly increases the difficulty of ex-isting models to identify the correct entity.In recent years,grid-tagging-based methods have achieved good performance in discontinuous NER.However,there are still some problems with the state-of-the-art(SOTA)grid-tagging model.(2)In emerging fields,the corresponding cor-pus is lacking,and building a sufficient corpus is time-consuming and complicated,so in such a few-shot scenario,it is difficult for existing models to recognize entities.Among them,prompt tuning methods achieve remarkable performance in few-shot NER,but they still fail to make full use of knowledge.This paper aims at building an entity recognition model through tag and knowledge augmentation in these two complex scenarios.The main research content of this article includes the following three aspects:1.Tag-enhanced Grid-tagging Model for Discontinuous NER.This paper design two kinds of Tag-Oriented Enhancement methods to optimize the SOTA grid-tagging model,which this papere call TOE.First,this paper design a Tag Representation Embedding Module(TREM)to force our model to consider not only word-word relations,but also word-tag and tag-tag relations.On the other hand,motivated by the Next-Neighboring-Word(NNW)and Tail-Head-Word(THW)tags in the SOTA model,this paper add two new symmetric tags,namely Previous-Neighboring-Word(PNW)and Head-Tail-Word(HTW),to model more fine-grained word-word relationships and alleviate error propagation from tag prediction.This paper conduct extensive experiments on three benchmark datasets,CADEC,Sh ARe13 and Sh ARE14,and our TOE model improves the results of state-of-the-art models by about 0.83%,0.05%and 0.66%in F1,proving its effectiveness.2.Threefold Knowledge-enhanced Deep Prompt Tuning Model for Few-shot NER.This paper design a Threefold Knowledge-enhanced Deep Prompt Tuning Model,which call TKDP.This paper propose incorporating the deep prompt tuning framework with threefold knowledge,including context knowledge,label knowledge and sememe knowledge.TKDP encodes the three feature sources and incorporates them into the soft prompt embeddings,which are fur-ther injected into an existing pre-trained language model to facilitate predictions.This paper conducted experiments on 5-/10-/20-shot settings on five benchmark datasets.In most cases,our method is superior to eight powerful baseline systems,showing great potential in few-shot named entity recognition.3.Few-shot Discontinuous NER Based on Tag-enhanced and Knowledge-enhanced.This paper use the tag-enhanced discontinuous entity recognition model and the knowledge-enhanced few-shot entity recognition model and their combination to solve the few-shot discontinuous entity recognition task.This paper conducted experiments under the 20-shot setting of the few-shot discontinuous entity recognition dataset,and compared with the baseline model and self-comparison.the experimental results show that the two models this paper propose can be well combined and can be used to handle more complex entity recognition tasks.This paper studies tasks in complex scenarios such as discontinuous entity recognition,few-shot entity recognition and few-shot discontinuous entity recognition,and designs corre-sponding solutions.The main innovation lies in the design of the tag representation embedding module and the expansion of the tag system,the design of the sememe integration module and a knowledge-enhanced deep prompt building module to integrate rich knowledge into the prompt,and conducted experiments on multiple datasets,and the effect is better than that of compared with the baseline model,it proves that the model this paper designed is effective.Additionally,this paper conduct more extensive experiments to analyze and understand our model.This paper hope that our work can provide valuable reference for the research of named entity recognition.更多还原