【作者】 崔得暄；

【导师】 卢炎生；

【作者基本信息】 华中科技大学 ， 计算机软件与理论， 2006， 硕士

【摘要】 作为从万维网上海量的文本信息资源中发现潜在的有价值知识的一种有效技术,Web文本挖掘正方兴未艾。Web文本分类研究是Web文本挖掘中的一个研究热点。在Web文本分类中,通过更好地体现Web文本数据的特性可以提高机器学习领域中通用分类算法的性能,因此很有必要研究如何更好地将通用的机器学习算法同Web文本数据相结合的方法。在垃圾邮件过滤这个文本分类的重要应用中,必须考虑不同类别的文档被错误分类而导致的不同损失,为此定义了一个损失函数,结合贝叶斯定理,将邮件表示为布尔型向量并用信息增益公式进行特征筛选,给出了一种基于最小损失的垃圾邮件过滤方法。在公共垃圾邮件数据集PU1上进行了试验,试验结果验证了定义损失函数的有效性。为了更充分地体现文档的内在特性,把Web文本表示成以单词为最小元素的序列,考虑特征间的相互位置关系隐含的丰富语义信息,将计算生物学中DNA序列分析问题的一种解决方法应用到文本分类中,使用特征序列来描述一个类别的内在特征,给出了一种文本分类方法SSAM。在Reuters-21578数据集上进行了试验,结果显示SSAM的分类效果优于传统的贝叶斯方法,且具有较快的分类速度。基于SSAM分类方法,将Web文本分类过程划分为创建数据集、Web网页预处理、训练学习和实际分类等步骤,设计出一个可处理中文文本的Web文本自动分类系统,并用Visual C#在PC机上实现了系统的原型。更多还原

【Abstract】 As an effective technique to discover potentially valuable knowledge through the massive text information resources in the WWW, Web Text Mining is in its ascendant. Web Text Classification is a hotspot in the field of Web Text Mining. The performances of generic classifying algorithms in machine learning can be improved by making better use of the characteristics of web text data, so it’s necessary to study some better methods of combining generic classifying algorithms with web text data.As an important appliance of text classifying, a junkmail filtering system must consider different impacts of misclassifying different classifications. After defining a loss function and combining it with Bayes theorem, a minimal loss based filtering method is designed, which represents a mail as a Boolean vector and selects features with IG. The experimental results on PU1 verify the efficiency of defining such a loss function.To make full use of the characteristics of web documents, a web text is represented as sequences, in which a minimal element is a word, in order to consider the rich semantic information implied by the mutual positions among terms. A solution to DNA sequence analysis problems in computational biology is applied to text classification to implement a text classifying methods called SSAM, which uses signature sequences to describe the characteristics of classifications. The experimental results on Reuters-21578 indicate SSAM behaves better than Na?ve Bayes, and it has a high classifying speed.The web text classification procedure is divided into several steps, namely, constructing text collection, preprocessing web pages, training and classifying. An SSAM-based automatic web text classifying system, which can process Chinese text, is designed, and its prototype is implemented using Visual C# on a pc.更多还原