【摘要】 文本分类算法一般采用宏平均精度、宏平均召回率以及宏平均F₁值作为评价指标,然而同一个分类器在不同数据集上所得的评测数值往往存在很大差异,使得评测数值只在特定的数据集上有价值,而在其他数据集上没有意义.为了解决这个问题.本文提出3个因素来刻画数据集对分类结果的影响,并利用这3个因素构造一种评测指标,newmacro-F₁.这一评测指标将数据集的因素从评测过程中独立出来,使得,newmacro-F₁表示的仅仅是分类算法本身.实验结果表明使用该评测指标同一分类器在不同的数据集上波动较小.通过分类器在1个数据集上的表现,可以近似计算得到该分类器在另一个数据集上的分类质量.更多还原

【Abstract】 Macro average precision,macro average recall and macro average F₁ are usually used to evaluate classification technique.But those measures are sensitive to the datasets which means the measures are only valid for specific dataset but invalid for the others.To solve this problem, three factors are proposed to describe how datasets affect the classification result.Then a new evaluation method of categorization called new-macro-F₁ is presented according to the three factors.Experimental results show that the new measure remains stable on different datasets and through the performance of an algorithm on one dataset,the precision of other datasets could be estimated with the help of new measure.更多还原