【作者】 周佳庆； 李晓燕； 陈珂； 胡天磊； 陈刚；

【Author】 ZHOU Jia-qing LI Xiao-yan CHEN Ke HU Tian-lei CHEN Gang (Computer Science College,Zhejiang University,Hangzhou 310027,China)

【机构】 浙江大学计算机学院；

【摘要】 随着中文搜索引擎技术和海量数据挖掘技术的飞速发展,高效精确的中文地址判重技术作为其核心技术之一已成为学术界研究的焦点和热点。目前,面向中文的地址判重研究还尚未充分展开,且现有工作在判断同一地址的多种表述时均依赖领域知识,导致判重能力严重受限。据此,本文提出一种自学习的中文地址判重方法。算法引入地址规范度的概念,用于描述某个地址符合规范的程度,通过计算中文地址的规范度对规范的地址提取可能存在的冗余信息,程序自我学习并用于后续判断。实验表明,此算法在保证解析精度的前提下,显著降低了地址判重中误判漏判的比例。更多还原

【Abstract】 With the rapid growth of technology in Chinese search engine and mass data mining,highly efficient and accurate Chinese address de-duplication algorithm,as the major technology,has become a challenging topic of current research.Recent years have witness few breakthroughs in new techniques in Chinese address de-duplicatioa Most of the existing algorithms always refer to domain knowledge in judging near-duplicate address,incurring poor de-duplication performance.In this paper,we propose a novel self-learning Chinese address de-duplication algorithm.Firstly, we introduce the definition of address standard degree to measure the standardization of one address.We extract the possible redundancy information from these standard addresses by calculating the standard degree of Chinese address, and use this information for latter address de-duplication through a self-learning process.Experiments on real datasets show that,the proposed algorithm dramatically reduces the misjudge rate of Chinese addresses without sacrificing the parse accuracy.更多还原