基于自动摘要的无障碍移动新闻系统的设计与实现

【作者】 王晓林；

【导师】 陈纯；

【作者基本信息】 浙江大学 ， 计算机科学与技术， 2017， 硕士

【摘要】 随着移动互联网的飞速发展和移动智能设备的大规模普及,人们获取新闻信息的渠道也越来越多。视障群体由于在视力方面的缺陷,仅能依靠听觉和触觉从外界获取信息,不能像健全人一样便捷地使用手机、平板电脑等智能设备,获取新闻信息的途径较为匮乏,选择性极少。现有的新闻类应用软件很少有针对视障群体做过适配和优化,视障人士几乎无法使用,阅读效率非常低。因此,开发一款面向视障群体的无障碍新闻类软件具有十分重要的意义。在此背景下,实验室与中国盲文出版社合作希望能开发一款适合视障群体使用的无障碍移动新闻系统。本文首先从保证新闻数据的全面性和多样性出发,提出并实现了一种支持动态页面爬取的新闻网络爬虫。在URL爬取方面,该爬虫以无界面浏览器Htm1Unit为基础,通过模拟用户点击或滑动等行为触发动态网页的脚本事件,从而解决了动态页面中URL难以获取的问题。然后,在URL去噪方面,本文设计了一种基于正则表达式的去噪方法,可以有效去除非新闻链接,从而提高新闻网络爬虫的效率,避免资源浪费。在最后新闻数据提取方面,采集并定制模版,实现了高准确率的基于模版的新闻信息提取方法。其次,为了提高视障群体阅览新闻的效率,提出并实现了一种基于TextRank改进的自动摘要方法。本文首先采用BM25相似度计算方法实现了经典的TextRank算法,然后将新闻标题和小标题的结构作用考虑进来,通过计算新闻标题、小标题与各句子间的相似度得到调整向量,最后通过调整向量影响各句子收敛后的TextRank得分,经过实验分析改进后的方法优于经典的TextRank方法。最后,本文在以上研究的基础上设计并实现了面向Android平台的基于自动摘要的无障碍移动新闻系统,视障群体通过无障碍客户端可以更便捷更高效地获取新闻信息。更多还原

【Abstract】 With the development of mobile Internet and the popularity of mobile devices,people get more and more diversified channels of news information.Due to the visual defect,visually impaired people can only rely on hearing and touch to get information from the outside world,not as easy as the normal use of touchscreen cellphones and tablets.Access to news is extremely limited and less selective.Very few news applications are targeting or optimized for visually impaired people.Therefore,the development of an accessible news software is of great significance.In this context,our laboratory cooperates with China Braille Press and hopes to develop a news system accessible to the visually impaired people.Firstly,we propose a news crawler that supports dynamic pages crawling to ensure the comprehensiveness and diversity of news data.In the aspect of URL crawling,the crawler uses the non-interface browser HtmlUnit to trigger the script events by simulating click and slide,which solves the problem that the URLs in the dynamic pages are difficult to obtain.Then,in the aspect of URL denoising,this pager designs a denoising method based on regular expression,which can effectively remove the non-news links,so as to improve the efficiency of crawlers and avoid the waste of resources.In the last stage for news data extraction,we propose and implements a template-based news extraction method with high accuracy.Secondly,in order to improve the efficiency of users’ reading,we proposed a new automatic summarization method based on TextRank.In this paper,the classical TextRank algorithm is implemented by using the BM25 similarity calculation method.Then we use the similarity between news headline,subtitles and sentences to adjust scores of sentences calculated by TextRank,which takes the structural affect of news headline and subtitles into account.The experimental results show that the improved TextRank method is better than the classical one.Finally,we implemented the automatic summarization based news system with an accessible Android application.更多还原