【作者】 梅涛；

【导师】 周荷琴；

【作者基本信息】 中国科学技术大学 ， 模式识别与智能系统， 2006， 博士

【摘要】 随着便携式电子记录设备的普及，越来越多的家庭用户开始使用这些设备来记录有意义的生活经历和日常生活片断，家用视频的数据量呈现出快速增长的趋势。但是，家用视频具有视觉和声音质量较差、内容没有经过编辑等特点。为了实现对家用视频内容的自动、有效的组织和管理，减轻普通家庭用户浏览、检索和共享视频内容的负担，视频内容分析已经成为家用视频领域一个重要的研究课题。对家用视频内容分析方法进行研究，不仅能为家庭用户提供个性化的多媒体服务，而且有助于推动视频领域的语义建模、机器学习和信息检索等相关课题的发展。由此可见，家用视频内容分析方法研究具有重要的理论和应用价值。 本论文从三个不同层次研究了家用视频内容分析中的关键问题，即质量评估、内容理解和内容表征，目标是实现一个自动理解、表征和浏览的家用视频处理系统。由于家用视频视觉质量较差的特点，为实现对家用视频内容的过滤，质量评估成为内容分析的一个基本步骤，处于最底层。内容理解主要解决对视频内容中的语义概念进行建模，处于比质量评估更高的层次。而内容表征则是基于质量评估和内容理解的结果，对家用视频内容以最紧凑和有效的方式进行表征；作为家用视频的一种应用，它处于最高层。这三个层次组成了家用视频内容分析的一个相对完整的框架。本文对家用视频内容分析方法进行了深入的研究，取得了以下成果： (1) 对于质量评估问题，针对目前家用视频领域缺乏专门的视觉质量评估方法这一现状，提出了一个全新的、面向家用视频的时空质量评估方法，并深入研究了影响整体视觉质量的几个关键时空因素以及这些因素和整体视觉质量的内在关系。与传统的基于单帧图像的质量评估方法不同的是，采用子镜头这种视频时序结构单元作为质量评估的基本单位。在子镜头上分析了一组影响视觉质量的时空因素，包括不稳定性、骤动性、失真性、模糊性、亮度性和方向性，并提出三种方法将这些时空因素融合到整体视觉质量的评估准则中，分别是基于用户研究、基于规则和基于学习的方法。为实现对视频内容的过滤，提出了一个可扩展的、基于视觉质量的家用视频摘要系统，目的是在保证视频摘要信息性的同时，保证其视觉质量最优。与基于用户注意力的视频摘要系统的对比实验表明，本文提出的基于质量的视频摘要系统对家用视频具有更好的适用性。 (2) 对于内容理解问题，从“用户拍摄意图”这一全新的角度来研究家用视频内容分析方法，并参考心理学上的意图产生机制，建立了对用户拍摄意图进行建模和挖掘的完整的理论和系统框架。首先，基于心理学上的意图机制和广泛的用户研究实验，定义了一组用户拍摄意图的概念，包括“静态场景”、“动态事件”、“特写画面”、“美丽风景”、“转换记录”、“长时间记录”和“随意记录”等。提出了一个对用更多还原

【Abstract】 With rapid adoption of consumer digital recorders as home appliances to capture people’s memorable experiences and document their daily lives, the quantity of home video data is increasing dramatically. However, home videos are usually characterized by the low visual and audio quality, as well as non-edited content. In order to alleviate the average home users’ effort to browse, retrieve and share personal videos, content analysis has become an important research issue in home videos for turning the raw data into a well-organized and easy-to-access database. Home video content analysis aims at not only providing personalized multimedia services for home users, but also enhancing the development of related issues in video content retrieval, such as semantic modeling, machine learning and information retrieval. Therefore, the research on methods for home video content analysis has both important theoretical and applied values.This thesis investigates the key problems of home video content analysis at three different levels: quality assessment, content understanding and content representation. Quality assessment is a fundamental step for filtering home video content due to the low visual quality. Content understanding enables modeling of semantic concepts at a higher level than quality. Content representation corresponds to the highest application level which can provide a compact and efficient representation of video content based on quality and understanding of content. As a result, these three levels build up a relatively comprehensive framework for home video content analysis. Accordingly, this thesis conducts a deep research on home video content analysis, and obtains the following achievements:(1) For quality assessment, we propose a novel spatio-temporal quality assessment scheme for home videos, in which a set of key spatio-temporal factors as well as the relationship betwen these factors and the overall visual quality are investigated. In contrast to existing frame-level-based quality assessment schemes, a type of temporal segment of video, sub-shot, is selected as the basic unit for quality assessment. A set of spatio-temporal visual artifacts, regarded as the key factors affecting the overall perceived quality (i.e. unstableness and jerkiness as temporal factors; infidelity, blurring, brightness and orientation as spatial factors), are mined from each sub-shot based on particular characteristics of home videos. The relationship between the overall quality metric and these factors are exploited by three different methods, including User Study-based, Rule-based and Learning-based. To filter home video content, we present a scalable quality-based home video summarization更多还原