【作者】 曹建荣；

【导师】 蔡安妮； 孙景鳌；

【作者基本信息】 北京邮电大学 ， 通信与信息系统， 2007， 博士

【摘要】 随着通信技术、宽带网络技术、音视频压缩技术以及计算机技术的发展，数字视频也越来越广泛地融入于人们的生活空间中。由于视频数据自身内容的丰富性和多样性、结构的复杂性以及具有时空多维结构性，如何有效地对这些视频数据进行组织、表达、存储和管理，以及如何对其进行快速检索与浏览等已成为视频领域内急待解决的重大课题。传统的数据管理与检索方案不能够很好地从巨大的视频数据源中找到所需要的信息，于是基于内容的视频检索系统便应运而生。本文主要针对风光纪录片这类视频，根据它们的视频的拍摄的风格和特点，对其基于内容的视频检索关键技术进行了研究，提出了一些新的算法，主要内容如下：在视频镜头分割方面，提出了一个基于支持向量机SVM(Support Vector Machine)的镜头边界检测算法。该算法利用视频压缩域中特征宏块类型、帧间对应宏块DC系数差和帧类型特征将视频帧分为发生切变的帧、发生渐变的帧和非镜头变换帧三类，从而实现视频的镜头分割。实验结果表明该算法对摄像机的运动和大物体的进入具有很好的鲁棒性，且没有大多数算法中阈值选择的困难，与经典的双阈值方法、模糊k均值聚类、k均值聚类和基于DCT系数以及宏块类型的镜头边界分割方法相比，边界检测精度明显高于这些方法，具有较高的镜头边界检测的查准率和查全率。另外，根据风光记录片的特点，还提出了一个抽取镜头关键帧的方法。在镜头关键帧语义提取方面，利用支持向量机通过视频镜头关键帧的彩色直方图和MPEG-7的边缘直方图特征，对镜头关键帧进行分类，从而得到了镜头关键帧的语义含义，实现了将关键帧图像的低层特征(颜色，边缘纹理)与高层语义特征的连接。对具有不同核函数的支持向量机的分类准确度进行了对比，结果显示二项式和RBF核函数的支持向量机具有较高的分类准确度。在视频摘要方面，提出一个根据高层语义概念特征对风光记录片形成视频摘要的算法。该算法利用支持向量机SVM对镜头关键帧分类，得到视频中镜头的语义概念，将同样语义概念的镜头归为一类。根据引入的“重要性”函数计算每个镜头的“重要性”函数值，然后由不同语义类中镜头的“重要性”来决定从该类中选取哪个镜头的关键帧作为视频摘要的构成帧。改变镜头重要性阈值的大小可以得到不同粒度的视频摘要。最后，将不同语义类中选取的关键帧根据时间顺序排序，从而构成静态浏览型的视频摘要。实验结果表明，得到的视频摘要概括了视频的主要内容，基本上满足了一个较好的视频摘要的要求。在视频场景分割和场景关键帧提取方面，首先提出的一个计算镜头颜色和纹理直方图的方法，即在压缩域中，根据镜头中所有Ⅰ帧的DC图求取镜头的颜色和纹理直方图。通过比较镜头间颜色和纹理直方图的相似度将相似的镜头聚类，从而得到场景，并且在聚类时自动计算阈值的大小。这样可以避免以镜头关键帧按相似度进行场景聚类或分割，且聚类和分割中的阈值大多根据经验设定时，由于镜头关键帧提取的不好以及阈值设定不恰当对场景聚类或分割性能的影响。实验表明，我们的分割算法是有效的，结果令人鼓舞。针对如何在镜头基础上进行聚类得到场景的问题，又提出了一个基于语义的场景分割算法。首先利用支持向量机SVM根据镜头关键帧的颜色和纹理特征提取了镜头关键帧的语义，并形成了关键帧的语义概念矢量，然后根据语义概念矢量对镜头关键帧聚类得到场景。为提取场景关键帧，构建了镜头选择函数，并根据该函数值的大小选择场景的关键帧。实验结果表明我们的场景分割算法较A．Hanjalic的方法有更高的查准率和查全率。在视频数据模型方面，提出了一个4层结构的语义视频数据模型，语义的提取通过SVM构成的分类器将关键帧分类自动得到。通过语义视频数据模型，可以在镜头或场景的底层特征层和高层语义层进行检索，不同层次的检索结果可以组合起来实现更加精确地检索。在视频检索算法方面，提出了一个基于相关反馈的视频检索算法，它利用信号在神经网络中来回传递来实现自动相关反馈，提高了检索精度，且最大限度地减少了相关反馈中人的参与。将该算法与基于关键帧视频检索(KFVR)方法进行了比较，结果表明该算法检索的准确度比用KFVR方法平均高6％以上。另外，还提出了一个压缩域中基于镜头的视频检索方法，通过提取镜头中Ⅰ帧DC图的颜色和纹理直方图，形成镜头的可变阿尔法颜色和纹理直方图，然后用镜头的颜色和纹理直方图作为特征进行视频检索。对不同度量距离的检索结果进行了分析，并与基于关键帧的检索方法进行了比较。实验表明，基于镜头的检索用L1度量距离比用L2和x2度量距离有更好的检索性能，而且该方法不用考虑镜头关键帧选择的问题，避免了基于关键帧检索中由于镜头运动等原因可能导致镜头关键帧选择不当对检索性能的影响，因而比基于关键帧检索具有更好的性能。更多还原

【Abstract】 With the advance of technology of computer, telecommunication, wide-band network and compression of video and audio, digital video is widely coming into the life of people. It is becoming the emergent and important issues for efficient organization, representation, store and management of video data as well as rapid retrieval and browsing of video data. The traditional data management and retrieval method cannot do better to search the requiring information from the huge amount of video data because of the diversity, complexity of structure of video data as well as temporospatial and multi-dimensional structure of video data. As a result the content-based video retrieval system emerges, as the times require.The key technologies of content-based video retrieval and indexing for scenery documentary are researched in this paper and some novel algorithms are presented. The main contents of this paper are as following:This paper presents an algorithm for shot boundary detection based on SVM (support vector machine) in compressed domain. It uses the features, such as the type of macroblock, the difference between DC coefficients of two co-located blocks in successive frames and the type of frame, to segment a video into the shots by classifying the frames into three classes, namely, the frames of cut change, gradual change and non-change. No thresholds, which are often hard to select in most shot detection methods, are involved in our algorithm. The results of experiment have shown that this algorithm is robust for motion of camera and large objects in video, and has better precision of shot boundary detection compared with the classic double-threshold method, fuzzy k-mean cluster, k-mean cluster and the boundary detecion method based the types of macro-block and DCT coefficient.A multi-class support vector machine (SVM) is used to classify the video key frames of scenery documentary. The color histogram features and MPEG-7 edge histogram features from the video key frames of scenery documentary are combined to classify the key frame in SVM in order to obtain the semantics of the key frames. The performances of SVMs with difference kernel functions have been tested and compared.Constructing the video summarization in semantic level is very important. An algorithm of video summarization based on support vector machine (SVM) in semantic level is presented for the natural scenery documentary. At first, the shot keyframes are classified by SVM. Then the frames constructing the video summarization are selected from the shot keyframes of every class by the importance function we introduce. The scalable video summarization from coarse to fine level of detail can be achieved by changing the threshold of important function. The experiment results indicate that the proposed algorithm performs satisfactorily.Most algorithms of scene segmentation are based on the key frames of shots. This paper presents an algorithm of scene segmentation based on shots directly. First, a method is introduced to extract the features of histogram of color and texture of shot in compressed domain and then these features are used to cluster the shots by shot similarity to construct the scene in the video clips. An adaptive threshold is computed for different video contents and types. Experiments are presented with promising results on several scenery documentary and story movies.The scene is a high-level temporal video segment. This paper presents a method of scene segmentation based on semantic. At first, the semantics of shot key frames are extracted based on the color and the texture of key frames using the SVM (support vector machine). The semantic concept vectors of shot key frames are formed. The shot key frames are clustered by the semantic concept vectors to construct the video scene. The shot select function is defined to extract the scene key frame based on the value of shot select function. The experimental results are shown that the recall and the precision of our algorithm are better than those of A. Hanjalic’s method.Video data modeling is an important issue for content-based retrieval. In this paper, we propose a semantic-based four-layer video data model for scenery documentary and our discussions focus on the extracting and representation of semantics of video data. The support vector machine (SVM) is used to bridge the gap between low-level visual features and high-level semantic concepts. Semantic concept vectors are defined to represent the semantics of shot and scene. The retrieval based on our video data model can be achieved at low-level feature layer and high-level semantic concept layer. The results of retrieval at difference layers can be integrated to improve the performance of retrieval.An algorithm of video retrieval based on relevance feedback is presented. It implements the automatic relevance feedback using the neural network and minimizes user interactions. The architecture of the neural network and the strategies of the retrieval have been instructed. The results of experiment have been given and compared with key frame-based video retrieval (KFVR). The precision of our algorithm is higher than KFVR about 6%.Shot is the basic unit of a video. This paper presents a method of computation of texture histogram of a shot and video retrieval based on shot in compressed domain. At first, the histograms of color and texture are extracted in all DC pictures of I frames in a shot, and then the alpha-trimmed average histograms (ATAH) of color and texture are computed. The ATAHs are used in Video retrieval with difference distance measures at shot level. The results of experiment show that the retrievals based on shot have good performance in distance measures of L1 and x~2 and the performance of retrieval based on shot is better than that based on key frame because the camera motion may result in inappropriate selection of key frame and influence the precision of retrieval of key frame.更多还原