【作者】 李锐；

【导师】 李辉；

【作者基本信息】 中国科学技术大学 ， 电路与系统， 2016， 硕士

【摘要】 高速发展的计算机水平和音频处理技术,使得人们对于从海量数据中获取到感兴趣的人声,越来越有需求。另一方面,如何对获取到的各类音频进行合理有效的管理,也是目前存在的一大挑战。在此背景下,为了满足上述需要,说话人分离这一关键技术应运而生,其主要涉及两个过程：说话人分割和说话人聚类。说话人分离系统几乎无任何可供参考的先验信息使用,容易受环境或者建模方法的影响,使得分割和聚类时说话人片段的类纯度得不到保证。另外,基于距离准则的层次聚类方式一旦出现聚类误差,会一直的向上传递。因此本文主要在说话人分割和聚类的建模方法以及类别提纯上展开了探索和研究,主要工作和创新点如下：第一,说话人分离前端语音端点检测(Voice Activity Detection, VAD)和类别提纯方面的研究。针对基线系统中存在的低能量语音难以召回以及噪声难以去除的情形,引入了深度学习的方法,在分离前端进行改进。针对层次聚类时出现聚类误差向上传递的情况,提出了基于贝叶斯信息准则(Bayesian Information Criterion, BIC)的短时类别提纯方法,削弱由层次聚类带来的误差向上传递的影响。实验结果表明,基于深度学习的语音端点检测,能有效的降低说话人分离时的虚警和漏警,并且降低说话人分离错误率,且基于短时BIC类别提纯的方法,能更新部分聚类错误的说话人片段,提高后续说话人聚类的类纯度。第二,说话人转折点检测建模方法的研宄。探索了基于深度神经网络(Deep Neural Network, DNN)的建模方法在说话人转折点检测中的应用,利用其强大的模型表达能力,提高说话人分割的准确性。实验结果表明,这种基于深度学习的转折点检测建模方法相比于传统的BIC建模方法,无论是在转折点检测的准确率和召回率,还是对整个系统的分离错误率(Diarization Error Rate, DER)来说,都获得了一定的效果提升。第三,说话人聚类时的因子分析建模方法研究。传统的以贝叶斯信息准则作为相似性度量的说话人分离技术,在短时对话的分离任务中能取得较好的效果,但是随着对话时长的增加,BIC的单高斯模型不足以描述不同说话人数据的分布,且层次聚类时区分相同说话人和不同说话人的门限值难以划定。针对此问题,本文尝试基于短时BIC和长时概率线性判别分析(Probabilistic LinearDiscriminant Analysis, PLDA)融合的方法,充分利用BIC在短时聚类的可靠性和PLDA在长时段上的优异区分性,来实现说话人聚类过程。实验表明,因子分析框架下的说话人信息建模方法能有效的降低说话人分离错误率,使得性能相对提升34.2%。第四,说话人聚类过程中的变分贝叶斯(Variational Bayesian, VB)调优方法研究。通过与因子分析框架下的全差异空间相结合,将传统的层次聚类方式转化为在保证最优化目标函数的前提下,对某个片段属于某个说话人的最大后验概率估计的一种软聚类方式。依据此方法,将层次聚类时因误差上传而出现错误的说话人片段进行重新调优,达到提高后续PLDA聚类的初始类纯度,进而降低说话人分离错误率。更多还原

【Abstract】 With the rapid development of computer level and the audio processing technology, there is a growing demand for people to get specific voice of interest from huge amounts of audio data. On the other hand, how to reasonable and effective make various types of audio document manage effectively, is also a major challenge. In this context, in order to meet these requirements, the speaker diarization as a critical technology comes into being, which mainly involves two processes:the speaker segmentation and the speaker clustering.There almost without any priori information to use as reference for speaker diarization system, and it is easily influenced by environment or modeling method, therefore the speaker’s class purity will not be guaranteed through clustering segments. In addition, the hierarchical aggressive clustering method based on distance criterion makes clustering error pass up. So this dissertation presents our works and research on speaker segmentation, clustering modeling method and category purification, the main work and innovations of the dissertation are as follows:Firstly, the research of the front-end voice activity detection (VAD) and class purification. According to the situations of low-energy speech and noise speech in baseline systems, we introduce the method of deep learning and improve the voice activity detection in the front-end of speaker diarization system. Also, the traditional method as Hierarchical Agglomerative Clustering (HAC), can lead to the clustering error layer transfer. To deal with this problem, category purification method based on short-time Bayesian Information Criterion (BIC) is presented, which can weaken the influence of the clustering error of the upward transmission. The experimental results show that, the voice activity detection based on deep learning, can effectively reduce the false alarm and miss alarm when deal with the speaker diarization problem, as a result, it also reduces the speaker diarizaition error rate; Meanwhile, clustering methods based on short-time BIC category purification can correct some wrong speaker clustering segments, which improve purity of back-end speaker clustering process.Secondly, the research of the modeling method of speaker change point detection. In order to improve the accuracy of the division of the speaker, we explored the application of speaker change point detection within Deep Neural Network (DNN) based modeling method, and using its powerful model expression ability. The experimental results show that the modeling method of speaker change point detection using deep learning has played an important role in promotion effect when comparing with the traditional modeling approach using BIC, whether in the detection accuracy and recall rate of the change point, or the speaker error rate of the entire separation system.Thirdly, the research of the speaker clustering modeling method based on factor analysis. When we use traditional Bayesian Information Criterion as the similarity measure for speaker diarization, it can obtain good result in a short dialogue task, however, with the increasing of the length of the conversation, single Gaussian model of BIC is hard to describe the distribution of different speaker data. What’s more, it is difficult to delineate the threshold when distinguish the same speakers and different speakers in the process of hierarchical aggressive clustering. Aiming at this problem, this paper attempts to explore a fusion method based on short-time BIC and long-term Probabilistic Linear Discriminant Analysis (PLDA), which make full use of the BIC reliability in short-term clustering and the excellent discriminating power of PLDA in long-term clustering. The experimental results show that under the framework of factor analysis, the speaker information modeling method can effectively reduce the speaker Diarization Error Rate (DER), and the performances is relatively improved 34.2%.Fourthly, the research of the speaker clustering optimization based on Variational Bayesian (VB) method. In combination with the total variability space under factor analysis framework, we convert the traditional hierarchical clustering method to a soft clustering method which maximum a posteriori probability for a segment belong to a speaker and guarantee the optimization of objective function. This variational Bayesian method can correct some wrong speaker segments when using hierarchical aggressive clustering, improve the back-end initial class purity of PLDA clustering and reduce the speaker separation error rate.更多还原