【摘要】 利用大量真实多模式行为数据进行学习训练、获取单模式行为的韵律模型以及多模式行为之间的协同关联模型的方法,来实现虚拟人多模式行为之间的协同.重点给出了多模式行为的韵律模型描述,同时给出基于手语韵律参数与语音韵律特征融合的协同控制韵律模型以及韵律参数获取方法,并运用于多模式行为协同控制中,取得了较好的实验结果.与传统的规则法相比,该学习方法更能刻画多模式之间协同关联的复杂性,更好地实现虚拟人多模式行为合成的逼真性.更多还原

【Abstract】 This paper proposes a multi-model behavior synchronizing prosody model and its application to Chinese sign language synthesis. Based on huge realistic multi-model behavior training data, the authors adopt learning the prosody mode for each single channel behavior data and further synchronizing relation model of all models, and present the framework for the multi-model synchronization in virtual human synthesis, including models of sign language, speech, facial expression and lip movement and so on. The formal description of multi-model prosody model is demonstrated in detail. Comparing to traditional regularity approaches, the learning based approach in this paper is more adequate to express complicatedly the multi-model synchronizing relationship, and to synthesize realistically the multi-model behavior of the virtual human. As the example, the synchronizing prosody model involving sign language prosody parameters and speech prosody parameters is given. The authors design an approach to compute the prosody parameters and apply it to control the virtual human’s multi-model behavior synchronously. Experiments based on the Coss ("863" speech material library) and Chinese sign language library show that the multi-model behavior synchronizing prosody model works well. It enhances the recognition rate of synthetic sign language by 5.94%.更多还原