【摘要】 噪声环境下视听语音不易被识别，为提升语音识别效果，提出改进粒子滤波跟踪的视听双模态语音识别方法。采用谱减法去除噪声数据，完成视听双模态语音的消噪处理；根据人语和唇动信息之间的相关性，采用改进粒子滤波跟踪方法提取视听双模态语音特征信息，构建transformer语音识别模型，将提取的特征信息输入到模型内实施并行训练，实现视听双模态语音的有效识别。实验结果表明，通过对上述方法开展信噪比测试、识别性能测试，验证了上述方法的可行性高、可靠性强。更多还原

【Abstract】 In noisy environments, audio-visual speech is not easily recognized. To improve speech recognition performance, an improved particle filter tracking audio-visual bimodal speech recognition method is proposed. Firstly, spectral subtraction was adopted to remove noise data, thus completing the noising removal of audiovisual dual-modal speech. Based on the correlation between human speech and lip movement information, an improved particle filter tracking method was adopted to extract audiovisual dual-modal speech feature information, and then a transformer speech recognition model was constructed. Finally, the extracted information was input into the model for parallel training, thus achieving the effective recognition for audiovisual dual-modal speech. The experimental results show that the proposed method show high feasibility and strong reliability after the signal-to-noise ratio test and recognition performance test.更多还原