节点文献
一种基于知识蒸馏和注意力损失的时间增量学习系统
A session-incremental learning system based on knowledge distillation and attention loss
【摘要】 目的 基于深度学习的运动想像脑机接口技术在智能康复领域具有很好的发展前景。然而,运动想像脑电信号(motor imagery-electroencephalogram, MI-EEG)是一种非平稳信号,其数据分布和特征空间会随着康复进程的推进而发生变化,这会导致卷积神经网络(convolutional neural network, CNN)模型的识别能力下降。为改善运动想像解码模型对时间的自适应性,本文提出一种基于知识蒸馏和注意力损失的时间增量学习系统(session-incremental learning system, SILS)。方法 首先,对运动想象脑电信号进行带通滤波和下采样等预处理,以增强与运动想象相关的信息;其次,设计一种多分支双注意力多模块卷积神经网络对多导联MI-EEG进行多尺度时间特征和空间特征的提取与整合,并基于注意力机制增强通道和空间维度的关键信息;然后,利用知识蒸馏技术及注意力损失改善增量时期解码模型持续学习新知识和保留旧知识的能力;进一步,基于最近邻法挑选少量优质旧样本进行数据回放,提升增量模型的抗遗忘性能力;最后,基于BCI Competition IV Dataset 2b公开数据集进行大量实验研究,通过可塑性和稳定性两项指标验证SILS性能。结果 SILS对第1~5阶段的数据依次取得79.21%、79.05%、89.06%、88.38%和88.47%的平均准确率,第5阶段的SILS对第1~4时段(session)数据的平均遗忘率分别为9.72%、9.10%、9.88%和6.04%。结论 SILS具有自动调整模型参数,保持持续学习和自我更新的能力,表现出很好的时间适应性和性能稳定性。
【Abstract】 Objective The brain-computer interface technology based on deep learning for motor imagery has a good development prospect in the field of intelligent rehabilitation. However, the motor imagery electroencephalogram(MI-EEG) signal is a non-stationary signal, its data distribution and feature space will change with the advancement of the rehabilitation process, which will cause the recognition ability of the convolutional neural network(CNN) model to decline. To enhance the temporal adaptability of the motor imagery(MI) decoding model, this paper proposes a session-incremental learning system(SILS) based on knowledge distillation and attention loss. Methods First, we performed band-pass filtering and down-sampling on the motor imagery EEG signals to enhance the information related to motor imagery. Next, a multi-branch, dual-attention, multi-module convolutional neural network was developed for extracting and integrating multi-scale temporal and spatial features from multi-lead MI-EEG data, utilizing an attention mechanism to amplify crucial channel and spatial information. Then, the ability of the incremental stage decoding model to continuously learn new knowledge and retain old knowledge was improved by using knowledge distillation technology and attention loss. Further, a small number of high-quality old samples were selected for data replay based on the nearest neighbor method to enhance the anti-forgetting performance of the incremental model. Finally, extensive experimental research was conducted by using the publicly available BCI Competition IV Dataset 2b, and the performance of SILS was verified through two indicators, plasticity and stability. Results SILS achieved average accuracies of 79.21%,79.05%,89.06%,88.38%,and 88.47% for stages 1 to 5,respectively, and the average forgetting rates of SILS for sessions 1 to 4 data in stage 5 were 9.72%,9.10%,9.88%,and 6.04%,respectively. Conclusions SILS has the ability to automatically adjust model parameters, maintain continuous learning and self-update, showing good temporal adaptability and performance stability.
【Key words】 electroencephalography; motor imagery; incremental learning; knowledge distillation; attention mechanism;
- 【文献出处】 北京生物医学工程 ,Beijing Biomedical Engineering , 编辑部邮箱 ,2024年06期
- 【分类号】TN911.7;R318
- 【下载频次】71