【作者】 杨刚；

【导师】 冯筠；

【作者基本信息】 西北大学 ， 计算机应用技术， 2019， 硕士

【摘要】 动作识别作为人机交互领域的一个重要研究方向,在生活中具有广泛的应用。传统基于视频图像和穿戴式传感器的动作识别技术受光线、视角、隐私以及便携性等条件限制,在一些场景中并不适用。而使用Wi-Fi信号进行动作识别可以有效地克服这些缺点。目前基于Wi-Fi的动作识别方法大多使用人工特征和传统机器学习分类算法,在复杂的手部动作识别任务上效果并不理想。因此,本文基于Wi-Fi信道状态信息(Channel State Information,CSI),针对手部动作中的手势和手语孤立词,分别提出了一种基于深度迁移学习的Wi-Fi手势识别算法和一种基于CNN和LSTM融合的Wi-Fi手语孤立词识别算法。主要内容如下:(1)为了克服现有方法提取手部动作片段不完整的问题,提出一种基于缓冲区技术的手部动作片段提取算法。首先将经过降噪处理的CSI序列划分为若干窗口,然后计算各窗口中CSI的方差均值,最后结合缓冲区技术与方差均值的阈值提取出完整的手部动作片段。(2)针对现有Wi-Fi手势识别方法准确率不高的问题,提出了一种基于深度迁移学习的Wi-Fi手势识别算法。首先将Wi-Fi手势识别问题转换为图像分类问题,然后使用迁移学习的方法实现手势识别。实验结果表明,在12类手势识别任务中取得了98%的准确率,比前沿算法提高了5.7%。(3)为了提高使用Wi-Fi实现手语孤立词识别的准确率,提出了一种基于CNN和LSTM融合的Wi-Fi手语孤立词识别算法。结合迁移学习提取的多层次图像特征和LSTM提取的时序特征构建了一种深度神经网络。实验结果表明,在100个手语孤立词的识别任务上取得了98.6%的准确率,相比于2018年的SignFi方法提高了1.6%。以上研究工作表明,本文提出的算法提高了基于Wi-Fi的手部动作识别的准确性和鲁棒性,丰富了人与智能设备之间的交互方式。更多还原

【Abstract】 As an important research direction in field of human-computer interaction,action recognition has a wide range of applications in life.Limited by conditions such as light,viewing angle,privacy,and portability,traditional action recognition techniques based on cameras and wearable sensors are not applicable in some scenarios.However,Wi-Fi based action recognition can effectively overcome these shortcomings.At present,Wi-Fi based action recognition methods using manually defined features and traditional classification algorithms are not suitable for complex hand motion recognition tasks.Therefore,based on Wi-Fi Channel State Information(CSI),this thesis proposes a gesture recognition algorithm using deep transfer learning and a isolated sign language recognition algorithm combining CNN and LSTM.The main contents include the following three points:(1)In order to overcome the problem that the existing method extracts the incomplete hand motion segment,we propose a hand motion segment extraction algorithm based on buffer technology.we firstly divide the denoised channel state information into several windows,then calculate the average of variance of the channel state information in each window.Finally,we extract the complete hand motion segment by using the buffer technology and the threshold.(2)In order to improve the accuracy of Wi-Fi based gesture recognition,a Wi-Fi based gesture recognition algorithm using deep transfer learning is proposed.we firstly convert Wi-Fi based gesture recognition into an image classification problem,and then use the transfer learning methods to complete gesture recognition.In our experiment,the algorithm achieves 98% accuracy in the dataset collected by us,which is 5.7% higher than state-ofthe-art algorithm.(3)In order to improve the accuracy of Wi-Fi based isolated sign language recognition,we propose a Wi-Fi based isolated sign language recognition algorithm combining CNN and LSTM.In this algorithm,we construct a deep neural network which combines the multilevel image features extracted by transfer learning and sequential features extracted by LSTMs.In our experiment,the accuracy of isolated sign language recognition is improved to 98.6% by this method,which is 1.6% higher than SignFi.In conclusion,our methods improve the accuracy and robustness of hand motion recognition based on Wi-Fi,and enrich the interaction between human and smart devices.更多还原