【作者】 陈波；

【导师】 田晓华； 王建民；

【作者基本信息】 哈尔滨理工大学 ， 电子科学与技术， 2020， 硕士

【摘要】 计算机视觉是一门利用硬件和算法让计算机从图片中或视频中读取信息的科学。计算机视觉包含图像分类技术、对象检测技术、目标跟踪技术、语义分割技术和实例分割技术等多种技术。而其中图像分类、对象检测等技术发展较为成熟,已被广泛应用。在教育领域,由于学生基数大,教师资源相对较少,导致教师无法实时地、完全地掌握学生的上课状态,教师与家长无法更有针对性地帮助学生。解决这一问题,可以有效提高学生的听课效率,帮助学生养成良好的学习习惯。针对该问题,本文提出基于深度学习的学生签到及上课状态检测系统,通过使用人脸检测、人脸识别、表情检测和疲劳检测四种深度学习图像分类和对象检测算法来判断学生的听课状态,并将检测识别的结果存储并显示在上位机上。搭配高计算能力的硬件平台,使该系统对视频的处理、分析能力满足实时性要求。首先将系统分为数据采集子系统、检测子系统和存储及可视化子系统,并据其功能提出了各个子系统的设计要求,描述了课题中发现的人脸识别算法对亚洲人脸识别能力差和表情识别算法的人脸检测能力差两种主要挑战。然后给出了系统软件和硬件设计的方案,搭建了硬件平台。之后给出了系统软件的具体设计方法:首先介绍了数据采集子系统的数据通路搭建,然后在检测子系统中完成了对输入图像的转码,设计了子系统内和子系统间的数据格式标准。之后介绍了所使用的人脸检测、人脸识别、表情识别和疲劳检测算法的结构与原理。针对高清图片人脸检测准确率低的问题,修改了多项参数,大幅度提高了网络准确度;针对亚洲人脸识别准确率低的问题,使用亚洲人脸库进行了调优训练;针对表情识别算法在噪声较大的图片中识别不到人脸的问题,修改了其中的人脸检测算法,提高了算法检测的准确率;针对疲劳检测算法中限制输入图片大小的人脸检测算法,改进了人脸检测算法,不仅提高了整个算法的速度,还让疲劳检测网络可以识别高清图片中的人脸。其中人脸检测算法准确率达到了97.52%,人脸识别算法准确率达到了92.20%,表情识别算法准确率为61.30%,且部分算法速度提高,子系统总参数减少。最后给出了存储及可视化子系统中进行学生签到和上课状态判定的软件设计,软件带有可视化用户界面。测试与验证结果表明,系统能够很好地满足图像采集、检测识别、结果判断、存储和上位机显示的设计要求,且该系统的检测识别在服务器上每秒可以处理5张图片,嵌入式边缘计算平台上每秒可以处理1张图片,达到了课题要求的速度效果。更多还原

【Abstract】 Computer vision is a technology that uses hardware and algorithms to help computers to read messages from pictures or videos.Computer vision includes a variety of techniques such as image classification,object detection,target tracking,semantic segmentation,and instance segmentation.Among these techniques,image classification,object detection have been relatively mature,and are widely.In the territory of education,the number of students is large and teachers are relatively small.As a result,teachers cannot grasp the students’ state of class in real time completely.Teachers and parents cannot help students more specifically based on their states.Solving this problem can effectively improve the efficiency of students’ lectures and help them develop good study habits.In response to this problem,this thesis proposes a deep learning-based student sign-in and class state detection system,which uses four deep learning image classification and object detection algorithms to detect the state of the student’s class,including face detection,face recognition,expression detection and fatigue detection.The results of the detection and recognition are stored and intuitively displayed on the host computer.With a platform which has high computation ability,the system’s ability to process and analyze video can meet real-time requirements.Firstly,this thesis divide the system into a data acquisition subsystem,a detection subsystem and a storage and visualization subsystem,and the design requirements of each subsystem are proposed according to its functions.The main challenge of the poor expression on face recognition of Asia and expression recognition is also proposed.Then the system software and hardware design plan is given,and hardware platform is built.After that,this thesis introduces the specific design method of the system software.In this section,the thesis introduces the construction of data path is introduced in data acquisition subsystem.Then the input image is transcoded in the detection subsystem,and the data format standards within and between subsystems are designed.Then the thesis introduces the structure and principle of the face detection,face recognition,expression recognition and fatigue detection algorithms used.In response to the low accuracy of face detection in high-definition pictures,several parameters have been modified to greatly improve the accuracy of the network;for the problem of low accuracy of Asian face recognition,the Asian face database has been used for fine-tuning the network;for the low detection rate of expression recognition algorithm,the face detection algorithm has been modified to improve the accuracy;this is also happened to fatigue detection algorithm,and the similar measure is taken.The accuracy rate of the face detection algorithm reached 97.52%,the face recognition algorithm is 92.20% and the expression recognition algorithm is 61.30%.The speed of some algorithms is improved,and the total parameters of the subsystem are reduced.Finally,the software design for student sign-in and class state determination in the storage and visualization subsystem is given.The software has a convenient user interface.The test and verification results show that the system can well meet the design requirements of image acquisition,detection and identification,result judgment,storage,and host computer display.The system can process 5 pictures per second which can process 1 picture per second on embedded edges computing platform The system can achieve the real-time requirement which is set by the subject.更多还原