【作者】 王雷；

【导师】 郑雅羽；

【作者基本信息】 浙江工业大学 ， 控制工程（专业学位）， 2020， 硕士

【摘要】 远程协助系统是一种能够实现远程专业人员指导现场人员操作的实时视频交互装置。现阶段,我国部分行业的专业技术人员紧缺,无法做到专业人员时刻在现场进行事件处理,造成了事件处理延后或者非技术人员操作失误等情况。针对上述问题,本文设计了一种具有音视频实时交互功能的远程协助系统。该系统的移动端以可穿戴设备为载体,解放了操作者的双手。除了传统的音视频交互外,该系统还支持协助者通过客户端软件对视频画面添加绘图信息,并同步显示给被协助者,通过多角度的协助手段减少非专业人员操作上的失误。本文主要工作内容如下:(1)对远程协助系统的整体设计。确定了系统需求以及总体的框架,进行了硬件系统的设计以及开发平台的搭建。(2)移动终端采集显示软件的设计与实现。采用STM32单片机作为数据传输中间介质,设计串口通信协议,完成IMX214图像传感器、VL53L0X距离传感器等外接设备与海思3519主控芯片的数据交互功能。在海思3519平台设计多进程任务,通过进程间通讯方式实现数据传递,完成对串口数据的处理。(3)视频直播功能的设计实现。采用H.264视频编码和AAC音频编码技术,提高了直播音视频质量并减少了网络带宽压力。进行重采样处理减少音频的杂音问题。采用时间戳同步技术对音视频数据进行同步。基于RTMP流媒体传输协议对数据帧进行打包,逐帧进行网络传输。搭建SRS流媒体服务器,将直播数据直接转发到客户端,在满足高并发性的同时降低网络延时。(4)基于QT的客户端软件设计。针对系统所需GUI,利用QT提供的窗口类及其方法,并结合信号与槽机制进行软件GUI设计。根据缓冲技术,创建缓冲区,通过临界区资源与读取缓冲区完成多线程同步,通过这种方式来提高音视频解码过程的平稳性。更多还原

【Abstract】 Remote Assistance System is a real-time video interaction device that enables remote professionals to guide field personnel.At this stage,the professional and technical personnel in some industries of China are in short supply.It is impossible for professionals to handle incidents on site at all times.This has caused delays in event processing or mishandling of non-technical personnel.Regarding the issue above,this thesis designed a remote assistance system with real-time video and audio interaction.The mobile terminal of the system uses the wearable device as a carrier to liberate the hands of the operator.In addition to traditional video and audio interactions,This system also supports the helper to add drawing information to the video screen through the client software,and simultaneously display it to the assisted person.Reduce the operational errors of non-professionals through multi-angle assistance.The main contents of this article are as follows:(1)The overall design of the remote assistance system.The system requirements and the overall framework are determined,and the hardware system design and development platform were built.(2)Design and implementation of mobile terminal acquisition and display software.Using STM32 microcontroller as the intermediate medium for data transmission.Design a serial communication protocol,complete the data interaction function between the external device such as IMX214 image sensor and VL53L0 X distance sensor and the Hisi 3519 control chip.Design multi-process tasks on the Hisi 3519 platform,realize data transmission through inter-process communication,and complete the processing of serial data.(3)Design and implementation of video live broadcast function.Using H.264 video coding and AAC audio coding technology improves the quality of live audio and video,reduces network bandwidth pressure.Resampling is performed to reduce the noise of the audio.Audio and video data is synchronized using a timestamp synchronization technique.The data frame is packaged based on the RTMP streaming media transmission protocol,and the network transmission is performed frame by frame.The SRS is built on the streaming media server to forward the live data directly to the client,which reduces the network latency while satisfying high concurrency.(4)Client software design based on QT platform.For the GUI required by the system,the window class and its method provided by QT are used,and the software GUI design is combined with the signal and slot mechanism.According to the cushion technology,Created buffer and the multi-thread synchronization is completed by the critical section resource and the read buffer,In this way,the smoothness of the audio and video decoding process is improved.更多还原