【作者】 李涵；

【导师】 曹汛；

【作者基本信息】 南京大学 ， 电子与通信工程（专业学位）， 2019， 硕士

【摘要】 随着超高清(UHD,Ultra-High Definition)视频、虚拟现实(VR,Virtual Reality)及增强视频(AR,Augmented Reality)等业务的蓬勃发展,视频观看正逐步从高清向超高清演进。十亿像素视频在成像分辨率上突破了人眼的视觉极限,不仅能够实现全局场景的无遗漏捕捉,还能保证局部细节的高清展示,做到大视场与局部细节兼顾。然而分辨率提升带来的带宽需求越来越大,加之客户端的解码能力受限,使得十亿像素视频的直播面临着巨大挑战,本文围绕亿级像素的交互式直播系统,主要内容及创新点包括:1.亿级像素视频的采集和制作。针对单个相机图像传感器成像分辨率与信噪比之间存在天然的矛盾,而现有亿级像素传感器存在无法适配高帧率的问题,本文提出采用阵列相机采集加拼接的形式,在提升分辨率的同时保证成像质量。为保证多相机采集图像的高效实时拼接,系统采用并行计算处理的方式对视频流进行参数设定,包括分辨率、码率、白平衡以及曝光等,并在客户端通过多路局部高清图像与单路低清广角图像之间的位置映射生成全景,以降低整个系统延迟,达到亿级像素直播实时性的强要求。2.视频推流及转发。实时传输亿级像素视频的海量数据需要网络高带宽的支持,给视频传输带来严峻挑战。为此,本系统在传输海量视频数据前对其进行压缩编码,并搭建了中央控制平台对各路相机进行实时流预览及远程调焦,并通过中央控制平台统一收集各路相机视频数据,实现了视频源数据的统一推流与转发。3.多尺度分层分块视频处理。为进一步降低亿级像素视频传输对网络带宽的需求,本系统基于人眼视觉特性,采用人眼视场区域内容高质量传输,视场区域外低质量传输的方式,从而兼顾大视场与局部高清细节视频。具体地,系统采集到的19路4K高清视频流进行了 3个分辨率层级的下采样,对于每一个4K分辨率原始视频,采用H264方式重新编码为1 6个960×540的视频切片,客户端在加载时,只需传输所需分辨率层级的视频切片数据,实现20Mbps即可观看亿级像素高清视频,从而降低了网络带宽需求。4.亿级像素视频的交互显示。为了满足不同客户端的视频观看需求,本系统通过客户端平台解码视频服务器处理后的多尺度编码视频,支持视频切换功能及缩放功能。本文采用视场外数据预取的动作,额外传输周围区域的数据进行缓存,保证视频在拖动时的画面一致性。目前,用户可随时随地通过客户端(PC/Mobile)观看直播状态,系统从采集端至客户端整体延迟在500ms内,满足直播需求。本文提出的亿级像素交互式场景视频直播系统未来可进一步应用在体育、会议等多种场景,引领新型直播业务形态。更多还原

【Abstract】 With the rapid development of UHD,VR and AR,video viewing is undergoing an evolution from high definition(HD)display to ultra high definition(UHD)display.Gigapixel video breaks through the visual limit of human eyes in imaging resolution.It can capture the whole scene and the local details,so both large field of view and local details can be ensured.But with the limitation of network bandwidth and client decoding ability,the gigapixel broadcast is facing great challenges.This paper focuses on how to realize gigapixel video broadcasting.The main contents and innovations include:1.Collect and produce gigapixel panoramic video.Aiming at the natural contradiction between the imaging resolution and signal-to-noise ratio of a single camera image sensor,and the problem that the existing billion-level pixel sensor cannot adapt to the high framerate.This paper proposes a method of acquisition and splicing based on parallel cameras to improve the resolution and ensure the image quality.In order to ensure the efficient real-time mosaic of multi-camera images,the system uses parallel computing to set the parameters of video stream,such as resolution,bitrate,white balance and exposure,and generates panorama by mapping the position between local high-definition images and a low-definition wide-angle images on the client side.So,the whole delay of the system is reduced and the system can achieve real-time gigapixel broadcasting.2.Transmission and forwarding of video streams.Real-time transmission of gigapixel video data requires the support of high bandwidth network,which brings serious challenges to video transmission.For this reason,the system compresses and encodes the massive video data before transmitting it,and builds a central control platform to preview and focus the real-time streams of all cameras.This platform collects the video data of all cameras,thus realizing the unified push and forward of video streams.3.Multiscale hierarchical block video processing.In order to further reduce the demand for network bandwidth for gigapixel video transmission,based on the characteristics of human vision,the system adopts the high-quality transmission of visual field area content and low-quality transmission outside the field of view,thus can achieve both large field of view and local high-definition detail video.Specifically,the 19 4K videos collected by system are sampled at 3 resolution levels.For each 4K resolution original video,the system re-encodes 16@960x540 slices by H264.When the client loads,it only needs to transmit the required resolution level video slices.So the client just needs 20Mbps to watch the gigapixel video,reducing the network bandwidth requirements.4.The interactive display of gigapixel video.In order to meet the video viewing needs of different clients,the system decodes multiscale video stream and supports video switching and zooming.To ensure the consistency of scene when the video is dragged,this paper caches the data outside of the view in advance.For now,users can open the client to watch live video at anytime and anywhere both in PC and mobile.The overall delay of the system from the acquisition terminal to the client is less than 500 ms,which meets the demand of live broadcasting.The interactive gigapixel broadcasting proposed by this paper would be used in sports,meeting and so on in the future,enriching the form of live broadcasting and leads the UHD broadcasting in the era of big video.更多还原