节点文献

三维高斯溅射超分辨视觉场景构建算法

Super-resolution visual scene construction algorithm based on 3D Gaussian splatting

  • 推荐 CAJ下载
  • PDF下载
  • 不支持迅雷等下载工具,请取消加速工具后下载。

【作者】 侯礼杰沈寅松刘晓晨陈雪梅贾瑞才范广伟申冲

【Author】 HOU Lijie;SHEN Yinsong;LIU Xiaochen;CHEN Xuemei;JIA Ruicai;FAN Guangwei;SHEN Chong;School of Instrument and Electronics, North University of China;School of Mechanical and Electrical Engineering, North University of China;Institute of Advanced Technology,Beijing Institute of Technology;School of Information and Electronic Engineering, Zhejiang University of Science and Technology;The 54th Research Institute of CETC;

【通讯作者】 申冲;

【机构】 中北大学仪器与电子学院中北大学机电工程学院北京理工大学前沿技术研究院浙江科技大学信息与电子工程学院中国电子科技集团公司第五十四研究所

【摘要】 目前,机器人技术在工业化与自动化产业中扮演着举足轻重的角色,然而,机器人在视觉信息感知领域仍面临感知精度不足等挑战。为提升机器人在复杂工作场景下的视觉信息精确感知能力,该文基于三维高斯溅射(3D Gaussian splatting, 3DGS)提出一种超分辨视觉场景构建算法。该算法引入真实世界增强型超分辨率生成对抗网络(real-world enhanced super-resolution generative adversarial networks, Real-ESRGAN)作为前置预处理技术生成高分辨率视图,并通过对生成的高分辨率视图下采样结果与原有的低分辨率视图对齐得到亚像素约束,进而增加三维重建细节部分的表征精细度。在场景构建过程中,自适应密度控制保证重建的精度,高斯快速可微光栅化器保证实时渲染速率。通过在具有复杂纹理、镜面反射等特征的多种场景实验验证表明,与传统3DGS相比:峰值信噪比(PSNR)指标平均提高7.81%,结构相似性指数(SSIM)指标平均提升4.31%,学习感知图像块相似度(LPIPS)指标平均降低38.35%。该算法可显著改善传统3DGS在低分辨率输入时出现的颜色渲染错误、针状伪影以及纹理信息缺失等问题,为机器人视觉信息感知提供新的技术支撑。

【Abstract】 Currently, robotics technology plays a pivotal role in industrialization and automation industries.However, robots still face challenges in the field of visual information perception, such as insufficient perception accuracy. To enhance the precise perception capabilities of robots in complex work scenarios, this paper proposes a super-resolution visual scene construction algorithm based on 3D Gaussian splatting(3DGS).This algorithm introduces real-world enhanced super-resolution generative adversarial networks(RealESRGAN) as a pre-processing technique to generate high-resolution views. By downsampling the generated high-resolution views and aligning them with the original low-resolution views, sub-pixel constraints are obtained, which helps enhance the representational fineness of details in 3D reconstruction. During the scene construction process, adaptive density control ensures the accuracy of the reconstruction, and a Gaussian fast differentiable rasterizer guarantees real-time rendering rates. Experiments in various scenes with complex textures and mirror reflections have shown that, compared to traditional 3DGS, the peak signal-to-noise ratio(PSNR) has increased by an average of 7.81%, the structural similarity index(SSIM) has improved by an average of 4.31%, and the learned perceptual image patch similarity(LPIPS) has decreased by an average of38.35%. This algorithm significantly improves upon the color rendering errors, needle-like artifacts, and missing texture information issues that occur with traditional 3DGS at low-resolution inputs, providing new technical support for robotic visual information perception.

【基金】 国家自然科学基金创新研究群体项目(51821003);山西省优秀青年基金(202103021222011);山西省重点研发计划(202202020101002);山西省基础研究计划(202303021211150);航空科学基金(2022Z0220U0002);山西省量子传感与精密测量重点实验室项目(201905D121001)
  • 【文献出处】 中国测试 ,China Measurement & Test , 编辑部邮箱 ,2024年09期
  • 【分类号】TP391.41;TP242
  • 【下载频次】196
节点文献中: 

本文链接的文献网络图示:

本文的引文网络