节点文献
三维高斯溅射超分辨视觉场景构建算法
Super-resolution visual scene construction algorithm based on 3D Gaussian splatting
【摘要】 目前,机器人技术在工业化与自动化产业中扮演着举足轻重的角色,然而,机器人在视觉信息感知领域仍面临感知精度不足等挑战。为提升机器人在复杂工作场景下的视觉信息精确感知能力,该文基于三维高斯溅射(3D Gaussian splatting, 3DGS)提出一种超分辨视觉场景构建算法。该算法引入真实世界增强型超分辨率生成对抗网络(real-world enhanced super-resolution generative adversarial networks, Real-ESRGAN)作为前置预处理技术生成高分辨率视图,并通过对生成的高分辨率视图下采样结果与原有的低分辨率视图对齐得到亚像素约束,进而增加三维重建细节部分的表征精细度。在场景构建过程中,自适应密度控制保证重建的精度,高斯快速可微光栅化器保证实时渲染速率。通过在具有复杂纹理、镜面反射等特征的多种场景实验验证表明,与传统3DGS相比:峰值信噪比(PSNR)指标平均提高7.81%,结构相似性指数(SSIM)指标平均提升4.31%,学习感知图像块相似度(LPIPS)指标平均降低38.35%。该算法可显著改善传统3DGS在低分辨率输入时出现的颜色渲染错误、针状伪影以及纹理信息缺失等问题,为机器人视觉信息感知提供新的技术支撑。
【Abstract】 Currently, robotics technology plays a pivotal role in industrialization and automation industries.However, robots still face challenges in the field of visual information perception, such as insufficient perception accuracy. To enhance the precise perception capabilities of robots in complex work scenarios, this paper proposes a super-resolution visual scene construction algorithm based on 3D Gaussian splatting(3DGS).This algorithm introduces real-world enhanced super-resolution generative adversarial networks(RealESRGAN) as a pre-processing technique to generate high-resolution views. By downsampling the generated high-resolution views and aligning them with the original low-resolution views, sub-pixel constraints are obtained, which helps enhance the representational fineness of details in 3D reconstruction. During the scene construction process, adaptive density control ensures the accuracy of the reconstruction, and a Gaussian fast differentiable rasterizer guarantees real-time rendering rates. Experiments in various scenes with complex textures and mirror reflections have shown that, compared to traditional 3DGS, the peak signal-to-noise ratio(PSNR) has increased by an average of 7.81%, the structural similarity index(SSIM) has improved by an average of 4.31%, and the learned perceptual image patch similarity(LPIPS) has decreased by an average of38.35%. This algorithm significantly improves upon the color rendering errors, needle-like artifacts, and missing texture information issues that occur with traditional 3DGS at low-resolution inputs, providing new technical support for robotic visual information perception.
【Key words】 3D reconstruction; robot perception; neural radiance fields; super-resolution; 3D Gaussian splatting;
- 【文献出处】 中国测试 ,China Measurement & Test , 编辑部邮箱 ,2024年09期
- 【分类号】TP391.41;TP242
- 【下载频次】196