【摘要】 针对点云神经辐射场在重建纹理复杂场景时容易产生伪影、边缘缺陷或者过度模糊的问题，文中提出一种基于Transformer的点云神经辐射场三维重建方法。引入Transformer架构对特征图进行位置编码，赋予网络全局感受野，聚合多视图上下文信息，得到高精度的神经点云位置坐标和置信度，减少重建场景中空洞缺陷部分的出现；设计神经点云特征生成网络，添加自注意力机制，增强神经点云的特征表达能力；通过体渲染网络聚合场景表面相邻神经点后，得到更准确的采样点辐射亮度和体密度，提高重建视图的清晰度，降低伪影效果。实验结果表明，基于Transformer的点云神经辐射场在NeRF Synthetic数据集上，峰值信噪比、结构相似度、学习感知图像块相似度的平均值相较于原始点云神经辐射场分别提升了2.37%、0.83%、10.2%，重建视图的细节特征更加丰富，视觉效果良好。更多还原

【Abstract】 Artifacts, edge defects or excessive blurring will occur to the point-based neural radiance fields when reconstructing scenes with complex textures, so a Transformer-based 3D reconstruction method for point-based neural radiance fields is proposed. The Transformer architecture is introduced for positional encoding of feature maps, endowing the network with a global receptive field and aggregating multi-view contextual information to obtain high-precision neural point cloud position coordinates and confidence, thereby reducing the occurrence of holes and defects in the reconstructed scenes. A neural point cloud feature generation network is designed, incorporating a self-attention mechanism to enhance the feature representation capability of the neural point cloud. More accurate sampled point radiance and volume density are obtained by means of aggregating adjacent neural points on the scene surface with a volume rendering network, which improves the sharpness of the reconstructed views and reduces artifacts. Experimental results show that the Transformer-based point-based neural radiance field achieves average improvements of 2.37%, 0.83%, and 10.2% in peak signal-to-noise ratio(PSNR), structural similarity index(SSIM), and learned perceptual image patch similarity(LPIPS), respectively, on the NeRF Synthetic dataset in comparison with the original point-based neural radiance field. The reconstructed views exhibit richer detailed features and better visual effects.更多还原