【作者】 韩冰；

【导师】 任爽；

【作者基本信息】 北京交通大学 ， 计算机科学与技术， 2024， 博士

【摘要】 随着激光雷达等三维扫描设备的普及和数据采集技术的发展,计算机视觉领域的研究逐渐从以图像和视频为代表的二维平面数据扩展到了三维空间数据。在这其中,点云是最灵活的且最常用的,完整保留了三维物体最原始的形状和几何信息。近年来,使用深度学习方法对各种点云任务进行分析与处理是学术界研究和工业界应用的重点和热点。点云上采样是其中一项重要的点云数据处理任务,其结果的好坏关系着多种下游任务的优劣。通过分析和总结现有点云上采样方法的优缺点,综合理论创新和实际应用,本文将点云上采样任务作为研究目标,将深度学习方法作为主要研究工具,针对各种难点问题(如仅提取单一类型的全局特征、单一尺度的局部特征、单一表征的点特征)提出了对应的特征融合方式,针对各种子任务(如几何上采样、任意上采样)设计了对应的上采样网络。在多个公开的合成物体数据集和真实场景数据集上的实验结果均表明,本文提出的方法在各自的子任务中都获得了一定程度的性能提升。具体而言,本文的主要创新性研究成果包含以下四个方面:(1)基于多类型特征融合的有监督几何上采样方法。针对点特征提取不够丰富的问题,本文提出了用于有监督几何上采样的图注意力卷积网络PU-GACNet,该网络可以有效且高效地从点云中提取并融合多种类型的特征。具体来说,首先设计了一个图注意力卷积模块作为特征提取器,使用了具有注意力机制的图卷积操作,可以更好地聚合全局特征和局部特征。其次设计了一个边缘感知shuffle模块作为特征扩展器,以平滑生成点及其近邻点的特征,同时有效地保留边缘特征。各种实验验证了PU-GACNet在有效性和高效性方面普遍优于基准方法。(2)基于多尺度特征融合的无监督几何上采样方法。为了提升模型在大规模真实扫描的点云数据上的泛化性,本文提出了一种用于无监督几何上采样的空间感知Transformer孪生网络UPU-SNet。该网络主要实现的功能流包括一个点重采样任务流和一个点上采样任务流,从点云中提取并融合高维多尺度特征,包括局部到全局特征和全局到局部特征。在特征提取器中,设计了一个全局局部Transformer模块,由多个级联的基于多头交叉注意力机制的子块组成,用于编码多尺度信息。类似地,设计了一个用于特征扩展的Transformer Shuffle模块,以有效地捕获空间关系。通过一系列定量和定性实验表明,与现有的无监督上采样方法相比,UPU-SNet实现了更优的上采样质量,并且与一些有监督上采样方法相比,也取得了有竞争力的结果。(3)基于多表征特征融合的自监督任意上采样方法。着眼于无监督上采样点分布不均匀和几何上采样点密度恒定不变的问题,本文提出了一种用于自监督任意上采样的点体素孪生网络S~3U-PVNet。该网络主要实现的功能流包括一个Down-Up任务流和一个Up-Down任务流,核心是一系列点体素特征融合模块,从点云中提取并融合不同表征方式的多粒度特征,即互补的粗粒度体素特征和细粒度点特征。这样在学习点之间的局部和全局空间关系的同时,解决由于点云的不规则性而带来的特征提取不准确的问题。大量实验表明,S~3U-PVNet的上采样性能更优且鲁棒性更强。无论是输入不同点数的点云还是生成不同点数的点云,S~3U-PVNet总能产生均匀的、稠密的、无噪声的上采样结果。此外,各种表面重建的结果也表明该上采样网络可以实现精确的三维重建。(4)基于多任务协同学习的一体化上采样方法。针对点云上采样如何更好地服务于下游任务的问题,本文提出了一种一体化点云上采样网络框架PU4MT用于多任务协同学习。该网络框架旨在将传统的只关注低级特征提取的上采样网络扩展为更适合多属性、多语义特征学习的上采样网络,包括处理点云着色和表面重建等低层次下游任务的多任务并行上采样网络PU4MT(P)、处理形状分类和部件分割等高层次下游任务的多任务串行上采样网络PU4MT(S)。多种任务的实验结果均表明,PU4MT可以进一步提升点云的生成效果和重建质量,改善点云的视觉可读性和语义可理解性。综上,本文创新性地提出了多种基于特征融合的点云上采样方法,提高了上采样的计算效率和网络性能,并进一步将这些方法应用在一些下游任务中。更多还原

【Abstract】 With the popularization of 3D scanning devices such as Li DAR and the development of data acquisition technologies,research in the field of computer vision has gradually expanded from 2D plane data represented by images and videos to 3D spatial data.Among these representations,point clouds are the most flexible and commonly used,preserving the most primitive shape and geometric information of objects.Using deep learning methods for analysis and processing of point clouds in various point cloud tasks has become a focus and hotspot in both academic research and industrial applications.Point cloud upsampling is one of the important point cloud data processing tasks,and the quality of its results directly affects the performance of various downstream tasks.By analyzing and summarizing the advantages and disadvantages of existing point cloud upsampling methods,and integrating theoretical innovation with practical applications,this thesis takes point cloud upsampling as the research objective and deep learning methods as the main research tools.Feature fusion modes are proposed to solve various difficult problems including extracting only single-type global features,single-scale local features,and single-representation point features.Deep learning-based upsampling networks are also designed for various subtasks,such as geometric upsampling and arbitrary upsampling.Experimental results on multiple public synthetic object datasets and real scene datasets show that the proposed methods achieve performance improvements in their respective subtasks.Specifically,the main innovative research contributions of this thesis include the following four aspects:(1)A supervised geometric point cloud upsampling method based on multi-type feature fusion.In response to the problem of insufficient point feature extraction,this thesis proposes a graph attention convolution network for supervised geometric upsampling,i.e.,PU-GACNet,which can effectively and efficiently extract and fuse multi-type features.First,a graph attention convolution module is designed as the feature extractor that utilizes graph convolution operation with attention mechanism to better integrate global and local features.Secondly,an edge-aware shuffle module is designed as the feature extender to smoothly generate features of points and their neighbor points,while effectively preserving many edge features.Various experiments verify that PU-GACNet generally outperforms baseline methods in terms of effectiveness and efficiency.(2)An unsupervised geometric point cloud upsampling method based on multi-scale feature fusion.Due to the weak generalization of large-scale real-scanned point cloud datasets and the lack of consideration for multi-scale point feature fusion,this thesis proposes a spatial-aware Transformer siamese network for unsupervised geometric upsampling,i.e.,UPU-SNet,which extracts and fuses high-dimensional and multi-scale local-to-global features and global-to-local features.The main flows implemented by this network include a point resampling taskflow and a point upsampling taskflow.In the feature extractor,a global-local Transformer module consists of multiple cascaded blocks based on the multi-head cross-attention mechanism to encode multi-scale information.Similarly,a Transformer Shuffle module is designed for feature extension to effectively capture spatial relationships.A series of quantitative and qualitative experiments can prove that UPU-SNet achieves better upsampling quality than unsupervised upsampling methods,and also generates comparable results with supervised upsampling methods.(3)A self-supervised arbitrary point cloud upsampling method based on multi-representation feature fusion.Focusing on the problems of nonuniform point distribution in unsupervised upsampling and constant point density in geometric upsampling,this thesis proposes a point-voxel siamese network for self-supervised arbitrary upsampling,i.e.,S~3U-PVNet.The main flows implemented by this network include a Down-Up taskflow and an Up-Down taskflow,with a series of point-voxel feature fusion modules,which can extract and fuse multi-granularity and multi-representation features,including complementary fine-grained point features and coarse-grained voxel features.This can reduce the inaccurate feature extraction caused by the irregularity of point clouds while learning the local and global spatial relationships between points.A large number of experiment results demonstrate that S~3U-PVNet has better upsampling performance and stronger robustness than many baseline methods.Whether it requires different numbers of inputs or generates different numbers of outputs,S~3U-PVNet can always produce uniform,dense,and noiseless upsampling results.In addition,various surface reconstruction results indicate that our upsampling network can achieve accurate 3D reconstruction task.(4)An integrated point cloud upsampling method based on multi-task collaborative learning.To address the issue of how point cloud upsampling can better serve downstream tasks,this thesis proposes an integrated point cloud upsampling network architecture for multi-task collaborative learning,i.e.,PU4MT.It extends the traditional upsampling networks that only focus on low-level feature extraction to novel upsampling networks that are more suitable for multi-attribute and multi-semantic feature learning.The parallel upsampling network PU4MT(P)handles low-level tasks including point cloud colorization and surface reconstruction,and the serial upsampling network PU4MT(S)handles high-level tasks including shape classification and part segmentation.The experimental results show that PU4MT can further improve the generation effect and reconstruction quality of point clouds,and enhance their visual readability and semantic comprehensibility.In summary,this thesis innovatively proposes various point cloud upsampling methods based on feature fusion modes,which improve the computational efficiency and network performance of upsampling,and further applies these methods to some downstream tasks.更多还原