【作者】 王辉；

【导师】 高宏娟； 张文学；

【作者基本信息】 宁夏大学 ， 计算机技术（专业学位）， 2024， 硕士

【摘要】 三维关键点检测是计算机视觉领域的基础任务,旨在3D点云中找到稳定、具有明确位置且在任意刚体变换下高度可重复的点,这些点在三维重建、自动驾驶、机器人导航以及虚拟现实等人工智能领域有着广泛的应用。现有的基于自监督学习的关键点检测方法能够从大量未标注的数据中学习特征表示,克服了数据集规模有限和数据样本稀疏的问题,然而也存在一些不足:特征提取阶段,由于没有标签信息作指导,导致提取的特征缺乏针对性,缺乏有效的几何信息和语义信息,最终影响关键点检测的准确性;对输出的关键点数量没有太多的控制,关键点过多或过少都会对后续点云重建有不利的影响;重建的点云缺乏局部细节信息,粗糙的重建效果导致关键点检测任务性能不佳。针对以上问题,本文的研究内容如下:（1）本文提出了一种基于多视图特征提取的三维关键点检测方法。采用传统渲染方式和Soft Rasterizer相结合的方法获取点云每个点的多视图补丁,并通过局部空间注意力机制提高对多视图补丁的特征提取能力,最后进行特征融合,获取点云的多视图特征,进而帮助模型更好的理解点云数据中的结构和语义信息,避免了特征提取不充分、缺乏有效几何信息的问题。通过生成式自监督的方法,利用显著的关键点信息重建原始点云,通过不断优化点云重构误差,最终检测到最优的关键点。该方法能够更好地理解三维形状的局部结构和特征,捕捉关键点周围的上下文信息,从而提高关键点检测的准确度。实验结果表明,与Harris-3D、UKPGAN等关键点检测方法相比,本文提出的方法在KeyPointNet、SMPL数据集上的mIoU值最高,其中在KeypointNet数据集上的mIoU指标提升最明显,达到了 72.4%,相较于UKPGAN的方法提高了 3.8%。（2）本文提出了一种基于精细化二次重建和稀疏控制的三维关键点检测方法,采用生成-对比的自监督学习策略,探究了关键点检测和点云重建之间的关系。为了检测出稀疏的关键点,通过GAN网络来控制关键点的概率分布;提出了精细化二次重建网络,通过连接2D网络特征、全局特征以及重建点云的逐点特征,通过卷积操作来预测点的偏移量。在粗重建的基础上生成精细化的三维点云,从而实现关键点的精准定位。实验结果表明,本文的方法不仅在KeypointNet、SMPL数据集上的mIoU指标超过其他关键点检测方法,而且在3Dmatch、ETH数据集上进行几何配准任务时,特征匹配召回率（Feature Matching Recall）、配准召回率（Registration Recall）、内点比例（Inlier Ratio）指标均高于Harris-3D、UKPGAN的方法,展现了出色的泛化性能。（3）本文设计开发了三维关键点检测系统,实现了检测三维数据关键点的功能。为满足用户多样化的需求,系统还实现了 ISS、Harris-3D、SIFT₃D等传统的关键点检测方法。更多还原

【Abstract】 3D keypoint detection is a fundamental task in the field of computer vision,aiming to find points in 3D point clouds,that are stable,well-defined locations,and highly repeatable under arbitrary rigid body transformations.Those points has a wide range of applications in 3D reconstruction,automatic driving,robot navigation,virtual reality,and other fields of artificial intelligence.Existing key point detection methods based on self-supervised learning can learn feature representations from a large number of unlabeled data,overcoming the problems of limited dataset size and sparse data samples.However,there are still some shortcomings:In the stage of feature extraction,because there is no label information to guide,the extracted features lack pertinence and effective geometric and semantic information,which ultimately affects the accuracy of key point detection.There is not much control over the number of output key points,and too many or too few key points will have adverse effects on the subsequent point cloud reconstruction.The reconstructed point cloud lacks local details,and the rough reconstruction results in poor performance of the key point detection task.Aiming at the above problems,the research content of this paper is as follows:（1）This paper proposes a 3D keypoint detection method based on end-to-end multi-view feature extraction.The traditional rendering method and Soft Rasterizer are combined to obtain multi-view patches of each point in the point cloud,and the local spatial attention mechanism is used to improve the feature extraction ability of multi-view patches.Finally,feature fusion is performed to obtain multi-view features of the point cloud.Thus helping the model to better understand the structure and semantic information in the point cloud data,avoiding the problem of inadequate feature extraction and lack of effective geometric information.Through generative self-supervision method,the original point cloud is reconstructed by using significant key point information,and the optimal key point is finally detected by constantly optimizing the point cloud reconstruction error.This method can better understand the local structure and features of 3D shapes,capture the context information around key points,and improve the accuracy of key point detection.The translation experiment results show that compared with Harris-3D,UKPGAN and other key point methods,the proposed method has the highest mIoU value on the KeyPointNet and SMPL data sets,and the mIoU index on the KeypointNet data set has the most obvious improvement,reaching 72.4%.Compared to the UKPGAN method,the improvement was 3.8%.（2）This paper proposes a 3D key point detection method based on refined secondary reconstruction and sparse control,adopts a generative-contrastive self-supervised learning strategy,and explores the relationship between key point detection and point cloud reconstruction.In order to detect sparse key points,the probability distribution of key points is controlled by GAN network.A refined secondary reconstruction network is proposed,which connects 2D network features,global features and point-bypoint features of the reconstructed point cloud,predicts the offset of points through convolution operations,and generates refined three-dimensional point clouds on the basis of rough reconstruction,so as to achieve accurate positioning of key points.The experimental results show that the MIOU index of the proposed method not only exceeds that of other key point detection methods on the KeypointNet and SMPL datasets,but also improves the performance of geometric registration tasks on 3Dmatch and ETH datasets.The indexes of Feature Matching Recall,Registration Recall and Inlier Ratio are all higher than those of Harris-3D and UKPGAN,showing excellent generalization performance.（3）This paper designs and develops a 3D keypoint detection system,which realizes the function of detecting keypoints of 3D data.To meet the diversified needs of users,the system also implements ISS,Harris-3D,SIFT₃D,and other traditional key point detection methods.更多还原