【作者】 王云；

【导师】 王松； 朱建科；

【作者基本信息】 天津大学 ， 工程（专业学位）， 2021， 硕士

【摘要】 随着现代科学技术的快速发展,逐渐构建起了庞大的相机系统,各类手机相机、GoPro等移动相机的更新迭代也推动了相机网络的发展。技术的革新为我们了解分析行人的特征及关系起到了非常重要的作用,也为目标之间的关联提供了丰富的信息来源。在不同的视觉应用中,从不同视角的图片中识别同一个目标起着非常重要的作用。然而,传统的基于视频的任务中往往采用的是单个或两个相机来获取视频,且大部分相机是固定的,因此会受到视野范围的限制。移动相机的出现很好的弥补了这一缺陷。因此本文利用多个可移动相机来更全面地获取场景中人物的信息,从而能更好的研究不同摄像机同时拍摄的多视角图像上识别同一个目标这一重要问题,即多视角多目标关联问题(MvMhA)。与先前的两个视角目标关联工作不同,本文着重研究具有多个(两个及以上)视角且视角不固定的更一般且更具挑战性的场景。另外,本文所研究场景中的目标可以出现在任意数量的视角中,这是先前未知的。针对以上挑战,本文开发了一个新的基于端到端深度网络的框架来解决此问题,分别基于循环神经网络(RNN)和图神经网络(GNN)两种方法对该问题进行了深入研究:(1)基于RNN的方法首先提取每个图像中每个目标的特征。通过计算特征之间的成对相似性得分来构建一个综合的亲和力矩阵。最后,利用提出的深度分配网络(DAN),将亲和力矩阵转换为分配矩阵,为MvMhA问题提供二进制匹配关联结果。(2)基于GNN的方法同样先利用特征提取网络获取每个目标的特征,紧接着以获取到的特征作为图神经网络的节点特征,构建多视角多目标的图匹配网络模型。此外,我们还建立了一个合成数据集、一个虚拟图像数据集和一个真实图像数据集,以验证所提出方法的有效性。本文还在其他三个公共数据集上进行了测试,结果显示该模型有非常好的跨域性能。更多还原

【Abstract】 With the rapid development of modern science and technology,large-scale camera system has become possible.In particular,the new generations of various mobile cameras,such as smart-phone cameras and GoPro has brought us the fast development of camera networks.These new techniques enable more effective and efficient understanding and analysis of the characteristics of and relationships among multiple targets,especially pedestrians,in various kinds of scenes.In many vision applications,identifying the same persons across different views plays an important role.However,traditional approaches often use a single or two cameras to obtain videos,and in most cases,cameras are fixed,which have limited field of view.The emergence of mobile cameras makes up for this limitation.In this thesis,we use multiple mobile cameras to study the important problem of identifying the same target on multi-view images that are taken by different cameras at the same time-we denote this problem to be Multi-view Multihuman Association(MvMhA).Different from the previous research work on human association across two views,this thesis focuses on more general and more challenging cases with multiple(two or more)views and non-fixed views.In addition,each involved person in the scene can appear in all the views,or only in a subset of views,which was not priorly known.To address these challenges,we develop a new framework based on end-to-end deep network and conduct in-depth research by developing two methods based on Recurrent Neural Network(RNN)and Graph Neural Network(GNN),respectively:(1)The RNN-based method first uses an appearance-based deep network to extract the features of each object on each image.A comprehensive affinity matrix is constructed by calculating the pairwise similarity scores between all the detected targets.Finally,we propose a Deep Assignment Network(DAN)to transform the affinity matrix into an assignment matrix,which provides a binary assignment result for MvMhA.(2)The GNN-based method also first uses the feature extraction network to extract the features of each object under each view,and then uses the obtained features as the node features of the graph neural network to construct a multi-view and multi-object graph matching network model.In addition,we build both a synthetic matrix dataset,a synthetic image dataset and a real image dataset to verify the effectiveness of the proposed method.We also test the trained network on other three public datasets,resulting in very good cross-domain performance.更多还原