【作者】 张琦；

【导师】 王庆；

【作者基本信息】 西北工业大学 ， 计算机科学与技术， 2021， 博士

【摘要】 光场以空间光线为基本单元,通过对光线位置和角度信息进行采样,可实现重聚焦、变视点、扩展景深等新颖应用,是计算机视觉与计算摄像学的重要理论创新点和技术突破口。但是,现有光场成像理论存在投影模型不统一、光场成像装置存在空间和角度分辨率折衷等问题,尚无法满足应用需求。本文以多视光场为研究对象,从光场相机投影模型出发,重点研究光场相机光线采样及变换过程,分析其对场景三维结构的影响。另一方面,本文从Plücker光线出发,重点研究光线空间对极几何,分析多视光场内在射影关系,进而研究多视光场相机自标定和三维重建方法。论文研究工作的主要创新点包括:(1)提出了统一描述异构光场相机的多投影中心模型。从传统相机的中心投影模型出发,推导了三维空间点投影变换矩阵,提出了隐含视点偏移的光场相机畸变模型,实现了基于空间点的光场相机标定方法。此外,分析了多投影中心模型对平面和二次曲线的映射,推导了共心二次曲线的共自配极三角形,阐述了其性质并在光场中重建共自配极三角形,实现了基于共心二次曲线的光场相机标定方法。仿真与真实光场数据的实验结果表明,多投影中心模型可统一描述光场相机的异构特性,并利用不同标定物精确标定光场相机。(2)提出了统一描述光场相机光线采样及变换的光线空间投影模型。从Plücker光线出发,提出了6×6光线空间内参矩阵和投影矩阵,分别描述了光场相机光线采样和变换过程。根据Plücker光线的数学定义,推导了其在Klein曲面的高维特性及光线空间投影变换的不变性。基于光线空间投影矩阵,建立了空间点与光线间线性约束,提出了光场相机标定方法,定义了异面光线间几何距离,并用于非线性优化。仿真与真实光场数据的实验结果表明,光线空间投影模型可统一描述光线采样及变换过程,并精确标定光场相机。(3)提出了描述多视光场关联关系的光线空间对极几何。从光线空间投影模型出发,研究了光线空间对极几何,描述了二视图光场间内在射影几何,其独立于场景结构,只依赖于光场相机的内外参数,推导了6×6光线空间基本矩阵,并给出其性质。通过分析光线空间基本矩阵的正交性约束和奇异性约束,提出了光线空间基本矩阵计算方法,定义了光线对称对极距离,并用于非线性优化。通过仿真与真实光场数据的实验,验证了光线空间基本矩阵计算方法的准确性与可靠性,展示了光线空间对极几何对于多视光场应用的理论指导意义。(4)提出了多视光场相机自标定和三维重建方法。从光线空间对极几何出发,构建了6×6光线空间单应,描述了多视光场间同一光线的关联性关系,分解了仅与旋转矩阵相关的光线空间无穷单应,推导了绝对二次曲线的光线束。根据光线空间无穷单应的旋转共轭,计算光线空间无穷单应,估计光场相机内参数及相对姿态,定义了光线间Sampson距离,并用于非线性优化,最终实现光场相机及场景的计算重构。仿真与真实数据的实验结果表明,所提算法在精确重构光场相机投影矩阵的同时,可直接从多视光场实现三维重建。更多还原

【Abstract】 Light field intrinsically represents the rays in 3D space.Based on the spatial and angular sampling of rays in 3D space,advanced imaging applications such as digital refocusing,shifting view and all-in-focus imaging have been investigated.Light field imaging is theoretical novelty and technical breakthrough in computational photography and computer vision.However,the existing light field imaging theory has some disadvantages,such as a non-uniform light field camera projection model,small baseline,and spatial and angular resolution trade-off,which is unable to meet the requirements of applications.In this thesis,the generic light field camera pro-jection model and the corresponding relationship between multi-view light fields are explored.On the one hand,ray sampling and transformation are exploited to deduce the relationship with geometric structure.On the other hand,with Plücker parameterization,the epipolar geometry is explored to describe the intrinsic projective geometry between light fields.The light field camera self-calibration and 3D reconstruction is then developed.The main contributions are:(1)A generic multi-projection-center(MPC)model for different light field cameras is pro-posed.According to the pinhole model for the traditional camera,an MPC model with 6 intrinsic parameters to characterize different imaging formations,including traditional light field camera and focused light field camera,is proposed.A 3D projective matrix is deduced to describe the relationship between geometric structure and the light field.A distortion model for the light field camera is proposed,where the shifting view points are considered.A light field camera calibration method is then proposed.Besides,the projections of an MPC model on conics and planes other than points are explored.The property and reconstruction of the common self-polar triangle with respect to concentric circle and ellipse are derived to calibrate a light field camera.Experimental results on both simulated and real scene data have verified the effectiveness and robustness of the MPC model for generic characterization of different light field cameras and calibration with different geometric objects.(2)A ray-space projection(RSP)model is proposed to uniformly describe ray sampling and ray transformation.With Plücker parameterization,a 6 × 6 ray-space intrinsic matrix and ray-space projection matrix are derived to describe ray sampling and ray transformation of a light field camera respectively.Also,based on the algebraic definition of Plücker rays,the self-constraint of a Plücker ray on Klein quadric is explored,and the Klein quadric transformation under RSP is invariant.Based on the RSP model,a linear point-ray constraint is deduced for light field camera calibration.The ray-ray geometric distance is defined for non-linear optimiza-tion.Experimental results on both simulated and real scene data have validated the effectiveness and robustness of RSP model for the uniform description of ray sampling and transformation and light field camera calibration.(3)A ray-space epipolar geometry is proposed to model correspondence between multi-view light fields.With the RSP model,ray-space epipolar geometry which intrinsically encap-sulates the complete projective geometry between two light fields is proposed.It is independent of scene structure and only depends on the intrinsic and extrinsic parameters of light field cam-era.A 6 × 6 ray-space fundamental matrix and its properties are then deduced to constrain ray-ray correspondences for general and special motions.According to the analysis of orthog-onal and singular constraints,a computation of the ray-space fundamental matrix is proposed.Besides,the ray symmetric epipolar distance is defined for non-linear optimization.Experimen-tal results on both simulated and real scene data have validated the effectiveness and robustness of the ray-space fundamental matrix estimation method,and have demonstrated the theoretical breakthrough of ray-space epipolar geometry for multi-view light fields based applications.(4)A self-calibration method and 3D reconstruction specifically designed for a light field camera are proposed.With the ray-space epipolar geometry,a 6 × 6 ray-space homography is established to relate the ray transformation among different light fields.The ray-space infinity homography is decomposed.A new concept of “rays of the absolute conic”(RAC)is defined and related to the ray-space infinity homography.Given that the ray-space infinity homography is a conjugate rotation,it can be accurately recovered from ray-ray correspondences and solved for self-calibration.The ray-ray Sampson distance is then defined for non-linear optimization.The 3D reconstruction of the light field camera and structure is finally implemented.Experi-mental results on both simulated and real scene data have validated that,the proposed method could simultaneously obtain superior self-calibration and 3D reconstruction.更多还原