【作者】 李敏；

【导师】 许端清； 刁常宇；

【作者基本信息】 浙江大学 ， 计算机科学与技术， 2020， 博士

【摘要】 从图像中获取物体表面的三维信息以及材质信息是计算机视觉、计算机图形学领域中的一项基本任务,在虚拟现实、逆向工程以及文化遗产保护等领域有着广泛的应用,但基于图像的三维重建方法通常不适用于包含大量高光反信息的物体,如塑料制品,瓷器和玉器等。如何获取表面存在高光现象的物体的三维模型以及表面材质属性是一个具有实际应用价值的问题。大部分基于图像的三维重建方法如结构光三维扫描和立体匹配等,是基于物体表面的特征进行三维重建,意味着当物体表面纹理较弱时重建后将无法得到精细的三维模型。而光度立体算法是基于光照模型提出,具有不受纹理影响和逐点(per-pixel)三维重建等优点,因此一直是三维重建领域的研究热点。典型的光度立体视觉算法是基于朗伯模型提出的,然而现实生活中几乎不存在朗伯反射体(理想漫反射体),不同的物体其表面具有不同的反射特性,这也使得如何有效重建非朗伯体的的三维模型成为巨大的挑战。针对上述挑战,本文关注日常生活中广泛存在的高光现象,对非朗伯物体的三维重建展开深入的研究。本文从减弱经典的光度立体算法对于光照以及物体表面反射模型约束的角度出发,融合其它三维重建算法,重建精细的物体表面形状,并在多个数据上进行了丰富的实验验证。此外,本文提出针对光度立体算法的扫描设备,并针对多视图光度立体算法提出公开数据集和评测标准。具体而言,本文的主要创新和工作可以总结为以下几个方面:1.利用光度立体求解的法向进行物体表面三维重建时,因高光阴影等现象的存在,导致所得到的三维模型存在高频细节完好但低频信息变形失真等问题。为此,本文将其它三维重建方法得到的三维形状与光度立体求出的物体表面法向相融合,提出一种基于凸优化的三维重建方法,通过全局优化进行高低分辨率信息的融合,阻止因异常点而使解空间陷入局部最优值的问题。本文提出的优化函数包含使用?1范数的数据项和全变分约束项两部分,通过将优化函数映射到高维空间生成变分模型,原问题变成存在全局最优解的凸问题。实验结果表明该优化框架可以有效重建高精度的三维模型。2.大部分光度立体算法为满足朗伯模型的假设而将高光点视为异常点直接丢掉,这样做尽管简化了算法复杂度但却不能有效的处理现实生活中普遍存在的高光现象且无法正确获取物体表面的反射系数。本文提出一种基于双色反射模型的方法,充分利用物体表明的漫反射分量和高光信息,解决如何有效求解非朗伯体表面法向和反射系数的问题。本文利用线性优化方法对输入图像进行高光和漫反射的分离,分离出的漫反射信息用于求解物体表面的法向,分离出的高光信息被用来进一步提高法向精度以及求解表面反射系数,实验结果表明该方法有效提升了求解的表面法向和反射系数的准确性。3.经典的光度立体是在固定视角下,利用不同方向的无穷远处光照得到多幅图像,并求解物体的表面法向,拍摄时要求相机和物体的位置不变,准确的说,最终只能得到物体的部分三维模型。由于较为严苛的实验环境,导致大部分已提出的光度立体方法仍是在实验室环境下进行评测,如何将放松光度立体算法的约束应用到实际环境中仍然是亟待解决的难题。本文主要研究了如何求解一般光源条件尤其是近光源条件下多视图光度立体的问题。在各向同性材质的假设下,本文提出便携的扫描原型系统,针对设备中因物体离相机过近而产生的近光源影响和相机透视投影影响,本文在对光源分布和设备标定进行深入分析后提出一种基于分块的计算策略。实验证明,该策略可以有效消除近光源和透视投影的影响,明显提高三维重建结果的质量。此外,为更好的探究多视图光度立体的优缺点,本文还提出了针对多视图光度立体算法的公开数据集和评测标准,以进一步了解光度立体算法目前存在的问题,推动光度立体算法在真实环境下的应用。更多还原

【Abstract】 Reconstructing the shape and reflectance of an object from images is a significant task in computer vision and graphics that has led to a variety of applications including virtual reality,inverse engineering,and cultural relics protection.However,most of the image-based 3D reconstruction methods can not be applied to specular objects,i.e.plastics,porcelain,or Chinese jade.It is practical and valuable to focuse on acquiring the shape and reflectance of non-Lambertian objects.Most image-based 3D reconstruction methods,i.e.structured light and stereo matching,fail on the texture-less surfaces due to the lack of image features.The classic photometric stereo problem assumes a Lmabertian surface reflectance model and recovers per-pixel normal maps without relying on image features.However,the real-world is full of non-Lambertian objects,covering various surface materials.It is a significant challenge to recover surface shapes and reflectance of non-Lambertian objects.Considering the above challenges,the thesis is about the most widely existed high-lights in daily life and carrys out researches on the 3D reconstruction of non-Lambertian objects.In this thesis,we relax constraints of lightings and reflectance models of classic photometric stereo.High detailed surfaces are reconstructed by combining with other3 D reconstruction methods.In addition,we build the scanning setup for photometric stereo and quantitatively evaluate state-of-the-art multi-view photometric stereo on a newly collected benchmark dataset,which is publicly available for inspiring future re-search.Specifically,the main innovations and work of this article can be summarized as follows:1.When performing 3D reconstruction from the normal estimated from photomet-ric stereo,the 3D model contains high-frequency details while the geometry is distorted due to the existence of highlights.In this thesis,we present a convex framework to acquire high resolution surfaces.It is typical to couple a structure-light setup and a photometric method to reconstruct a high resolution 3D surface.We derived a global optimizer to fuse high frequency details and low frequency geometry information for preventing the solution stuck in a local minima.We develop a convex variational model by incorporating a total variation(TV)reg-ularization term with a data term to generate the surface.Through relaxing the model to an equivalent high dimensional variational problem,we obtain a global minimizer of the proposed problem.Results on both synthetic and real-world data show an excellent performance by utilizing our convex variational model.2.Most photometric stereo methods discard specular outliers directly in order to meet the Lambertian assumption.Although the complexity of the algorithm is simpli-fied,it cannot deal with the common specular reflection in real life effectively and cannot obtain surface reflection correctly.The thesis presented a new method based on dichromatic model aiming to solve the non-Lambertian surface normal and reflectance by making full use of the diffuse and specular components.We separate the diffuse component and the specular component based on the dicromat-ical model by a linear optimization.The diffuse component is utilized to compute surface normals while the highlight is employed to improve the accuracy of the estimated surface normals and reflectance.The experimental results demonstrate that the method is effective to improve the accuracy of the surface normal and reflectance.3.The classic photometric stereo assumes a Lambertian model,a fixed camera,and known directional illuminations to compute the normal map from multiple images,only reconstructing parts of the 3D model.Most proposed photometric stereo methods are still limited to the laboratory environment,so it is a huge challenge to apply the method into real life.The thesis presents a method to capture 3D shape with a multi-view photometric stereo technique under relaxed lighting conditions.The capture setup we proposed in this thesis works for general isotropic materi-als.To eliminate the effects of the near-light and perspective camera condition,a block based strategy is proposed after thorough analysis on lighting distributions and setup calibrations.The experimental results prove that the block based strat- egy can eliminate the near-light and perspective camera model effects and improve the quality of the reconstructed 3D model obviously.In addition,we also quantita-tively evaluate state-of-the-art multi-view photometric stereo on a newly collected benchmark dataset,which is publicly available for inspiring future research.更多还原