【作者】 杨丹；

【导师】 刘日晨；

【作者基本信息】 南京师范大学 ， 计算机科学与技术， 2023， 硕士

【摘要】 传统的医学数据可视化系统对于三维医学体数据的显示,基本都是使用二维的计算机屏幕和传统的外部设备,如鼠标、键盘等进行交互探索。二维的计算机屏幕在显示三维体数据时,无法获得深度信息,用户无法看清体数据的内部细节。相比于传统的计算机窗口界面,VR的优势在于更加真实的渲染效果和体验。但在目前的沉浸式系统中,可供用户使用的交互工具并不多,而且手柄控制器、定位器校准和边界设置对用户来说很麻烦,三维数据深度信息的视觉混乱和3D遮挡,使得3D空间中的沉浸式插图十分困难。因此,本文设计了从2D交互式解释空间到3D沉浸式可视化空间的自动迁移方法,通过该方法,可以在沉浸环境中自动生成所需的视觉样式。具体而言,本文的工作主要包括:首先,本文介绍了医学数据二维解释和三维沉浸式探索的国内外研究现状,并讨论了论文的研究背景和意义,对于医学数据的二维解释可视化、风格迁移和沉浸式环境中的解释性可视化发展现状进行了分析和介绍。在提出的整个方案中,系统基于浏览器页面对二维数据进行交互式解释,基于Unity平台展示迁移后的三维体数据,最终用Oculus Quest2设备对三维体数据进行探索。其次,在本文中介绍了整个系统的框架。本篇论文的方法主要思想是通过风格迁移,将二维交互式解释迁移到三维空间,以此获得用户满意的效果和需求。先通过信息熵算法推荐一组具有高熵的切片作为候选,用户可以选择自己的目标切片。然后用户可以在前端界面通过充分利用交互工具以及OpenCV图像处理算法,通过利用二维图像处理的成熟性,风格化解释各种切片数据。此后,基于均值漂移的种子点推荐算法在切片上推荐种子点,然后通过基于密度的区域增长算法将二维切片的风格样式迁移到三维空间中,最后用户(一个或多个)生成的所有风格通过沉浸式光线投射算法都被整合合并到沉浸式设备中。在论文中提出的方法,是充分利用了OpenCV以及其他交互式解释在二维空间的成熟性,并将样式迁移到三维空间,从而解决三维空间中难以改变数据风格样式的困难,借助二维解决三维中的难点。同时,利用二维迁移三维,避免了三维数据视觉混乱和3D遮挡带来的交互困难和复杂程度,极大提高了三维体数据风格解释效率。最后,对于提出的方法中的所有算法进行设计并实验。主要包括信息熵推荐切片算法、基于均值漂移的种子点推荐算法、基于密度的区域增长算法、沉浸式光线投射算法和多摄像机融合算法。对于提出方法所实现的实验效果也进行了评估与测试。对于所用到的CT手部数据、头部数据、肺部数据分别进行测试,并邀请用户进行对系统的可用性和表达性方面进行评估。更多还原

【Abstract】 Traditional medical data visualization systems for the display of 3D medical vol-ume data basically use 2D computer screens and traditional external devices,such as mouse and keyboard,for interactive exploration.The 2D computer screen does not have access to depth information when displaying 3D volume data,and the user cannot see the internal details of the volume data.Compared to the traditional computer window interface,VR has the advantage of more realistic rendering effect and experiences.However,in current immersive systems,there are not many interaction tools available for users,and handle controller,locator calibration and boundary setting are trouble-some for users.The visual clutter of 3D data depth information and 3D occlusion make immersive illustration in 3D space very difficult.Therefore,in this paper,we design an automatic migration method from 2D interactive space to 3D immersive visualization space,by which the required visual styles can be automatically generated in the immer-sive environment.Specifically,the work in this paper focuses on:Firstly,this paper introduces the current state of domestic and international re-search on 2D illustration and 3D immersive exploration of medical data,and discusses the research background and significance of the paper.The current state of development of 2D illustration visualization,style transfer and illustrative visualization in immersive environments for medical data is analyzed and introduced.In the proposed whole scheme,we illustrate the 2D data interactively based on the browser page,display the transferred 3D volume data based on the Unity platform,and finally explore the 3D volume data with the Oculus Quest2 device.Thereafter,a mean drift based seed point recommendation algorithm recommends seed points on the slices,then the stylized styles of 2D slices are transferred to 3D space by a density based region growing algo-rithm,and finally all the styles generated by the user(one or more)are integrated and merged into the immersive device by an immersive ray casting algorithm.Secondly,in this paper,the framework of the whole system is presented.The main idea of our approach is to transfer 2D interactive illustration to 3D space through style transfer in order to obtain satisfactory results and demands from users.We first recom-mend a set of slices with high entropy as candidates through the information entropy algorithm,and users can choose their target slices.The user can then illustrate various slice data stylistically in the front-end interface by leveraging interactive tools as well as OpenCV image processing algorithms by exploiting the sophistication of 2D image processing.Thereafter,a mean drift based seed point recommendation algorithm recommends seed points on the slices,then the stylized styles of 2D slices are trans-ferred to 3D space by a density based region growing algorithm,and finally all the styles generated by the user(one or more)are integrated and merged into the immersive device by an immersive ray casting algorithm.The method proposed in the paper takes full advantage of OpenCV and other in-teractive illustrations in 2D and transfer the style to 3D space,thus solving the difficulty of changing the style of data style in 3D space with the help of 2D.At the same time,using 2D to transfer 3D avoids the difficulty and complexity of interaction caused by visual confusion and 3D occlusion of 3D data,and greatly improves the efficiency of3D volume data style illustration.Finally,all algorithms in the proposed method are designed and experimented.They mainly include information entropy recommendation slicing algorithm,seed point recommendation algorithm based on mean drift,density-based region growth algorithm,immersive light projection algorithm,and multi-camera fusion algorithm.The experi-mental results achieved by the proposed methods are also evaluated and tested.The CT hand data,head data,and lung data used were tested separately and users were invited to evaluate the usability and expressiveness aspects of the system.更多还原