【作者】 刘阳；

【导师】 陈大力；

【作者基本信息】 东北大学 ， 控制工程， 2021， 硕士

【摘要】 铝合金是工业生产上应用最为广泛的有色金属材料之一,由于其良好的材料性能,铝合金被广泛的应用于汽车、航天、航海以及铁路运输等领域。铝合金材料的性能主要取决于其微观组织结构,因此通过微观组织图像分析合金性能是专家学者常用的方法之一。本文以铝合金微观组织图像为研究对象,结合深度学习、图像处理、语音识别以及自然语言处理等方法,实现了铝合金微观组织图像问答系统,具体工作如下:(1)提出了基于视觉特征和文本信息融合的微观组织图像问答方法。该方法首先采用特征提取网络提取微观组织图像和问题文本特征,然后通过特征融合方法对图像特征和文本特征进行融合,最终通过深度分类网络得到问题的答案。在该方法中,我们使用了 ResNet深度网络处理图像,使用TextCnn、FastText、LSTM等深度网络处理文本。为了加强网络对微观组织图像的“理解”,引入了注意力机制,使神经网络可以根据问题文本提取出图像的显著性区域,进而得到准确答案。为了验证方法的有效性,我们根据微观组织图像的特点建立了图像问答数据集,大量对比实验结果验证了所提出方法的有效性。(2)提出了基于图像语义信息的微观组织图像问答方法。该方法在上述微观组织图像问答方法基础上,使用MaskRcnn深度实例分割网络,实现了关于微观组织量化的问答功能。在该方法中,我们根据铝合金微观组织图像自身特点,对损失函数进行了改进,并且进行了对比实验,获得了良好的实例分割效果。为了验证方法的有效性,我们建立了微观组织图像实例分割数据集以及图像问答数据集。大量实验结果表明所提出方法能够准确地回答给定的各种问题。(3)设计并实现了铝合金微观组织图像问答系统。该系统主要集成了图像标注、图像预处理、图像分割、图像问答等功能模块。为了方便用户使用,我们加入了智能语音识别功能模块,实现了语音输入问题和语音播报答案功能。该系统能够为铝合金微观组织分析工作提供帮助,辅助专业人员提高微观组织分析效率。更多还原

【Abstract】 Aluminum alloy is one of the most widely used nonferrous metal materials in industrial production.Because of its good material properties,aluminum alloy is widely used in automobile,aerospace,navigation and railway transportation.The properties of aluminum alloy mainly depend on its microstructure,so analyzing the properties of aluminum alloy by microstructure image is one of the common methods used by experts and scholars.In this paper,the aluminum alloy microstructure image is taken as the research object,combined with deep learning,image processing,speech recognition and natural language processing methods,the aluminum alloy microstructure image question answering system is realized,the specific work is as follows:(1)A question answering method based on visual features and text information fusion is proposed.Firstly,the feature extraction network is used to extract the features of microstructure image and question text,then the feature fusion method is used to fuse the image features and text features,and finally the answer to the question is obtained through the deep classification network.In this method,we use ResNet deep network to process image and TextCnn,FastText,LSTM deep network to process text.In order to enhance the network’s "understanding" of the microstructure image,the attention mechanism is introduced,so that the neural network can extract the salient region of the image according to the question text,and then get the accurate answer.In order to verify the effectiveness of the proposed method,we set up an image Q&A data set according to the characteristics of microstructure images.A large number of comparative experimental results verify the effectiveness of the proposed method.(2)A method of question and answer for micro organization image based on image semantic information is proposed.Based on the above question answering method of microstructure image,this method uses MaskRcnn deep instance segmentation network to realize the question answering function of microstructure quantification.In this method,according to the characteristics of aluminum alloy microstructure image,we improved the loss function,and carried out a comparative experiment,and obtained a good example segmentation effect.In order to verify the effectiveness of the method,we set up the segmentation data set of microstructure image instance and the image question answering data set.A large number of experimental results show that the proposed method can answer all kinds of given questions accurately.(3)A question answering system of aluminum alloy microstructure image is designed and implemented.The system mainly integrates image annotation,image preprocessing,image segmentation,image question answering and other functional modules.In order to facilitate the use of users,we add the intelligent speech recognition function module to realize the function of voice input questions and voice broadcast answers.The system can provide help for microstructure analysis of aluminum alloy and help professionals to improve the efficiency of microstructure analysis.更多还原