【作者】 王伟；

【导师】 尚赵伟；

【作者基本信息】 重庆大学 ， 计算机科学与技术， 2024， 硕士

【摘要】 数据增广是深度学习中自动扩充训练数据的有效技术。很多任务中获取到充足训练样本的难度很高。数据增广可用来增加训练数据的数量和多样性,缓解模型过拟合,提高模型泛化能力和性能,其研究价值较高。本文主要工作如下:(1)当训练数据和测试数据风格差异较大时,常见的数据增广方法无法有效地增强深度模型的泛化性能。针对该问题,我们改进了使用不确定性建模域偏移(简称DSU),并且提出一种新的图像数据增广方法,该方法由两个关键部分组成,即为DSU使用稳健统计学和控制方差系数(简称RCDSU)和特征数据增广(简称Feature DA)。RCDSU使用稳健统计学来计算特征统计量(均值和标准差),以削弱异常值的影响,使统计量接近真实值,提高深度学习模型的鲁棒性。RCDSU通过控制方差系数,使特征统计量在语义保持的情况下发生偏移,并增大偏移范围。Feature DA同样控制方差系数,生成语义不变的增广特征,增加增广特征的覆盖范围。RCDSU和Feature DA在特征空间中进行风格转换和内容转换,分别提高模型在风格和内容层面的泛化能力。在照片、艺术绘画、卡通和素描(简称PACS)多风格分类任务上,RCDSU加Feature DA实现了有竞争力的准确率。在PACS数据集中添加高斯噪声后,RCDSU加Feature DA对异常值表现出很强的鲁棒性。Feature DA在CIFAR-100图像分类任务上取得了优异的成绩。(2)人工收集大量数据通常成本较高。针对该问题,我们提出一种随机策略混合数据增广方法(简称RSMDA),通过自动生成新的样本来扩充小规模数据集。首先使用多模态大模型和图生文提示工程模板来提取图像语义文本数据,得到图像对应的文字描述,并使用一种基于提示工程的数据清洗方法(简称DTC)对其进行数据清洗,用来避免文本混合时的语义冲突。然后使用一种基于提示工程的文本混合策略,随机策略混合(简称RSM),将该清洗后的文字描述与另一张图像清洗后的文字描述进行混合,得到混合后的文字描述。接着再使用多模态大模型将混合后的文字描述进行文生图操作,得到增广后的图像。其中RSM由三种文本混合策略组成,包括并集混合,加权混合和前景背景交叉混合。RSM从这三种混合策略中等概率随机选择一种策略来对文本进行混合操作。实验部分使用CIFAR100-Subset和Caltech101数据集来进行增广操作,将我们的方法跟其它数据增广方法对比。在增广后的数据集上执行图像分类任务时,在我们的方法得到的增广数据集上训练出来的模型分类准确度更高。更多还原

【Abstract】 Data augmentation is an effective technique for automatically expanding training data in deep learning.It is very difficult to obtain sufficient training samples for many tasks.Data augmentation can be used to increase the quantity and diversity of training data,alleviate model overfitting,and improve model generalization ability and performance,which has high research value.The main work of this thesis is as follows:(1)When there is a large style difference between training data and testing data,common data augmentation methods cannot effectively enhance the generalization performance of the deep model.To solve this problem,we improve modeling Domain Shifts with Uncertainty(DSU)and propose a new image data augmentation method which consists of two key components,namely,using Robust statistics and controlling the Coefficient of variance for DSU(RCDSU)and Feature Data Augmentation(Feature DA).RCDSU calculates feature statistics(mean and standard deviation)with robust statistics to weaken the influence of outliers,making the statistics close to the real values and improving the robustness of deep learning models.By controlling the coefficient of variance,RCDSU makes the feature statistics shift with semantic preservation and increases shift range.Feature DA controls the coefficient of variance similarly to generate the augmented features with semantics unchanged and increase the coverage of augmented features.RCDSU and Feature DA are proposed to perform style transfer and content transfer in the feature space,and improve the generalization ability of the model at the style and content level respectively.On Photo,Art Painting,Cartoon,and Sketch(PACS)multi-style classification task,RCDSU plus Feature DA achieves competitive accuracy.After adding Gaussian noise to PACS dataset,RCDSU plus Feature DA shows strong robustness against outliers.Feature DA achieves excellent results on CIFAR-100 image classification task.RCDSU plus Feature DA can be applied as a novel semantic data augmentation method which is suitable for datasets with large style differences between training and testing data.(2)Manually collecting large amounts of data is usually costly.To address this problem,we propose a random strategy mixed data augmentation method(RSMDA)to automatically generate new samples to expand small-scale datasets.First,we use a multimodal large model and a picture-to-text prompt engineering template to extract image semantic text data and obtain the text description corresponding to the image.We then use a data cleaning method based on prompt engineering(DTC)to clean the data to avoid semantic conflicts when mixing text.Then,we use a text mixing strategy based on prompt engineering,random strategy mixing(RSM),to mix the cleaned text description with the cleaned text description of another image to obtain a mixed text description.Then,we use a multimodal large model to perform text-to-picture operations on the mixed text description to obtain an augmented image.RSM consists of three text mixing strategies,including union mixing,weighted mixing,and foreground-background cross mixing.RSM randomly selects one strategy from these three mixing strategies with medium probability to perform mixing operations on the text.In the experiment,we use the CIFAR100-Subset and Caltech101 datasets for augmentation operations and compare our method with other data augmentation methods.When performing image classification tasks on the augmented dataset,the model trained on the augmented dataset obtained by our method achieves higher classification accuracy.更多还原