【作者】 杨海青；

【导师】 关东海；

【作者基本信息】 南京航空航天大学 ， 工程硕士（专业学位）， 2022， 硕士

【摘要】 存储固件的损坏大多是硬盘故障造成的,且硬盘每年的更换数量达到万亿级别,硬盘故障检测一直是个棘手的问题,提前预测其故障是应对防止数据丢失的关键因素。为提高硬盘故障检测的效率,过去数年间,许多ML方法被广泛用于硬盘故障检测,并取得不错的效果。然而这些方法通常因标记SMART数据不足导致所能提供的检测效果有限,模型泛化能力差;一些方法没有考虑到硬盘数据具有的时序特征,仅仅考虑了点异常;此外,单一模型通常有个峰值,不足以达到本研究所期望的检测效果。针对单一模型检测效果不佳的问题,本文提出了一种基于异构集成学习的硬盘故障预测方法。针对大量的无标记数据,使用无监督自编码器将其潜在空间的重构误差作为异常分数。对于SMART数据本身具有的时序特征,将第一个阶段的异常分数作为样本标签,训练长短期记忆网络模型,并预测RMSE;同时模型结合GAN来增强数据。最后,通过多组实验证明本方法在保证较高精确率的前提下能够提高硬盘故障检测的召回率。针对硬盘样本标记困难、无监督自编码器的训练没有标记样本的引导问题,本文提出了一种基于主动学习和注意力机制的半监督硬盘故障预测方法。采用不确定标记的主动学习方法对无标记样本打标签,从而有效解决标记难度大和成本高的问题;同时,进一步改进损失函数,一个超参数来衡量标记样本在损失函数中的权重,跟踪它对模型的影响;在主动学习的基础上,加入注意力机制来更精准的重构样本。最后,通过多组实验比对,发现所提方法具有更高的F1均衡分数,验证了其有效性和先进性。更多还原

【Abstract】 The damage of storage firmware is mostly caused by hard disk failure,and the number of hard disk replacement reaches trillion every year.Hard disk failure detection has always been a thorny problem.Predicting its failure in advance is the key factor to prevent data loss.In order to improve the efficiency of disk anomaly detection,in the past few years,many ML methods have been widely used in disk anomaly detection,and achieved good results.However,these methods usually provide limited detection effect and poor model generalization ability due to insufficient labeled SMART data.Some methods do not take into account the timing characteristics of hard disk data and only consider point failure.In addition,a single model usually has a peak,which is not enough to achieve the desired detection effect.To solve the problem of poor detection effect of single model,this paper proposes a disk failure prediction method based on heterogeneous ensemble learning.For a large number of unlabeled data,we use unsupervised autoencoders to take the reconstruction error of potential space as the anomaly fraction.In view of the timing characteristics of SMART data,the abnormal scores in the first stage are used as sample labels to train the Long-Short Term Memory(LSTM)and predict RMSE.The model is combined with GAN to enhance the data.Finally,several groups of experiments show that this method can improve the recall rate of hard disk fault detection on the premise of ensuring high accuracy.A semi-supervised disk failure prediction method based on active learning and attention mechanism is proposed to solve the problems of hard disk sample labeling difficulty,unsupervised autoencoder training process without labeled sample supervision and so on.The active learning method of uncertain labeling is adopted to label unlabeled samples so as to effectively solve the problem of high difficulty and high cost of labeling.At the same time,the loss function is further improved,and a super parameter is used to measure the weight of the labeled sample in the loss function and track its impact on the model.On the basis of active learning,attention mechanism is added to reconstruct samples more accurately.Finally,through multiple groups of experiments,the proposed method has higher F1 equilibrium scores,which proves its effectiveness and advancement.更多还原