As the current recognition methods have high misclassification rate for the defects on the surface of hot rolled steel strip,a new method of extracting image feature is proposed in view of wavelet packet decomposition.Through inputting the energy feature of the extracted image into BP neural network classifier,three defects of pit,inclusion and scar can be identified.Simulation results show that this method has high recognition rate,robust noise immunity and good scalability.