It usually need different ways to process different objects in the manufacturing, Therefore, firstly we need to distinguish the categories of objects to be processed, then the machine will know how to deal with the objects. In order to automatically recognize the category of the irregular object, this paper extracted the improved Hu's moments of each object as the feature by the way of processing images of the working platform that the irregular objects are putting on. This paper adopts the variable step BP neural network with adaptive momentum factor as the classifier. The experiment shows that this method can effectively distinguish different irregular objects, and during the training of the neural network, it has faster convergence speed and better approximation compared with the traditional BP neural network