Edge detection is an old and hot topic in image processing, pattern recognition and computer vision. Numerous edge detection approaches have been proposed to gray images. It is difficult to extend these approaches to color image edge detection. A novel edge detection method based on mathematical morphology for color images is proposed in this paper. The proposed approach firstly compute vector gradient based on morphological gradient operators, and then compute the optimal gradient according to structure elements with different size. Finally, we use a threshold to binary the gradient images and then obtain the edge images. Experimental results show that the proposed approach has advantages of suppressing noise and preserving edge details and it is not sensitive to noise pixel. The finally edge images via the proposed method have high PSNR and NC compared with the traditional approaches.