A quaternion model for describing color image is proposed in order to evaluate its quality. Local variance distribution of luminance layer is calculated. Color information is taken into account by using quaternion matrix. The description method is a combination of luminance layer and color information. The angle between the singular value feature vectors of the quaternion matrices corresponding to the reference image and the distorted image is used to measure the structural similarity of the two color images. When the reference image and distorted images are of unequal size it can also assess their quality. Results from experiments show that the proposed method is better consistent with the human visual characteristics than MSE, PSNR and MSSIM. The resized distorted images can also be assessed rationally by this method.