Color models are widely used in image recognition because they represent significant information. On the other hand, texture analysis techniques have been extensively used for facial feature extraction. In this paper, we extract discriminative features related to facial attributes by utilizing different color models and texture analysis techniques. Specifically, we propose novel methods for texture analysis to improve classification performance of race and gender. The proposed methods for texture analysis are based on Local Binary Pattern and its derivatives. These texture analysis methods are evaluated for six color models (HSV, L*a*b*, RGB, YCbCr, YIQ and YUV) to investigate the effect of each color model. Further, we configured two novel color models which are suitable for gender and race classification of face images. We perform experiments on the FERET database. Experimental results show that the proposed approaches are effective for classification of gender and race.