With the fruitful achievement of deep learning in computer vision, transfer learning is becoming a profoundly developing and potential research field to overcome its shortcomings because transfer learning can reach desired goal with fewer training data and less running time. Tree identification is meaningful and important for many fields, such as phytology, ecology, traditional Chinese medicine and so on. In this paper, transfer learning is employed in the proposed tree image recognition method which is prepared by the per-trained networks: Inception-V3 and Inception-ResNet-v2 networks; and 22 ornamental tree species with more than 22,000 images from ImageNet database are collected for retraining. After 10,000 rounds of training for these networks, the cross-accuracy of Inception-V3 and Inception-ResNet-v2 networks is reach 87.8 and 91.2%, respectively. To test the performance of the proposed method, it is compared with conventional neural network and convolutional neural network (CNN) using the same data. Our work mainly includes to analyze the recognition effect of "bad image" for the proposed inception networks, such as easier confusing or bad shooting images and so on; besides, we also collect another easily confused tree images for testing the performance of the proposed method. The experiments show that the proposed method is effective than the common methods.