Ear biometrics, which involves the identification of a person from an ear image, is challenging under unconstrained image capturing scenarios. Studies in Ear biometrics reported that the Convolutional Neural Network is a better alternative to classical machine learning with handcrafted features. Two major concerns in CNN are the requirement of enormous computing resources and large datasets for training. The pretrained network concept helps to use CNN with smaller datasets and is less demanding on hardware. In this paper, three pre-trained CNN models, AlexNet, VGG16, and ResNet50 are used for ear recognition. The fully connected classification layers of the nets are trained with AWE, an unconstrained ear dataset. Alternatively, the CNN layers’ output (the CNN features) are extracted, and an SVM classification model is built. To improve the classification accuracy, the training dataset size is increased through data augmentation. Data augmentation improved the classification accuracy drastically. The results show that ResNet50, with the fully connected classification layer, results in higher accuracy.