The state-of-the-art trackers using deep learning technology have no special strategy to capture the geometric deformation of the target. Based on that the affine manifold can better capture the target shape change and that the higher level of Convolutional Neural Network (CNN) can better describe semantic information of objects, we propose a new tracking algorithm combining affine transformation with convolutional features to track targets with dramatic deformation. First, the affine transformation is applied to predict possible locations of a target, then a correlative filter is designed to compute the appearance confidence score for determining the final target location. Furthermore, a standard discriminative correlation filter is used to develop the effect of convolutional features, which is more efficient than other methods used for CNN Networks. Comprehensive experiments demonstrate the outstanding performance of our tracking algorithm compared to the state-of-the-art techniques in the public benchmarks.