Aiming at the target recognition and tracking within video image, a combined two step bits operation algorithm which based on an improved binary robust invariant scalable keypoints (BRISK) algorithm was proposed. The improvements include descriptor building and feature matching methods. During descriptor building, a point pair selection strategy for reducing relativity and scope-rotation two steps feature descriptor were put forward. In feature matching stage, the algorithm which combined'SHIFT''XOR' binary operations was used. To further improve feature matching speed, an accelerated method with partial matching and Hamming weight threshold checking was introduced. The result of experiments demonstrate that the proposed algorithm achieves the average target tracking speed of 80 frames per-second, which is faster than other corner feature algorithms. Therefore, it could meet the demand of real-time target recognition and tracking from video image more effectively.