Aiming at the characteristics of large amount of information included by the catadioptric panoramic reconnaissance image and the useful information inside the image not conveniently observed by naked eyes, through researching the characteristics of the current moving targets detection methods, a differential block elimination moving target detection algorithm was presented. The key techniques of this algorithm included: first, determine the best segmentation threshold; the second is to determine the block width; the third is the judging method to determine which blocks can be eliminated and which blocks can be kept. Experimental results show that the algorithm can not only well detect the real moving targets, but also well prevent the false targets. The algorithm effect is obvious, but simulation time has no much increase.