In order to validate accuracy of modeling and simulation of infrared imaging sensor system, a method was proposed based on full reference image quality assessment. Actual infrared image and the simulated image were compared with pixel by pixel comparison method and statistical analysis, and the assessment was obtained based on histogram statistics, structural similarity and fidelity indicators. Experimental results indicate that the comprehensive evaluation index is consistent with histogram data and frequency spectrum, and it is accord with the subjective evaluation results. The evaluation method can effectively validate accuracy of modeling and simulation of infrared imaging sensor system.