Focused on a single target tracking in clutter with two sensors, a dual-mode fusion tracking algorithm based on nearest neighbor extended Kalman filter within IR/MMW angle observations association was presented. After figuring out the suspected pairs of observations (SPO) according to their spatial distribution models of observations originated from the target and clutter, most clutter was eliminated. The remained SPO were fused to single observations in terms of observation noise variance. The fused observation which is nearest to the prediction was used to update the target track, which utilized an extended Kalman filter. The algorithm can be applied to track a single target in mediate clutter density for its simplified association logic and low computation load, which is suitable for engineering implementation. The simulation results reveal the validity of the algorithm.