Most bearing-only localization methods neglect uncertainty in observer position. Recently, Vaghefi extends the maximum likelihood bearing-only localization method based on uncertainties in observer position, it raises model's accuracy effectively, but Vaghefi's new model doesn't consider the geometrical constrain between observers and object. In this paper, a geometrically constrained optimization model was proposed based on Vaghefi's model with considering the observer position error, sequential propagation (SQP) optimization method was used to correlate the object's position, Hessian matrix of SQP was simplified, and the parameters' analytical equations were shown. Because of consideration of geometrical constrain and observer position's error at the same time, simulation results show the new model is more accurate than the maximum likelihood method and Vaghefi's model.