To enhance the efficiency of aerodynamic shape optimization based on deep reinforcement learning (DRL), a DRL-based aerodynamic optimization framework incorporating surrogate models is proposed. Deep reinforcement learning suffers from low sample efficiency, in aerodynamic optimization, it requires extensive use of flow field numerical simulations to obtain sample data. However, high-precision flow field numerical simulations come with a substantial computational cost in terms of time. By introducing surrogate models, it can rapidly generate sample data, reduce the dependence on high-precision flow field numerical simulations, and improve the efficiency of aerodynamic shape optimization. Taking the transonic airfoil drag reduction optimization design and inverse design as examples, the effectiveness of the DRL-based aerodynamic optimization framework incorporating surrogate models is analyzed. In transonic airfoil drag reduction optimization design, both DRL and DRL based on surrogate models achieve optimal solutions, eliminate the shock waves. However, the use of surrogate models reduces the number of high-precision flow field numerical simulations by 67%, significantly improving optimization efficiency. In airfoil inverse design, the pressure distribution and shape of the inversely designed airfoil obtained through DRL based on surrogate models show good agreement with the target airfoil. Both examples illustrate the effectiveness of the proposed DRL-based aerodynamic optimization framework incorporating surrogate models.