Testing and verification is still an open issue on the way to fully automated driving. Simulations can help to reduce the required testing efforts, however, classical simulators based on physical models and heuristics, such as the intelligent driver model (IDM), show limited model accuracy on a microscopic scenario level. In turn, learning-based driver models are often capable to predict human driver’s behavior accurately, but are difficult to tailor such that they follow an intended scenario description. In this work, we propose a software-in-the-loop framework to combine a learned model with a rule-based logic layer and a kinematic vehicle model of a classical traffic simulator. Thus, the merits of both, classical simulators and learning-based models are exploited. We demonstrate with a case study of evaluating a motion planning scheme that the simulator fits well with the needs of testing such methods. Furthermore, we show by experiments with real-world traffic data from a traffic surveillance system that the proposed simulator yields realistic behavior of the simulated road users.