As a typical artificial intelligence system, a safe and comfortable control system is essential for self-driving vehicles to have the same level of driving ability as human drivers. This paper proposes a novel control system for autonomous driving vehicles based on mixture modules, which aims to ensure the accuracy of path tracking while meeting the requirements of safety and ride comfort. The mixture modules consist of a lateral controller to control the steering wheel angle of the vehicle for path tracking and a longitudinal controller to adjust the speed of the vehicle. We conducted a series of experiments on our simulation platform and real self-driving vehicles to test the proposed control system and compared it with the traditional methods which are widely used. The experimental results indicate that our control system can run effectively on real vehicles. It may accurately track the intended driving path and adjust the driving speed comfortably and smoothly, which demonstrates a high level of intelligence.