In this paper, we provide a matrix-analytic solution for randomized load balancing models (also known as supermarket models) with phase-type (PH) service times. Generalizing the service times to the phase-type distribution makes analysis of the supermarket models more difficult and challenging than that of the exponential service time case which has been extensively discussed in the literature. We describe the supermarket model as a system of differential vector equations, provide a doubly exponential solution to the fixed point of the system of differential vector equations, and analyze the exponential convergence of the current location of the supermarket model to its fixed point.