Sparse decomposition has been widely used in numerous applications, such as image processing, pattern recognition, remote sensing and computational biology. Despite plenty of theoretical developments have been proposed, developing, implementing and analyzing novel fast sparse approximation algorithm is still an open problem. In this paper, a new pursuit algorithm Double Least Squares Pursuit (DLSP) is proposed for sparse decomposition. In this algorithm, the support of the solution is obtained by sorting the coefficients which are calculated by the first Least-Squares, and then the non-zero values over this support are detected by the second Least-Squares. The results of numerical experiment demonstrate the effectiveness of the proposed method, which is with less time complexity, more simple form, and gives close or even better performance compared to the classical Orthogonal Matching Pursuit (OMP) method.