As the identification of surface defects is very important in fruit automatic detection, a new method for the detection of fruit surface defects based on fractal dimension is suggested. In this method, fruit image was collected using computer vision system. The fractal dimension of fruit image was calculated by an improved ‘box dimension’. The fruit fractal dimension reflects the three dimensional characteristics of the fruit as well as information of the fruit surface. The detection of surface defects of fruits was performed according to a given threshold of fruit image fractal dimension. The results on Fuji apple fruits showed that the improved ‘box dimension’ method was effective and reliable in the detection of fruit defects for its improvement in the accuracy in the calculation of the fractal dimension.