In this work, we investigate a fast background elimination front-end of an automatic bacilli detection system. This background eliminating system consists of a feature descriptor followed by a linear-SVMs classifier. Four state-of-the-art feature extraction algorithms are analyzed and modified. Extensive experiments have been made on real sputum fluorescence images and the results reveal that 96.92% of the background content can be correctly removed from one image with an acceptable computational complexity.