The use of Open Source Software (OSS) components has become a viable alternative to Commercial Off-The-Shelf (COTS) components in product development. Since the quality of OSS products varies widely, both industry and the research community have reported several OSS evaluation methods that are tailored to the specific characteristics of OSS. We have performed a systematic identification of these methods, and present a comparison framework to compare these methods.