In this paper, we present a study on emotion-based music information retrieval using lyrics information. Listeners want to search the lyrics of music suitable for his/her emotion (impression of music), by using an information system from music libraries. As a solution of listeners’ needs, we have designed a system that retrieve the lyrics of music based on the emotion (or the impression) suitable for a listener’s feelings that the listener has selected, from 9 emotions and 9 impressions. We select the words, i.e. verb and adjective, from the bridge part of the lyrics of music that express emotion in lyrics by using natural language processing. We summarize the words into the representative words by using a dictionary of synonyms. We make a model that estimates a listener’s 9 emotion/impression of the representative words by using a machine learning method. And listeners want to understand why the recommended music by a system is suitable for his/her emotion/impression. Therefore, we select the representative words most related to a listeners emotion/impression and we use the selected words as the explanation of reason to a listener. We have made each model of emotion and impression for 9 subjects and have evaluated the accuracy of the model. We also have investigated the selected representative words related to emotion/impression.