Acknowledging the current digitalizing of buildings and their existence as interactive objects, this article sets out to consolidate Human-Building Interaction (HBI) as a new research domain within HCI. It exposes fundamental characteristics of HBI such as user immersion in the “machine” and extensive space and time scales, and proposes an operational definition of the domain. Building upon a comprehensive survey of relevant cross-disciplinary research, HBI is characterized in terms of dimensions representing the interaction space and modalities that can be invoked to enhance interactions. Specific methodological challenges are discussed, and illustrative research projects are presented demonstrating the relevance of the domain. New directions for future research are proposed, pointing out the domain’s potentially significant impact on society.