Adaptation in context-aware ubiquitous environments and adaptive systems is becoming more and more complex. Adaptations need to take into account information from a plethora of heterogeneous sensors, while the adaptation decisions often imply personalised aspects and individual preferences, which are likely to change over time. We present a novel concept for lifelong learning from sensor data streams for predictive user modelling that is applicable in scenarios where simpler mechanisms that rely on pre-trained general models fall short. With the LiLoLe-Framework, we pursue an approach that allows ubiquitous systems to continuously learn from their users and adapt the system at the same time through stream-based active learning. This Framework can guide the development of context-aware or adaptive systems in form of an overall architecture.