Smart city is one of the most known Internet of Things (IoT) applications. The smart city services improve user’s daily lives. However, security and privacy issues are slowing down their adoption. In addition, the characteristics of IoT devices, applications and users make security implementation of the considered applications a challenging task. To address these issues, we present, in this paper, a new context-aware security and privacy architecture for the IoT. Thanks to the “as a service” approach, this new architecture will be user-centric. It will also support known context-aware security issues: dynamicity, flexibility. In addition, it will address mobility, customization of security and privacy services, and support for generic IoT applications, particularly for smart city. To do so, a knowledge plane allowing effective management of context-awareness is proposed. A security and privacy plane allowing better implementation of context-aware security and privacy mechanisms is also proposed. This will be done through dynamic composition of context-based micro services. The role of the different components of these two planes are also described.