Although several physical implementations of the smart factory have been introduced, a viable implementation or platform has not been proposed to make human workers perceive smart factory services more effectively and naturally with respect to task assistance. In this paper, we propose an AR/VR-based Live Manual, a new approach to provide workers with user-centric smart manufacturing services in IoT-enabled smart factory testbeds. The proposed live manual can make both on-site and remote workers do their tasks more effectively by utilizing and synchronizing AR with VR, which can help them to make better decisions. The synchronized dual view between AR and VR can help workers to understand the situation more easily and clearly depending on the worker’s contexts and manufacturing situations.