Sharing Semantic Web datasets provided by different publishers in a decentralized environment calls for efficient support from distributed computing technologies. Moreover, we argue that the highly dynamic ad-hoc settings that would be pervasive for Semantic Web data sharing among personal users in the future pose even more demanding challenges for enabling technologies. We propose an architecture that is based upon the peer-to-peer (P2P) paradigm for ad-hoc Semantic Web data sharing and identify the key technologies that underpin the implementation of the architecture. We anticipate that our (current and future) work will offer powerful support for sharing of Semantic Web data in a decentralized manner and becomes an indispensable and complementary approach to making the Semantic Web a reality.