FDFtNet: Facing Off Fake Images Using Fake Detection Fine-Tuning Network - ICT Systems Security and Privacy Protection Access content directly
Conference Papers Year : 2020

FDFtNet: Facing Off Fake Images Using Fake Detection Fine-Tuning Network

Hyeonseong Jeon
  • Function : Author
  • PersonId : 1117619
Youngoh Bang
  • Function : Author
  • PersonId : 1117620
Simon S. Woo
  • Function : Author
  • PersonId : 1117621


Creating fake images and videos such as “Deepfake” has become much easier these days due to the advancement in Generative Adversarial Networks (GANs). Moreover, recent research such as the few-shot learning can create highly realistic personalized fake images with only a few images. Therefore, the threat of Deepfake to be used for a variety of malicious intents such as propagating fake images and videos becomes prevalent. And detecting these machine-generated fake images has been more challenging than ever.In this work, we propose a light-weight robust fine-tuning neural network-based classifier architecture called Fake Detection Fine-tuning Network (FDFtNet), which is capable of detecting many of the new fake face image generation models, and can be easily combined with existing image classification networks and fine-tuned on a few datasets. In contrast to many existing methods, our approach aims to reuse popular pre-trained models with only a few images for fine-tuning to effectively detect fake images. The core of our approach is to introduce an image-based self-attention module called Fine-Tune Transformer that uses only the attention module and the down-sampling layer. This module is added to the pre-trained model and fine-tuned on a few data to search for new sets of feature space to detect fake images. We experiment with our FDFtNet on the GANs-based dataset (Progressive Growing GAN) and Deepfake-based dataset (Deepfake and Face2Face) with a small input image resolution of 64$$\times $$×64 that complicates detection. Our FDFtNet achieves an overall accuracy of 90.29% in detecting fake images generated from the GANs-based dataset, outperforming the state-of-the-art.
Fichier principal
Vignette du fichier
497034_1_En_28_Chapter.pdf (9.63 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03440846 , version 1 (22-11-2021)





Hyeonseong Jeon, Youngoh Bang, Simon S. Woo. FDFtNet: Facing Off Fake Images Using Fake Detection Fine-Tuning Network. 35th IFIP International Conference on ICT Systems Security and Privacy Protection (SEC), Sep 2020, Maribor, Slovenia. pp.416-430, ⟨10.1007/978-3-030-58201-2_28⟩. ⟨hal-03440846⟩
27 View
3 Download



Gmail Facebook Twitter LinkedIn More