Review Jurnal — JRC Deepfake Detection via joint Reconstruction and Classification

Title : JRC: Deepfake Detection via Joint Reconstruction and Classification Authors : Bosheng Yan, Chang-Tsun Li, Xuequan Lu Year : 2024 Published in : Neurocomputing, Volume 598, Article №127862 DOI : 10.1016/j.neucom.2024.127862

Review By : Dedy Wibowo Academic Advisor (DR Arya Adhyaksa Waskita S.SI.,M.SI)

Background and Motivation

Deepfake technology, which uses AI to manipulate facial images and videos, has become a serious threat to the authenticity of digital content. Most existing deepfake detection methods perform well on training data (intra-dataset) but struggle when applied to new, unseen data (inter-dataset). This limitation arises from their dependence on global features and binary supervision. To address this, the authors propose a novel approach called JRC (Joint Reconstruction and Classification), which aims to enhance model generalization in deep fake detection tasks.

Key Contributions and Innovations

Two-Branch Convolutional Autoencoder (CAE) architecture is a neural network design that processes input data through two parallel convolutional pathways (branches) before merging their outputs for reconstruction. This structure is particularly effective for tasks requiring the integration of different types of features, such as spatial and spectral information in images.

The proposed JRC model utilizes a dual-branch architecture:

One branch reconstructs the original face from a potentially manipulated image and The other branch classifies the input as real or fake.

• Shared encoder branch — compresses input (usually face images or frames) into a latent representation.
• Reconstruction branch — decodes from that latent space to reconstruct the original input.
• Classification branch — uses the same latent representation to predict whether the input is real or fake.

This joint design enables shared learning between reconstruction and classification tasks, enhancing detection accuracy.

Methodology

The model is based on a Convolutional AutoEncoder with two main branches:

• Reconstruction Branch: Attempts to reconstruct the original face from the deepfake input.
• Classification Branch: Predicts whether the image is authentic or manipulated.

The training process is end-to-end, using a combined loss function that balances reconstruction and classification objectives.

Experimental Results

• JRC outperforms other state-of-the-art methods on three widely-used datasets.
• The model demonstrates excellent generalization capabilities, especially in cross-dataset evaluations.
• Joint learning enables better separation of real vs. fake features even in challenging scenarios.

Conclusion

This article introduces an innovative and effective method for deep fake detection through Joint Reconstruction and Classification (JRC). By combining two synergistic tasks within a single model, the approach achieves superior performance and generalization. JRC represents a significant advancement in combating the growing threat of deepfakes and offers a promising foundation for future research and real-world applications.