Evaluating Synthetic Data Quality in Imbalanced Datasets: Traditional AI (SMOTE) vs Generative AI (CTGAN & Gaussian Copula):

In real time, most of the time we handle datasets that are unstructured and imbalanced. Various strategies are used to restructure the imbalanced datasets. In addition, the choice of the AI model is domain specific.

Two types of datasets were used for analysis. The first dataset belongs to the healthcare domain contains multimode data types. The second dataset belongs to the banking domain which contains continuous numeric data types.

In traditional AI, (Synthetic Minority Over-sampling Technique) SMOTE was used for analysis. Since tabular data were used, in Gen AI model, Conditional GAN(CTGAN) and Gaussian Copula were used for the analysis.

The quality of the synthetic data was evaluated:

1) Correlation Difference: Measures the extent to which the relationships between variables change compared to the original data.

· Low value → relationships are preserved well

· High value → dependency structure is distorted

2. KS Test: Kolmogorov–Smirnov (KS) test, typically used to compare whether two distributions (e.g., real vs. synthetic data, or train vs. test data) are statistically similar.

· KS statistic (0 to 1) → measures distance between distributions

· p-value → tells whether the difference is statistically significant.

SMOTE Vs CTGAN in Healthcare Domain:

A metabolic syndrome dataset sourced from Kaggle was used for analysis. The dataset contains imbalanced binary target data. To run the model effectively, synthetic data was generated to balance the dataset. Two different approaches were used for analysis.

Using both SMOTE and CTGAN, synthetic data were generated in the given dataset.

1) Correlation Difference:

SMOTE: 0.010, CTGAN: 0.041

· SMOTE maintains the local neighborhood structure very well.

· CTGAN introduces synthetic realism in samples but weakens inter-feature biological relationships.

2) KS Test:

· SMOTE achieved the lowest mean KS statistic (0.0535), indicating strong preservation of marginal feature distributions. However, CTGAN obtained a higher KS value (0.2666), reflecting greater deviation in individual feature distributions.

· These results suggest that while CTGAN offers generative flexibility, SMOTE remains more reliable for structured clinical tabular data under limited sample conditions.

SMOTE is more reliable for clinical predictive modelling in this dataset than CTGAN.

SMOTE Vs Gaussian Copula in Banking Domain:

Fraud credit card dataset sourced from Kaggle was used for analysis. The dataset holds imbalanced binary target data. To run the model effectively, synthetic data was generated to balance the dataset. Two different approaches were used for analysis.

In the given dataset, using both SMOTE and Gaussian Copula GAN, synthetic data was generated.

Since the size of the data is too large, synthetic dataset with Gen AI was generated using Gaussian Copula.

1) Correlation Difference:

SMOTE: 0.255,

Gaussian Copula: 0.0002

· Gaussian Copula preserves global feature dependency structure significantly better than SMOTE, as evidenced by a substantially lower average correlation difference (0.0022 vs 0.2557).

· However, SMOTE introduces local interpolation bias that disrupts multivariate relationships

2) KS-Test:

Gaussian Copula achieved a lower mean KS statistic (0.0993) compared to SMOTE (0.2259), indicating superior preservation of irrelevant feature distributions. However, SMOTE exhibited substantial distributional drift due to its interpolation-based sampling mechanism, which does not model global data distributions.

These findings highlight that generative probabilistic models are more suitable than interpolation-based methods for realistic synthetic data generation in fraud detection scenarios.

Handling Imbalance dataset- Tradition AI vs Gen AI