DeiT Baseline: Fix Output Consistency + Reduce Overfitting Risk

[lukhsaankumar]

Issue Type

• Model: ML model bug, training issue, or architecture problem
• Data: Dataset issue, preprocessing bug, or data pipeline problem
• Web: Frontend bug or UI issue in the Next.js dashboard
• API: Backend API bug or FastAPI endpoint issue
• Research: Research question or experimental feature request
• Documentation: Documentation bug or improvement needed
• Bug: General bug fix needed
• Enhancement: New feature or improvement request

Description

DeiT runs are strong but have overfitting signs and some inconsistency risk (distilled tuple outputs + possible label/metric confusion). Improve correctness + stability without changing overall approach.

Deliverable

Update vit_transfer_baseline_deit.ipynb with:

• Label mapping sanity check (required)
  • print a few sample file paths + folder name + label
  • enforce standard: 0=real, 1=fake everywhere
  • ensure confusion matrix axis labels match mapping
• Use a real validation split
  • create train/val split from train (stratified)
  • keep test untouched for final-only
• Consistent handling of distilled outputs
  • if model returns (logits, dist_logits), combine explicitly:
    • logits = (logits + dist_logits)/2 (or pick one, but be consistent)
• Anti-overfitting knobs (pick 2–3)
  • stronger but reasonable aug: resized crop, blur, jpeg compression, random erasing
  • cosine LR schedule + warmup
  • early stopping on val AUROC or fake-F1
  • keep/confirm dropout + label smoothing

Additional Context