DAVID-Net: 160-Run Experimental Archive and Benchmark Telemetry

GitHub Core Model Datasets Live Demo

Official experimental archive, multi-seed checkpoints, training event streams, and evaluation telemetry for DAVID-Net (Decoupled Audio-Visual Forensic Network).

This repository contains the persistent, uncompressed experimental record for the entire 160-run / 5-seed empirical campaign (135 core campaign runs across 27 experimental configurations + 25 auxiliary baseline, scaling, and recovery runs, seeds $\mathcal{S} \in {7, 42, 123, 456, 2024}$).

DAVID-Net Pipeline Architecture


πŸ“Œ Experimental Campaign Overview

The empirical validation of DAVID-Net reflects a comprehensive matrix of 160 multi-seed training runs partitioned across five primary experimental stages:

  1. Stage-0: Quadrant-Aware Contrastive Pretraining (QACP) (30 runs):
    • 6 self-supervised pretraining variants $\times$ 5 random seeds (seeds 7, 42, 123, 456, 2024).
    • Generates proxy synthetic media from pristine recordings using self-blended video and vocoder copy-synthesis speech.
  2. Stage-1: In-Domain Baselines & Proposed Architecture (35 runs):
    • 7 model architectures $\times$ 5 random seeds: Video Probe, Audio Probe, AASIST, EfficientNet-B4, Late Fusion, DAVID-Net Phase A, and DAVID-Net End-to-End Phase B.
    • Evaluated under strict identity-disjoint FakeAVCeleb partitioning (0.0% train/test subject identity overlap).
  3. Systematic Component Ablation Study (55 runs):
    • 11 ablation variants $\times$ 5 random seeds: isolating cross-modal synchronization (no_sync), subspace disentanglement (no_disentangle), localization heads (no_loc), modality dropout (no_moddrop), single-task loss (single_task), pretraining (no_qacp), and sub-component pretraining objectives.
  4. Leave-One-Generator-Out (LOGO) Study (15 runs):
    • 3 withheld generator families (logo_wav2lip, logo_fsgan, logo_faceswap) $\times$ 5 random seeds to measure zero-shot cross-generator generalization.
  5. Auxiliary Baseline & Scaling Campaign (25 runs):
    • Additional baseline explorations (AVFF, AVoiD-DF, Cross-Attention) and scaled data sweeps across all 5 seeds.

πŸ”¬ In-Domain Benchmark Summary ($N=5$ Seeds)

All 25 evaluated model configurations averaged over five independent seeds ($\mathcal{S} \in {7, 42, 123, 456, 2024}$):

Model / Configuration Seeds Video AUC Audio AUC Clip AUC
aasist 5 β€” 1.000 Β± 0.000 0.748 Β± 0.004
audio-probe 5 β€” 0.999 Β± 0.001 0.732 Β± 0.001
avff 5 0.978 Β± 0.003 1.000 Β± 0.000 0.981 Β± 0.003
avoid-df 5 0.922 Β± 0.003 1.000 Β± 0.000 0.938 Β± 0.004
cross-attn 5 0.938 Β± 0.007 0.999 Β± 0.001 0.939 Β± 0.008
davidnet 5 0.979 Β± 0.003 0.999 Β± 0.002 0.976 Β± 0.006
davidnet-compose_quad 5 0.979 Β± 0.004 1.000 Β± 0.000 0.978 Β± 0.003
davidnet-e2e 5 0.976 Β± 0.005 1.000 Β± 0.000 0.978 Β± 0.005
davidnet-legacy 5 0.985 Β± 0.004 1.000 Β± 0.000 0.985 Β± 0.002
davidnet-logo_faceswap 5 0.912 Β± 0.006 1.000 Β± 0.000 0.903 Β± 0.003
davidnet-logo_fsgan 5 0.934 Β± 0.004 1.000 Β± 0.000 0.932 Β± 0.007
davidnet-logo_wav2lip 5 0.425 Β± 0.018 0.994 Β± 0.010 0.919 Β± 0.014
davidnet-no_disentangle 5 0.979 Β± 0.002 1.000 Β± 0.000 0.979 Β± 0.001
davidnet-no_loc 5 0.980 Β± 0.004 1.000 Β± 0.000 0.981 Β± 0.004
davidnet-no_moddrop 5 0.980 Β± 0.003 1.000 Β± 0.000 0.981 Β± 0.004
davidnet-no_qacp 5 0.981 Β± 0.002 1.000 Β± 0.000 0.983 Β± 0.003
davidnet-no_sync 5 0.973 Β± 0.005 1.000 Β± 0.000 0.975 Β± 0.011
davidnet-qacp_no_copysynth 5 0.980 Β± 0.003 0.999 Β± 0.001 0.979 Β± 0.006
davidnet-qacp_no_mismatch 5 0.976 Β± 0.003 1.000 Β± 0.000 0.977 Β± 0.003
davidnet-qacp_no_selfblend 5 0.977 Β± 0.006 1.000 Β± 0.001 0.978 Β± 0.006
davidnet-qacp_scaled 5 0.979 Β± 0.002 1.000 Β± 0.000 0.978 Β± 0.006
davidnet-single_task 5 0.978 Β± 0.003 1.000 Β± 0.000 0.978 Β± 0.005
effnet-b4 5 0.996 Β± 0.001 β€” 0.926 Β± 0.002
late-fusion 5 0.914 Β± 0.020 1.000 Β± 0.000 0.918 Β± 0.015
video-probe 5 0.896 Β± 0.008 β€” 0.830 Β± 0.007

Key Diagnostic Findings

  • Decoupled Attribution: Multi-task learning does not handicap individual modality branches: DAVID-Net achieves $0.979 \pm 0.003$ Video AUC and $0.999 \pm 0.002$ Audio AUC, while reaching $0.976 \pm 0.006$ Clip AUC, outperforming monolithic Late Fusion ($0.918 \pm 0.015$).
  • Cross-Generator Floor (LOGO): Maintains $\ge 90.3%$ Clip AUC across all withheld generator families under leave-one-generator-out evaluation.
  • Empirical Calibration: Attains low Expected Calibration Error (ECE, 15 bins) of $0.032$ (visual) and $0.024$ (acoustic) on the test split.
  • Zero Cross-Stream Confusion: Analysis of the normalized confusion matrix reveals 0.0% confusion between opposite single-stream manipulations (RVFA vs. FVRA).

πŸ“ Repository Organization & File Hierarchy

The repository is structured strictly according to publication-grade open-science standards:

MIHMahmudEli/davidnet-experiments/
β”œβ”€β”€ data/
β”‚   β”œβ”€β”€ SPLITS_SHA256.json      # Cryptographic SHA-256 hashes of all partitions
β”‚   β”œβ”€β”€ feature_set.json        # Preprocessing & feature extraction parameters
β”‚   β”œβ”€β”€ manifests/              # Manifest JSONL indices for 7 benchmark datasets
β”‚   └── splits/                 # Identity-disjoint strict & LOGO train/val/test splits
β”œβ”€β”€ experiments/
β”‚   └── EXP_{ID}_{NAME}_s{SEED}/
β”‚       β”œβ”€β”€ best_model/
β”‚       β”‚   β”œβ”€β”€ model.safetensors   # Pinned model weights for this seed (SafeTensors)
β”‚       β”‚   β”œβ”€β”€ config.json         # Architecture parameters
β”‚       β”‚   └── meta.json           # Validation metrics & checkpoint metadata
β”‚       β”œβ”€β”€ configs/
β”‚       β”‚   β”œβ”€β”€ config.json         # Full hyperparameter configuration
β”‚       β”‚   └── environment.json    # Hardware, OS, and PyTorch runtime telemetry
β”‚       β”œβ”€β”€ metrics/
β”‚       β”‚   β”œβ”€β”€ training_history.csv  # Per-epoch loss, learning rates, and validation metrics
β”‚       β”‚   β”œβ”€β”€ test_metrics.json     # Final test evaluation metrics
β”‚       β”‚   β”œβ”€β”€ confusion_matrix.csv  # 4-quadrant assignment counts
β”‚       β”‚   └── classification_report.csv
β”‚       └── figures/                # Per-run ROC, PR, calibration, and distribution plots
β”œβ”€β”€ peer_review_audit/
β”‚   β”œβ”€β”€ profiling/              # Nvidia T4 runtime latency, memory, and FLOPs profiling
β”‚   └── stats/                  # DeLong test $z$-scores, $p$-values, and bootstrap 95% CIs
β”œβ”€β”€ registry/
β”‚   └── experiments.json        # Master campaign tracking manifest (hashes, epochs, status)
β”œβ”€β”€ assets/
β”‚   └── fig_davidnet.png        # High-resolution architectural overview
β”œβ”€β”€ .gitattributes             # Git LFS configuration
└── README.md                  # This Model Card & benchmark leaderboard

πŸ’» Programmatic Usage: Loading Run Artifacts & Weights

You can inspect and load the weights, training logs, or configs of any run programmatically using huggingface_hub and safetensors:

import pandas as pd
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

repo_id = "MIHMahmudEli/davidnet-experiments"

# 1. Download training history for EXP_011 (DAVID-Net Phase A, Seed 42)
history_path = hf_hub_download(
    repo_id=repo_id,
    filename="experiments/EXP_011_davidnet_s42/metrics/training_history.csv"
)
df_history = pd.read_csv(history_path)
print("Training Epochs:", len(df_history))

# 2. Download checkpoint weights (SafeTensors)
weights_path = hf_hub_download(
    repo_id=repo_id,
    filename="experiments/EXP_011_davidnet_s42/best_model/model.safetensors"
)
state_dict = load_file(weights_path)
print(f"Loaded {len(state_dict)} tensor keys successfully.")

πŸ“‹ Complete 160-Run Campaign Registry

ID Configuration Seed Status Test Clip AUC
EXP_001 video-probe 42 completed 0.8373
EXP_002 audio-probe 42 completed 0.7294
EXP_003 late-fusion 42 completed 0.9307
EXP_004 aasist 42 completed 0.7471
EXP_005 effnet-b4 42 completed 0.9232
EXP_006 qacp 42 completed β€”
EXP_007 qacp-no_sync 42 completed β€”
EXP_008 qacp-no_mismatch 42 completed β€”
EXP_009 qacp-no_copysynth 42 completed β€”
EXP_010 qacp-no_selfblend 42 completed β€”
EXP_011 davidnet 42 completed 0.9725
EXP_012 davidnet-no_qacp 42 completed 0.9857
EXP_013 davidnet-no_sync 42 completed 0.9810
EXP_014 davidnet-no_disentangle 42 completed 0.9802
EXP_015 davidnet-no_loc 42 completed 0.9799
EXP_016 davidnet-single_task 42 completed 0.9747
EXP_017 davidnet-no_moddrop 42 completed 0.9762
EXP_018 davidnet-compose_quad 42 completed 0.9766
EXP_019 davidnet-qacp_no_mismatch 42 completed 0.9796
EXP_020 davidnet-qacp_no_copysynth 42 completed 0.9701
EXP_021 davidnet-qacp_no_selfblend 42 completed 0.9708
EXP_022 davidnet-logo_wav2lip 42 completed 0.9274
EXP_023 davidnet-logo_fsgan 42 completed 0.9350
EXP_024 davidnet-logo_faceswap 42 completed 0.9012
EXP_025 qacp-legacy 42 completed β€”
EXP_026 davidnet-legacy 42 completed 0.9826
EXP_027 davidnet-e2e 42 completed 0.9793
EXP_028 video-probe 123 completed 0.8349
EXP_029 audio-probe 123 completed 0.7336
EXP_030 late-fusion 123 completed 0.9273
EXP_031 aasist 123 completed 0.7535
EXP_032 effnet-b4 123 completed 0.9259
EXP_033 qacp 123 completed β€”
EXP_034 qacp-no_sync 123 completed β€”
EXP_035 qacp-no_mismatch 123 completed β€”
EXP_036 qacp-no_copysynth 123 completed β€”
EXP_037 qacp-no_selfblend 123 completed β€”
EXP_038 davidnet 123 completed 0.9829
EXP_039 davidnet-no_qacp 123 completed 0.9797
EXP_040 davidnet-no_sync 123 completed 0.9578
EXP_041 davidnet-no_disentangle 123 completed 0.9799
EXP_042 davidnet-no_loc 123 completed 0.9768
EXP_043 davidnet-single_task 123 completed 0.9779
EXP_044 davidnet-no_moddrop 123 completed 0.9803
EXP_045 davidnet-compose_quad 123 completed 0.9802
EXP_046 davidnet-qacp_no_mismatch 123 completed 0.9739
EXP_047 davidnet-qacp_no_copysynth 123 completed 0.9752
EXP_048 davidnet-qacp_no_selfblend 123 completed 0.9770
EXP_049 davidnet-logo_wav2lip 123 completed 0.8979
EXP_050 davidnet-logo_fsgan 123 completed 0.9330
EXP_051 davidnet-logo_faceswap 123 completed 0.9083
EXP_052 qacp-legacy 123 completed β€”
EXP_053 davidnet-legacy 123 completed 0.9865
EXP_054 davidnet-e2e 123 completed 0.9723
EXP_055 video-probe 456 completed 0.8204
EXP_056 audio-probe 456 completed 0.7319
EXP_057 late-fusion 456 completed 0.9004
EXP_058 aasist 456 completed 0.7447
EXP_059 effnet-b4 456 completed 0.9274
EXP_060 qacp 456 completed β€”
EXP_061 qacp-no_sync 456 completed β€”
EXP_062 qacp-no_mismatch 456 completed β€”
EXP_063 qacp-no_copysynth 456 completed β€”
EXP_064 qacp-no_selfblend 456 completed β€”
EXP_065 davidnet 456 completed 0.9726
EXP_066 davidnet-no_qacp 456 completed 0.9828
EXP_067 davidnet-no_sync 456 completed 0.9814
EXP_068 davidnet-no_disentangle 456 completed 0.9776
EXP_069 davidnet-no_loc 456 completed 0.9858
EXP_070 davidnet-single_task 456 completed 0.9819
EXP_071 davidnet-no_moddrop 456 completed 0.9855
EXP_072 davidnet-compose_quad 456 completed 0.9828
EXP_073 davidnet-qacp_no_mismatch 456 completed 0.9802
EXP_074 davidnet-qacp_no_copysynth 456 completed 0.9833
EXP_075 davidnet-qacp_no_selfblend 456 completed 0.9840
EXP_076 davidnet-logo_wav2lip 456 completed 0.9328
EXP_077 davidnet-logo_fsgan 456 completed 0.9316
EXP_078 davidnet-logo_faceswap 456 completed 0.9054
EXP_079 qacp-legacy 456 completed β€”
EXP_080 davidnet-legacy 456 completed 0.9877
EXP_081 davidnet-e2e 456 completed 0.9748
EXP_082 audio-probe 7 completed 0.7315
EXP_083 qacp-no_selfblend 7 completed β€”
EXP_084 late-fusion 2024 completed 0.9032
EXP_085 qacp-no_copysynth 2024 completed β€”
EXP_086 davidnet-no_qacp 2024 completed 0.9806
EXP_087 qacp-no_mismatch 2024 completed β€”
EXP_088 qacp-no_copysynth 7 completed β€”
EXP_089 qacp 2024 completed β€”
EXP_090 video-probe 2024 completed 0.8260
EXP_091 qacp-no_selfblend 2024 completed β€”
EXP_092 video-probe 7 completed 0.8334
EXP_093 late-fusion 7 completed 0.9302
EXP_094 aasist 2024 completed 0.7455
EXP_095 qacp-no_sync 2024 completed β€”
EXP_096 qacp-legacy 2024 completed β€”
EXP_097 davidnet-qacp_no_mismatch 2024 completed 0.9761
EXP_098 davidnet-logo_wav2lip 2024 completed 0.9096
EXP_099 davidnet-qacp_no_selfblend 7 completed 0.9742
EXP_100 qacp 7 completed β€”
EXP_101 qacp-no_sync 7 completed β€”
EXP_102 qacp-no_mismatch 7 completed β€”
EXP_103 qacp-legacy 7 completed β€”
EXP_104 effnet-b4 2024 completed 0.9279
EXP_105 davidnet-legacy 2024 completed 0.9831
EXP_106 davidnet-compose_quad 7 completed 0.9740
EXP_107 davidnet-logo_fsgan 7 completed 0.9401
EXP_108 davidnet-no_moddrop 7 completed 0.9782
EXP_109 audio-probe 2024 completed 0.7313
EXP_110 davidnet-logo_wav2lip 7 completed 0.9256
EXP_111 davidnet-qacp_no_mismatch 7 completed 0.9750
EXP_112 davidnet-no_loc 7 completed 0.9804
EXP_113 davidnet-qacp_no_selfblend 2024 completed 0.9837
EXP_114 davidnet 2024 completed 0.9694
EXP_115 davidnet-no_disentangle 2024 completed 0.9799
EXP_116 davidnet-no_moddrop 2024 completed 0.9841
EXP_117 davidnet-no_sync 2024 completed 0.9826
EXP_118 aasist 7 completed 0.7507
EXP_119 davidnet-no_disentangle 7 completed 0.9778
EXP_120 davidnet-no_sync 7 completed 0.9700
EXP_121 davidnet-legacy 7 completed 0.9831
EXP_122 davidnet-qacp_no_copysynth 7 completed 0.9858
EXP_123 davidnet-compose_quad 2024 completed 0.9770
EXP_124 davidnet-no_qacp 7 completed 0.9852
EXP_125 davidnet-qacp_no_copysynth 2024 completed 0.9795
EXP_126 davidnet-logo_fsgan 2024 completed 0.9220
EXP_127 davidnet-no_loc 2024 completed 0.9838
EXP_128 davidnet-single_task 2024 completed 0.9841
EXP_129 davidnet-e2e 2024 completed 0.9845
EXP_130 davidnet-logo_faceswap 7 completed 0.9010
EXP_131 davidnet-single_task 7 completed 0.9730
EXP_132 effnet-b4 7 completed 0.9268
EXP_133 davidnet 7 completed 0.9807
EXP_134 davidnet-e2e 7 completed 0.9797
EXP_135 davidnet-logo_faceswap 2024 completed 0.9008
EXP_136 avoid-df 42 completed 0.9400
EXP_137 avoid-df 2024 completed 0.9313
EXP_138 avoid-df 123 completed 0.9376
EXP_139 avoid-df 456 completed 0.9422
EXP_140 cross-attn 123 completed 0.9346
EXP_141 cross-attn 2024 completed 0.9320
EXP_142 avoid-df 7 completed 0.9376
EXP_143 cross-attn 456 completed 0.9377
EXP_144 cross-attn 7 completed 0.9371
EXP_145 cross-attn 42 completed 0.9523
EXP_146 avff 123 completed 0.9797
EXP_147 avff 42 completed 0.9841
EXP_148 avff 2024 completed 0.9846
EXP_149 avff 7 completed 0.9816
EXP_150 avff 456 completed 0.9769
EXP_151 qacp-scaled 7 completed β€”
EXP_152 qacp-scaled 123 completed β€”
EXP_153 qacp-scaled 456 completed β€”
EXP_154 qacp-scaled 42 completed β€”
EXP_155 qacp-scaled 2024 completed β€”
EXP_156 davidnet-qacp_scaled 7 completed 0.9823
EXP_157 davidnet-qacp_scaled 123 completed 0.9804
EXP_158 davidnet-qacp_scaled 42 completed 0.9816
EXP_159 davidnet-qacp_scaled 456 completed 0.9796
EXP_160 davidnet-qacp_scaled 2024 completed 0.9682

βš–οΈ Ethics & Responsible AI Statement

All experimental evaluations in this repository were conducted under strict academic research ethics. Seven benchmark corpora were acquired under their respective research end-user licensing agreements; no raw biometric media is redistributed. The system is strictly defensive by design and contains no generative synthesis capability. Model outputs provide posterior probability calibration signals to assist human forensics experts, not autonomous legal determinations.


πŸ“œ Citation

@article{hossain2026davidnet,
  title={DAVID-Net: Disentangled Audio-Visual Forensic Attribution under Identity-Disjoint Evaluation},
  author={Hossain, Moshin Ibna and Khan, Alif Hasan and Shakib, Shadman and Jam, MD. Yeahyea and Hasan, Mohammad Mahmudul},
  journal={Machine Learning with Applications},
  year={2026},
  note={Under review}
}

@thesis{hossain2026thesis,
  title={Decoupled Multimodal Forensic Attribution and Desynchronization Detection},
  author={Hossain, Moshin Ibna and Khan, Alif Hasan and Shakib, Shadman and Jam, MD. Yeahyea},
  school={American International University-Bangladesh},
  year={2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using MIHMahmudEli/davidnet-experiments 1