What makes the DDL dataset useful for deepfake detection and localization?

Binary labels leave localization unsupported

Most deepfake datasets supply an image-level or video-level 0-1 label saying whether a sample is fake. That label is enough to train a classifier, but it says nothing about which region was manipulated or which stretch of a video was altered. The paper argues this is why methods that try to output spatial manipulation masks or temporal forgery segments still perform below what interpretable detection needs: the annotations they would have to learn from are largely absent.

The paper's comparison table makes the gap concrete. DF40, described as the latest comparable dataset, covers 40 forgery types and 1.1M+ fake samples but offers only image-level binary labels. Earlier localization-oriented datasets go the other way: Dolos annotates local manipulation masks in single-face scenarios, OpenForensics and FFIW extend mask annotation to multi-face scenarios, but all three are limited in forgery types and do not cover audio-video forgeries. LAV-DF is the first audio-video temporal localization dataset but has only 10K samples; AV-Deepfake1M scales to millions of samples yet contains a single video forgery method and two audio forgery types.

Masks and segments answer different questions

DDL is split into two parts that support different tasks. DDL-I holds unimodal images and targets spatial forgery localization, where the model must mark the manipulated region. DDL-AV holds multi-modal audio-visual content and targets temporal forgery localization, where the model must identify the forged time segment. The two annotation counts therefore cannot be read as one number: 1.18M+ spatial masks and 0.23M+ temporal segments describe different sample populations and different outputs.

The paper stresses that its masks come from the forgery generation process rather than from post-hoc human annotation. It contrasts this with work that manually annotates the existing FF++ dataset to obtain relatively precise masks, arguing that post-processing cannot substitute for annotations produced by the actual forgery process. This matters for readers comparing datasets: a mask count alone does not establish that masks were derived the same way.

Eighty methods span seven generation architectures

The dataset's method coverage is the first of four stated design innovations. DDL draws on 80 deepfake techniques across seven generation architectures: GANs, diffusion models, VAEs, Normalizing Flows, NeRFs, autoregressive models, and commercial software. The paper reports a breakdown of 11 commercial software tools alongside the research architectures, and states that this breadth covers both visual and audio modalities.

Coverage of manipulation modes is the second innovation. In the spatial domain DDL includes face swapping, face reenactment, full-face synthesis, and face editing; in the temporal domain it includes deletion, replacement, and insertion of forged content. The paper also introduces three modes it describes as new: hybrid face forgery, audio-visual asynchronous manipulation, and audio-visual full synthesis. The third innovation is scenario and modality diversity, with three scenarios (single-face, multi-face, audio-visual) and three modalities (image, video, audio).

Localization scores fall sharply on the test split

The benchmark table separates a validation set from a test set, and the gap between them is the paper's clearest evidence that its test split is harder. On DDL-I, MSCCNet reaches 99.83 accuracy, 99.95 AUC, 98.14 F1, and 96.79 IoU on the validation set; on the test set the same model records 78.17 accuracy, 88.88 AUC, 66.39 F1, and 49.69 IoU. SparseViT moves from 99.70 accuracy and 95.94 IoU on validation to 85.47 accuracy and 53.13 IoU on test.

The pattern holds across model families. IML-ViT falls from 90.39 accuracy and 69.28 IoU on validation to 80.35 accuracy and 41.93 IoU on test, while Mesorch drops from 96.56 IoU to 35.52 IoU. Detection-only baselines also decline but by less dramatic margins: UnivFD goes from 83.64 to 76.12 accuracy, DualNet from 80.97 to 74.65, and SPSL from 81.34 to 72.79. Because IoU measures region overlap rather than classification correctness, the steep IoU drops indicate that locating the manipulated area is the harder part of the task on this test set.

Audio-visual temporal localization stays near chance

On DDL-AV the reported metrics are accuracy, AUC, AP, and AR, which describe temporal segment prediction rather than spatial overlap. UMMAFormer is the strongest of the three baselines on validation, with 78.87 accuracy, 93.35 AUC, 72.28 AP, and 58.68 AR. On the test set it records 54.39 accuracy, 60.07 AUC, 55.75 AP, and 56.14 AR.

The other two baselines fall further. BA-TFD drops from 67.68 accuracy and 95.23 AUC on validation to 42.71 accuracy and 68.60 AUC on test, with AP falling from 47.20 to 26.35 and AR from 36.17 to 23.53. BA-TFD+ goes from 70.71 accuracy and 94.90 AUC to 42.74 accuracy and 54.40 AUC. The near-halving of accuracy between validation and test on this split is the paper's strongest signal that audio-visual temporal localization under distribution shift is not solved by current baselines.

The test split shares only 21.05% of sources

The paper states that most existing datasets build training and test sets from the same distribution, which can let a model overfit and then fail on unseen conditions. DDL deliberately separates the two distributions along two axes: the sources of real data and the generation model types used. The measured result is that test samples in DDL overlap training sources by only 21.05%, compared with 98.98% for DF40 and 100% for both AV-Deepfake1M and ForgeryNet.

This number is the key to reading the benchmark table. The validation-to-test drops are not evidence that the models are poorly trained; they are evidence that the test set was constructed to differ from what the models saw. A reader comparing DDL scores with scores on a same-distribution benchmark would be comparing different things, because the denominators and the source overlap differ.

A localization benchmark, not a deployment guarantee

The paper positions DDL as support for next-generation detection, localization, and interpretability methods, and mentions potential use in rigorous domains such as legal proceedings. Those are stated aims rather than demonstrated outcomes: the evidence packet reports dataset construction and baseline benchmark numbers, not field deployment or legal validation. The benchmark results show that current baselines degrade on the out-of-distribution split; they do not show that any baseline is adequate for real-world forensic use.

Two scope conditions follow directly from the design. First, the headline counts mix tasks: 1.4M+ forged samples is the overall total, while 1.18M+ masks apply to spatial localization and 0.23M+ segments to temporal localization, so the annotation density differs by task. Second, the benchmark covers ten named baselines across the two subsets, so conclusions about which method family is best are bounded by that baseline list. The paper also states that all source data is licensed for reuse and privacy-compliant, with usage agreements restricting access to research purposes.

Prioritize the source-overlap number over raw scale

For a reader deciding whether to use DDL, the most decision-relevant figure is not the 1.4M+ sample count but the 21.05% source overlap. Scale alone is already available elsewhere: DF40 offers 1.1M+ fake samples and MFFI 0.7M, and ForgeryNet matches DDL's order of magnitude at 1.4M+. What separates DDL is the combination of fine-grained spatial and temporal annotations with a test split deliberately built to differ from training.

The practical split is therefore by task. If the goal is spatial localization on images, DDL-I supplies 1.18M+ masks and a test set where the best reported IoU is 53.13, leaving clear headroom. If the goal is temporal localization on audio-visual content, DDL-AV supplies 0.23M+ segments and a test set where the best reported accuracy is 54.39. In both cases the number to watch is the validation-to-test gap, because that gap is what the dataset was designed to expose.

Sources used in this answer