Generated by Answer me with HTML 0.5.7 · 2026-10-26 01:40 · ★ Star on GitHub

Speech Synthesis & Audio Deepfake Detection

Research questions, current approaches and open problems | 2025–2026 | Updated 26 October 2026

Research perspective. Synthesis aims to produce the intended words, voice and expression reliably. Detection aims to remain reliable when generators and recording conditions change.

This map connects six research questions to representative papers. The annotated literature catalog contains 59 records with primary-source links and evidence boundaries.

A

The research landscape

Area What researchers are trying to achieve What makes the problem difficult
Speech synthesis Generate accurate, expressive speech with independent control of voice, style and timing. These properties interact. Better control can reduce naturalness, voice similarity or stability.
Audio deepfake detection Identify synthetic or edited speech across unfamiliar generators, speakers and channels. Detectors can learn dataset shortcuts and fail when the test conditions change.
Shared representation foundation Learn which information speech features preserve, discard or expose. A feature useful for linguistic content may hide or mix speaker, emotion and synthesis evidence.

Six questions organize the field: control, representation and training, delivery, generalization, localization, and realistic evaluation.

Reading guide. Panels B–D cover synthesis; E–G cover detection. Panel H proposes research directions for discussion.

Evidence guide. This is a research synthesis, not a universal SOTA ranking. Individual results depend on data, baselines, latency settings and evaluation protocols.

B

Synthesis: can we control how speech sounds without changing who speaks?

Problem. A reference recording mixes voice identity, emotion, rhythm and recording conditions. Copying one property can accidentally copy another.

Current approach Representative work What the work is doing
Separate voice and expression IndexTTS2 · 2025; FC-TTS · 2026 Use different conditioning information for voice and style. Add duration or consistency objectives.
Put separation inside the speech representation DisCo-Speech · 2025; Kanade · 2026 Separate stable speaker-related information from changing speech content and expression.
Learn expressive scene-level behavior Fish Audio S2 · 2026; Drama 3 · Oct 2026 Train for instructed expression, multiple speakers and longer conversational scenes.

Open problem. Can a model combine an unfamiliar speaker, emotion and language while preserving all three correctly?

A useful experiment. Hold the spoken words fixed. Change only the requested emotion, then measure voice identity, emotion and pronunciation separately.

Research implication. The presence of separate controls is not enough. The stronger question is whether those controls remain independent outside the training combinations.

C

Synthesis: what should the model predict, and how should the components learn together?

Problem. Speech can reconstruct well yet remain difficult to predict. Separately trained components can also disagree about which information matters.

Current approach Representative work What the work is doing
Design better discrete speech units SiTok · Feb 2026; ReLMCodec · Aug; Q-SPT · Oct Balance acoustic detail, linguistic structure, token rate and predictability.
Model continuous speech or refine token groups in parallel VoxCPM2 · June 2026; Luna-TTS · Aug Explore alternatives to predicting every discrete speech unit one after another.
Coordinate training across components End-to-end discrete-token TTS · June 2026; Qwen-Audio-3.0-TTS · July Optimize representations and generators using downstream reconstruction, recognition and preference feedback.
Make feedback specific to local errors LoCA · 20 Oct 2026 Use text alignment to assign feedback to the acoustic regions responsible for content errors.

Open problem. Which representation gives the best generation quality at a fixed training and inference budget?

Evidence to request. Compare final speech, content errors and generation cost. Reconstruction scores or token bitrate alone cannot settle this question.

D

Synthesis: can quality survive low latency and long recordings?

Problem. Fast output leaves less future text available. Long recordings expose accumulated errors and drift in voice or expression.

Current approach Representative work What the work is doing
Read incoming text and speak incrementally VoXtream · 2025; VoXtream2 · 2026 Train generation around limited future context and changing speaking-rate requests.
Reduce generation cost ZipVoice · 2025; TontaubeV1 · Sept 2026 Use compact or distilled generators, hierarchical speech modeling and bounded context.
Preserve long-context behavior VibeVoice · 2025; MOSS-TTS · 2026 Model extended speech and multiple speakers using scalable speech representations.
Evaluate the whole recording SwanBench-Speech · 2026 Test diverse long-form scenarios across acoustics, content and expression.

Open problem. How much future text is necessary for good phrasing, and how can the system recover after a bad decision?

Evaluation requirement. Report first-audio latency, sustained generation speed, content errors and long-form consistency together.

Task boundary. Streaming audio output and streaming text input are different capabilities. Scripted dialogue synthesis also differs from interactive full-duplex conversation.

E

Detection: will the detector work on the next generator?

Problem. A detector can recognize familiar artifacts or recording conditions instead of learning evidence that transfers to new synthesis systems.

Current approach Representative work What the work is doing
Evaluate transfer explicitly TwinShift · 2025; Spoof-SUPERB · 2026 Separate speaker and generator shifts; compare pretrained speech representations under common tests.
Track changes after generator adaptation GenTraceBench · Sept 2026 Ask whether a generator's detectable fingerprints survive fine-tuning and preference training.
Learn new attacks over time RF-Prompt · Sept 2026 Retain reusable real-speech information while adding knowledge of new fake mechanisms.
Improve deployment efficiency CoReLoop · Sept 2026; Task-aware compression · Oct Refine a frozen detector or compress it while trying to retain detection knowledge.

Open problem. Can the detector reject unfamiliar fakes without increasing false alarms on unfamiliar genuine speech?

Evidence to request. Hold out generator families, speakers, languages and recording sources. Test a fixed decision threshold on future data.

Practical implication. Lower average error on one dataset does not establish dependable deployment.

F

Detection: where is the fake region, and what evidence supports the decision?

Problem. A recording can be mostly genuine with only a few words changed. A single real/fake score hides both location and reasoning.

Current approach Representative work What the work is doing
Build realistic neural-editing tests PartialEdit · 2025 Test edits produced by speech-editing models rather than only simple splices.
Separate boundary finding from classification Split and Conquer · Apr 2026 Locate acoustic transitions, then judge the resulting segments.
Jointly classify and localize with a small model Half-Truth / CAFNet · revised Sept 2026 Fuse acoustic features and predict edited-region boundaries.
Extend existing detectors to conversations Zero-shot localization · Sept 2026 Convert window-level scores into coherent intervals without retraining the detector.
Audit the cues behind a score What Do Detectors Hear? · June 2026 Combine time-aligned attribution, cue annotation and masking experiments.

Open problem. Can evidence remain localized and reliable for short, seamless edits from unseen generators?

A caution from current evidence. CAFNet's checked revision reports strong in-domain performance but near-chance binary transfer to PartialSpoof.

Evaluation requirement. Report missed edits, boundary accuracy and false alarms on genuine conversational transitions. Explanations require validation, not only attractive heatmaps.

G

Detection: do laboratory results survive real audio conditions?

Problem. Compression, replay, emotion, background sound and language changes can alter the same acoustic cues that detectors use.

Stress test Representative work What the work is doing
Broad shared evaluation ASVspoof 5 · 2026; AUDDT · revised June Evaluate attacks and detector behavior across shared resources or many existing datasets.
Channels, processing and replay RADAR · 2026; AT-ADD · revised Oct Test media transformations, unseen synthesizers and realistic recording effects.
Emotion and mixed sound AffectDF · Aug 2026; MixFake · revised Oct Expand evaluation beyond clean, neutral, isolated speech.
Languages and provenance marks LRLspoof · 2026; Watermark impact · Aug Test language transfer and interactions between watermarks and blind detectors.

Open problem. Which failures come from the generator, the recording channel, or a mismatch in genuine-speech data?

Evidence to request. Report results by condition, with false-alarm rates and threshold calibration. Avoid hiding failures inside pooled averages.

Provenance boundary. A watermark can supply positive evidence when recovered. Its absence does not establish authenticity.

H

Research opportunities for discussion

The following are proposed research directions inferred from the literature. They are not established results.

Candidate direction Concrete question What a convincing study would measure
Independent expressive control Can voice, emotion and language remain independent for unfamiliar combinations? A crossed test of those factors, with pronunciation, identity and expression assessed separately.
Representation design for generation Which information must remain in the speech representation at a fixed compute budget? Matched-budget comparisons of predictability, quality, controllability and latency.
Reliable incremental synthesis Can a streaming model correct phrasing before errors become audible? Quality versus text lookahead; first-audio latency; failure frequency on long, difficult text.
Detection under chronological change Can a detector stay calibrated after a generator is fine-tuned or replaced? Train on earlier systems; test later systems and new genuine recordings at a fixed threshold.
Forensics for neural speech editing Can a detector locate a short edit after compression or replay? Unseen editing models, short altered spans, boundary accuracy and genuine-speech false alarms.
Joint synthesis–detection benchmark Which synthesis improvements actually remove transferable detection cues? Matched text and speakers across generator versions, styles and channels; both quality and forensic transfer.

Discussion prompts. Which failure matters most in our target application? Which dataset can isolate it? What result would disprove our hypothesis?

Coverage and evidence. The catalog contains 32 synthesis records and 27 detection or provenance records from 2025–2026.

All records have checked primary metadata and abstracts. Selected full-text sections informed specific caveats; experiments were not reproduced.

This is a broad curated map, not a claim of exhaustive coverage. Search scope and evidence notes document its limits.

Files. Annotated catalog · BibTeX bibliography · Usage instructions.