Beyond Fisher: High Fidelity, Full-Duplex Multilingual Conversational AI Datasets

Every major full-duplex voice AI model traces its training data back to the same source: the Fisher English corpus^[1], a catalog of two-party telephone calls recorded at 8 kHz in 2004.

Meta's dGSLM,^[2] the foundational full-duplex model, trained on 2,000 hours of Fisher. SyncLLM^[3] used 1,927 hours of Fisher as its real-data anchor; the remaining 99% of its 215,000-hour training set was synthesized by TTS. SALM-Duplex^[4] and PersonaPlex^[5] (the two openly available end-to-end duplex speech-to-speech models) are both grounded on the same telephony-grade audio. PersonaPlex draws speaker voice samples from Fisher and adjacent audiobook corpora, then synthesizes the conversations themselves.

The recent survey From Turn-Taking to Synchronous Dialogue^[6] names data scarcity as the primary bottleneck to progress, more limiting than architecture. The Full-Duplex-Bench evaluation^[7] finds that no open-source model achieves both natural backchanneling and appropriate interruption behavior simultaneously, and attributes the gap to training data, not model design.

The field has asked for better data. The Ocular Full-Duplex Hi-Fi corpus is our answer.

How Labs Have Patched the Gap

Rather than collect better data, labs have patched the Fisher gap with increasingly elaborate simulations.

Synthetic TTS data does not contain real prosody, real disfluency, or real overlaps. SyncLLM is illustrative: the model learns the structural pattern of conversation from synthesized dialogue, then anchors to telephony-grade audio for acoustic grounding. The structure is fake; only the acoustics (and only at 8 kHz) are real.

SALM-Duplex and PersonaPlex go further. Each layers its TTS dialogue on top of a different real-audio backbone: SALM-Duplex mixes VoxPopuli's^[12] European Parliament recordings (16 kHz) with 8 kHz Fisher, while PersonaPlex draws speaker voices from Libriheavy^[13] audiobook readings and synthesizes the conversational structure on top of them.

For barge-in (the moment a speaker interrupts mid-utterance), SALM-Duplex inserts a programmatic 0.64-second silence followed by the interrupting voice. PersonaPlex uses "negative-duration silence" to stitch the interrupting voice on top of the interrupted speaker's audio.

These are engineering approximations of a real human behavior. A barge-in involves overlapping prosody, pitch alignment between both speakers, and a floor negotiation (one speaker yields) that unfolds over hundreds of milliseconds with acoustic cues at every step. The simulation captures none of that.

That this is a data problem and not an architecture problem is visible directly in the benchmark numbers. The Thinking Machines Interaction Model^[11] scores 77.8 on Full-Duplex-Bench against ~50 for the best openly available models, and the researchers attribute the gap not to architectural superiority but to proprietary access to high-quality real conversational data.

What the Field Has Explicitly Asked For

Three recent papers read as direct calls for better full-duplex training data:

From Turn-Taking to Synchronous Dialogue:^[6] A survey of full-duplex spoken language models that classifies architectures and unifies evaluation across temporal dynamics, behavioral arbitration, semantic coherence, and acoustic performance. Its headline finding: the open problems aren't architectural — they're "synchronous data scarcity, architectural divergence, and evaluation gaps." Solving the first one unblocks the other two.

Full-Duplex-Bench:^[7] Evaluates full-duplex models across pause handling, backchanneling, turn-taking, and user interruption using 727 samples drawn from CANDOR,^[9] ICC,^[10] and GPT-4o / ChatTTS synthetic audio. The authors explicitly note the scarcity of high-quality annotated full-duplex data as a limitation of the benchmark itself: a corpus with labeled backchannel timing, barge-in types, and turn-mechanics would let the benchmark be run at scale with ground truth rather than proxy metrics.

FLEXI:^[8] Introduces six interaction scenario types (including emergency and emotional support) and finds significant open-source/commercial gaps in every scenario. The authors identify per-scenario training data coverage as a key missing ingredient. A corpus annotated with scenario types and role dynamics directly addresses this.

The Ocular Full-Duplex Hi-Fi Corpus

Ocular Full-Duplex Hi-Fi is a studio-grade conversation corpus built deliberately against the constraints Fisher imposed. Every parameter (sample rate, channel topology, capture device, acoustic environment, speech mode) is set against what Fisher allowed.

Per-session quality gate. Before recording begins, each session must pass a quality check: spectral energy verified above 2 kHz, noise floor below threshold, peak levels within headroom, SNR confirmed. Sessions that fail the gate don't record. Fisher had no equivalent gate; it accepted whatever the telephone network delivered.

Unscripted conversation. Speakers are not given prompts, not read-speech, not asked to demonstrate specific behaviors. Conversations are collected worldwide across diverse accents, ages, and topics. Natural barge-ins, real backchannels, and genuine hesitation occur because the conditions for them to occur are present.

Why fidelity matters above 4 kHz. Fisher captured what telephone infrastructure allowed: 8 kHz audio, capped at 4 kHz by the network, through handset microphones, in whatever acoustic environment callers happened to be in at the time. The acoustic signals that separate human speech from TTS output (the audible breath before a response, the prosodic fall of a completed clause, the 2–5 kHz presence region that carries vocal warmth, the overlap of two voices at full fidelity) all live above 4 kHz. Fisher cannot contain them. Every model trained primarily on Fisher is learning from data that is acoustically blind to the layer it most needs.

You can hear the difference in the samples below: the breaths, the 2–5 kHz presence band, and the unmixed overlap, all preserved.

Samples

Two unscripted conversations, captured per-speaker on isolated channels. Speaker A and Speaker B recorded simultaneously on separate studio-grade wired microphones, time-synchronized, 48 kHz / 24-bit lossless.

Conversation 7

Channel isolation, side by side

Same conversation, two simultaneous wired microphones in different rooms. The pair below is the raw per-speaker capture, what the model would actually train on.

Play both

Speaker A · Studio-grade mic · Room 1

0:00 / 0:00

Speaker B · Studio-grade mic · Room 2

0:00 / 0:00

Two speakers, separate rooms, separate microphones, time-synchronized. Each channel is clean isolation: no bleed, no merge, no re-encode.

Conversation 8

Swapped rooms, same isolation

The same two speakers swap microphone and room assignments. Cross-correlation between channels comes back at Pearson r = 0.001, so what you're seeing is two independent captures, not one mic bleeding into the other.

Play both

Speaker A · Studio-grade mic · Room 2

0:00 / 0:00

Speaker B · Studio-grade mic · Room 1

0:00 / 0:00

What Annotation of This Data Enables

The samples above are raw capture: 48 kHz / 24-bit, channel-separated, unprocessed. What makes a corpus training-ready is the annotation layer on top of that capture. Because the audio exists at full fidelity with independent channels, it can support annotation depth that lower-quality or merged captures cannot: turn-construction unit boundaries and transition-relevance places (TRPs),^[14] backchannel timing, and barge-in type all require prosodic detail that 8 kHz telephony erases.

The table below maps each annotation type to the specific research gap it closes and the benchmark metric it directly improves. This is the layer that every paper in this space has identified as missing.

Each annotation layer is a direct training signal for a metric the field already uses to evaluate models. The annotation protocol covering all seven layers (plus prosodic markers, EQ trajectory, and semantic role dynamics) is available on request.

How This Corpus Compares

The table below lines up Ocular Full-Duplex Hi-Fi against Fisher and the three open simulated training sets across the axes every recent full-duplex paper names as a training bottleneck.

Every empty cell in this table is a gap the evaluation literature has named as a bottleneck. The interactive comparison below picks each of those corpora apart on the audio itself.