How Much Full-Duplex Data Do You Need? What Published Systems Used

The short answer

Among published systems that learned turn-taking from real conversation with each speaker on a separate channel, that slice runs from 344 hours (J-Moshi, Japanese) to about 2,000 hours (Fisher, English). It sits on top of a much larger base. Moshi paired its 2,000 hours of Fisher with 7 million hours of mono audio and more than 20,000 hours of synthetic dialogue. Several models report no real two-channel conversation at all and handle turns in a different way.

So the answer to "how much?" has two parts: a base of speech that can be large, cheap, and single-channel, and a smaller set of real two-channel conversation in each language you ship, where the published range is a few hundred to a few thousand hours.

What each system trained on

"Real two-channel" means recorded human conversation with one speaker per channel, used to train the model directly. Hours are as each paper reports them.

Model	Real two-channel conversation	Other training data	Paper
dGSLM (2022)	Fisher, 2,000 h	None; textless model trained on Fisher only	arXiv 2203.16502
Moshi (2024)	Fisher, 2,000 h, upsampled from 8 to 24 kHz	7M h mono audio; diarized pseudo two-stream audio; 20k+ h synthetic dialogue	arXiv 2410.00037
SyncLLM (2024)	Fisher, 2,000 h	212k h synthetic dialogue rendered from text; Llama 3 8B base	arXiv 2409.15594
NTPP (2025)	Fisher, reported as 2,200 h, upsampled to 16 kHz	About 140,000 h of speech from three datasets	arXiv 2506.00975
J-Moshi (2025)	344 h Japanese, 201 h of it in-house	About 60,000 h of J-CHAT, diarized into two streams; 602 h synthetic (J-Moshi-ext)	arXiv 2506.02979
PersonaPlex (2026)	1,217 h of Fisher in the released checkpoint	Moshi weights; 2,250 h synthetic service and QA dialogues	arXiv 2602.06053
Raon-SpeechChat (2026)	13.21K h real conversation, mostly single-channel and diarized	106.33K h synthetic conversation; 1.38M h for the base model	arXiv 2605.23912
MiniCPM-o 4.5 (2026)	Not reported	Millions of hours of unlabeled speech; voice-actor dialogues; web audio-video	arXiv 2604.27393
Covo-Audio-Chat-FD (2026)	Not reported	About 8M h of pretraining audio; turn-based dialogues converted to two channels	arXiv 2602.09823
Freeze-Omni (2024)	None	110,000 h internal ASR data; about 3,000 h of TTS speech; 60,000 synthesized QA samples	arXiv 2411.00774
OmniFlatten (2024)	None	About 100K h of ASR and TTS data; 2,000 h of simulated two-channel dialogue	arXiv 2410.17799
SALMONN-omni (2025)	None	LibriSpeech and GigaSpeech ASR; about 730k QA and 80k synthesized conversations	arXiv 2505.17060
LSLM (2024)	None	585 h of LibriTTS; spoken commands as interruptions; MUSAN noise	arXiv 2408.02622

Two notes on the Fisher rows. The LDC catalog lists Fisher English as 11,699 calls totaling about 1,960 hours (LDC2004S13), and papers round it differently. Moshi also used 170 hours of separate-channel conversation, but to train its multi-stream TTS and to fine-tune the text model on transcripts, not to train Moshi directly, so it is left out of the real column.

What the table shows

Fisher is the shared English layer. dGSLM, Moshi, SyncLLM, and NTPP all used it for two-channel training. PersonaPlex starts from Moshi's weights, which already saw Fisher, and its released checkpoint adds 1,217 hours of Fisher again to improve backchanneling and expression.
Real conversation is a small share of the total. SyncLLM's 2,000 hours of Fisher is under 1% of its dialogue data next to 212,000 synthetic hours. In Raon-SpeechChat, real conversation is about 11% of the 119K hours of full-duplex training data.
Real conversation is where timing comes from. SyncLLM's authors use synthetic dialogue for meaning and bring in Fisher in the last stage to learn turn-taking cues. Moshi's paper says its simulated multi-stream data was not enough to capture natural conversation, and it fine-tunes on Fisher for that. Synthetic data still helps on top: in J-Moshi, adding 602 hours of synthetic dialogue improved meaningfulness over the 344-hour model.
Language matters. J-Moshi generated 5.0 seconds of overlap per minute against 1.2 for English Moshi, and the ground-truth Japanese dialogue had 8.1 against 3.3 for English. The authors note the conditions differ, but the direction matches earlier findings that Japanese conversation has more overlap and backchannels.
Diarized mono is the common substitute. Moshi's post-training, J-Moshi's 60,000 hours of J-CHAT, and most of Raon-SpeechChat's real conversation were single-channel recordings split by speaker diarization. Each paper still uses true two-channel data or synthetic data for the fine overlap behavior.
No real two-channel data usually means a different design. Freeze-Omni predicts at each chunk whether to keep listening or respond, SALMONN-omni trains on generated barge-in and backchannel scenarios, and LSLM learns to stop when interrupted. These are closer to the turn-based end of the full-duplex versus cascade spectrum than Moshi-style models that generate both streams.

Why the real slice stays small

Two-channel conversation is hard to find at scale, which fullduplex.ai's post on the data ceiling covers in detail. The table shows the effect: nearly every English system reaches for the same corpus. Fisher also carries limits into each model that uses it. It is 8 kHz telephone audio, so Moshi upsampled it to 24 kHz with a super-resolution model and NTPP upsampled it to 16 kHz. It is English only, recorded in 2003 between strangers given an assigned topic (LDC2005S13). Its license depends on LDC membership terms, which matters for a commercial model.

Teams outside English build their own. More than half of J-Moshi's 344 hours came from the lab's in-house corpora, with the rest from CALLHOME Japanese, the CSJ, and a travel agency dialogue corpus.

How to size your own purchase

The papers give reference points, and these decisions move the number for your model.

Separate the base from the conversation set. The large pretraining corpus can be mono, scraped, or synthetic. The part you are buying is the real two-channel conversation, and published systems used a few hundred to a few thousand hours of it per language.
Count per language. J-Moshi needed its own Japanese data even with Moshi's English training behind it. Budget a conversation set for each language you ship, and expect the turn-taking style to differ between them.
Match the sample rate to the output. If your model generates speech at 24 kHz or higher, 8 kHz telephone audio has to be upsampled, and the model learns from the upsampled version. Native wideband recordings avoid that step.
Match the conversation style. Fisher is strangers on assigned topics. PersonaPlex added synthetic service dialogues to teach role adherence, and SyncLLM used CANDOR as an out-of-distribution test. If you deploy into support calls, tutoring, or robots in a room, ask how much of the sample looks like that setting.
Measure a curve on your own model. Train on slices of the conversation set and evaluate on held-out real conversation with benchmarks such as Full-Duplex-Bench and TurnBench. PersonaPlex reports that Full-Duplex-Bench scores were strong with limited data while role adherence kept improving as data grew, so the answer depends on which behavior you measure. fullduplex.ai's evaluation page lists the options.

Before you sign, check the sample itself. Our buyer's guide lists the questions to ask about channels, consent, and license.

Where oto fits

oto builds full-duplex conversational speech datasets in English, Japanese, and Korean, with one speaker on each channel.

English beyond Fisher. The licensed catalog includes otoSpeech 1,125h of English conversation (3,878 sessions), and the open otoSpeech-full-duplex-280h is 48 kHz, channel-separated audio under CC BY 4.0.
Japanese and Korean. Japanese free conversation, role-play, and scripted dialogue at 48 kHz, and Korean free conversation at 44.1 kHz with ASR transcripts for both speakers.
Turn-taking labels. otoSpeech-full-duplex-turn-104h has 17 hand-annotated turn-taking event labels and is the training set for Sesame's TurnBench.
Pricing. Licensed datasets run from $10 to $100 per hour of audio. The license and labels such as turn-taking annotation add to the price, and custom collection to specification is quoted per project. Contact agent@oto.earth.

Data for voice AI

Request samples or a quote

Tell us what you are building, and we will send samples and pricing for the data that fits.

Request samples See the datasets Commission a collection

Frequently asked questions

How many hours of data did Moshi train on?

About 7 million hours of mono audio for pretraining, 2,000 hours of two-channel Fisher conversation for fine-tuning, and more than 20,000 hours of synthetic dialogue for instruction tuning.

How much real conversation data does a full-duplex model need?

Published systems used between 344 hours (J-Moshi, Japanese) and about 2,000 hours (Fisher, English) of real two-channel conversation, on top of much larger single-channel or synthetic data.

Can a full-duplex model be trained only on synthetic dialogue?

Some published models report no real two-channel data, but they mostly predict turn states or train on generated interruption scenarios. In the papers reviewed here, the models that generate natural overlap and backchannels fine-tuned on real conversation.

Why do so many full-duplex models use Fisher?

It is one of the few large English corpora with each speaker on a separate channel. It is also 8 kHz telephone audio from 2003, so models that generate wideband speech upsample it.

How much full-duplex data do I need for Japanese or Korean?

J-Moshi used 344 hours of Japanese two-channel dialogue after pretraining on about 60,000 hours of diarized Japanese audio. Plan a separate conversation set for each language, since turn-taking differs between languages.

For more datasets, benchmarks, and models, see the fullduplex.ai catalog.

otoearth, Inc.