Free Conversation, Role-Play, or Scripted: What Each Teaches a Speech Model

Natural timing and specific content pull in opposite directions

On a familiar topic, people speak naturally, interrupt, and say "mm" while the other person talks. Specialist content rarely comes up on its own. A script can put any content in, and reading it costs natural delivery. Role-play sits between the two: the scenario brings the content, and the speakers choose their own words.

A speech model can learn three things from conversation data, and each style supplies them in different amounts:

• Timing. Overlap, backchannels, holds, and response gaps. A full-duplex model learns these from the data (backchannel guide).
• Content. Vocabulary and the shape of a task, such as a hotel booking change or a support call.
• Alignment. How exactly the transcript matches the audio, which matters most for TTS and for ASR evaluation.

What the styles put into the audio

oto measured three sets of Japanese two-speaker samples, each recorded with both people in the same session on separate 48 kHz channels (otoSpeech Japanese). Speaking time was estimated from the signal level on each channel. A short response is a burst under one second while the other person talks, a rough stand-in for a backchannel.

These are 29- to 39-second clips chosen to show the data, so the table describes the samples only. Role-play had about half the overlap of free conversation, and the scripted clips had none.

Framing changes timing even when nobody follows a script. TurnBench recorded six conversation types, with speakers told the type and a starting topic but never told to interrupt or backchannel (Jiang et al., 2026). Overlap per dialogue still varied about threefold:

Language also changes timing. In the reference data reported for J-Moshi, overlap ran 8.1 seconds per minute in Japanese and 3.3 in English. The two figures come from different corpora, so treat the gap as indicative (Ohashi et al., 2025).

Free conversation

Free conversation teaches timing. It carries the overlaps, backchannels, fillers, and long holds that a full-duplex model or an end-of-turn detector has to learn. Fisher, the corpus behind Moshi's full-duplex fine-tune, pairs strangers on an assigned topic, and CANDOR records open-ended video calls.

It is the right base when timing is the goal. otoTurn, an end-of-turn model fine-tuned on spontaneous two-channel conversation with turn-taking labels, cut false interruptions in spontaneous speech by 47.6% at 300 ms (oto, 2026).

It gives little control over content. Domain terms, numbers, addresses, and task steps appear only by chance, and transcripts are made after recording, so expect some transcription errors.

Role-play

Many task-oriented speech corpora are role-played. Published examples:

• SpokenWOZ. 250 participants made 5,700 dialogues by phone. One played a user with a generated task goal, the other an agent searching a database, for 249 hours in total. The authors added a backchannel act to the annotation scheme (Si et al., 2023).
• AppTek Call-Center Dialogues. Role-played agent and customer calls with no scripted text, in 14 English accents and 16 service topics, 128.6 hours of 16 kHz audio with one channel per speaker. It was built for evaluation only (paper).
• Perit Duplex Support Conversations. 45 performed support calls, 5.10 hours at 48 kHz with one speaker per channel. Agent and customer roles swap between files (card).
• Travel Agency Dialogue Corpus. 115 hours of operator and customer role dialogue over Zoom, one of the five stereo corpora J-Moshi fine-tuned on (Ohashi et al., 2025).
• Expresso, improvised section. Actors get a situation and a character, such as two drivers after a car accident, and improvise. The dialogues are stereo with one channel per actor (Nguyen et al., 2023).

Role-play teaches content together with some natural timing. oto's Japanese role-play covers scenarios from a souvenir shop and a hotel booking change to a trade show booth and a manager one-on-one. In the samples above it kept 10% overlap and 6.4 short responses per minute: less than free conversation, but enough that real turn-taking stays in the audio.

Fixed roles teach a model one side of the call, so check whether speakers swap. If the brief told actors where to interrupt, the interruptions follow the brief.

Scripted dialogue

Scripted data gives exact text. Every word is known in advance, so alignment is clean and specified vocabulary is guaranteed. That suits TTS, and several published corpora use it:

• DailyTalk. 2,541 dialogues adapted from the DailyDialog text corpus and recorded in a studio by two voice actors for conversational TTS. The actors were told to perform the conversations rather than only read them, and to add fillers in about half (Lee et al., 2022).
• HumDial. The ICASSP 2026 challenge had professional actors perform LLM-written scripts with barge-ins and side talk written in, so interruptions land at meaningful points (paper).
• Moshi's TTS data. Kyutai trained its multi-stream TTS on 170 hours of "natural and scripted conversations" recorded with one channel per speaker (Défossez et al., 2024).

oto's scripted Japanese set has scenes such as a hotel check-in and a cancelled train, and every recording comes with its script and a timed transcript. The samples had no overlap and no short responses. Clean turns like these suit TTS, and they will not teach a model to handle interruptions.

Which style for which job

A product often needs more than one style. One split that follows from the table: free conversation for timing, role-play for the domain, and scripted lines for TTS and terms. The training data guide covers the channel and license checks that apply to all three.

What to check in each style

• Free conversation: the topic list, whether speakers knew each other, and measured overlap and backchannel rates on samples you pick yourself.
• Role-play: whether speakers used their own words or lines, the scenario list, whether roles swap, and whether the brief mentioned interruptions or backchannels.
• Scripted: whether the script and a timed transcript ship with each recording, how far the speakers departed from the script, and whether fillers were added on instruction.
• All three: one speaker per channel, the sample rate, and a license that covers your use.

Where oto fits

• Japanese in all three styles. otoSpeech Japanese (free conversation), otoSpeech Role-Play, and otoSpeech Scripted, all two-speaker at 48 kHz with one speaker per channel. Samples are on oto.earth/datasets/japanese, and licensing is on request.
• English free and task-oriented conversation. otoSpeech-full-duplex-280h and 141h are open under CC BY 4.0. otoSpeech-full-duplex-task-oriented-20h covers seven collaborative tasks, with task stimuli, interaction logs, and reference answers.
• Collection to specification. Other scenarios, speakers, and languages, including role-play and scripted designs.
• Pricing. Licensed datasets run from $10 to $100 per hour of audio, and the license and labels add to the price. Custom collection is quoted per project. Write to agent@oto.earth.

Frequently asked questions

What is the difference between role-play and scripted speech data?

In role-play, speakers get a scenario and use their own words. In scripted data, they read written lines, and each recording usually ships with its script.

Is scripted dialogue good for training a full-duplex model?

Not on its own. oto's scripted Japanese samples had no overlap and no short responses, so they will not teach interruption or backchannel handling.

Which speech data style is best for TTS?

Scripted dialogue, for its exact alignment and controlled vocabulary. DailyTalk, recorded from scripts by two voice actors, was built for conversational TTS.

Does role-play data sound natural?

Partly. In oto's Japanese samples, role-play had about half the overlap of free conversation (10% against 19%) and kept frequent short responses.

Are there role-play datasets for call-center voice agents?

Yes: SpokenWOZ, AppTek Call-Center Dialogues (evaluation only), and Perit Duplex Support Conversations, each with different licenses and audio formats.

Do Japanese conversations overlap more than English ones?

The reference data reported for J-Moshi shows 8.1 seconds of overlap per minute in Japanese against 3.3 in English, from different corpora.

For more conversational datasets in every style, see the fullduplex.ai catalog.

Related guides

• Japanese Conversational Speech Datasets for Full-Duplex Models
• Real vs. Synthetic Dialogue for Speech-to-Speech Training
• Turn-Taking Labels: What to Annotate for Voice Agents