How to Check a Two-Channel Speech Dataset in 30 Minutes

What gets sold as two-channel

Full-duplex models learn turn-taking, backchannels, and interruptions from the moments when both people are talking. Those moments are only usable when each voice sits clean on its own track. Six formats show up on spec sheets, and each leaves a different fingerprint in the numbers.

The last row is what you want. The training data overview on fullduplex.ai covers why, and our buyer's guide covers what to ask before a sample arrives.

Before you start

Pick the conversation yourself, from the full file list. A clip chosen by the vendor tends to be its best one. Ask for the delivery format (WAV or FLAC), not a preview MP3, since lossy compression changes the spectrum you are about to measure. If the dataset has transcripts, get the transcript for the same conversation.

Then read the header. ffprobe -v error -select_streams a:0 -show_entries stream=codec_name,sample_rate,channels,bits_per_raw_sample -of default=nw=1 sample.flac prints the codec, sample rate, channel count, and bit depth. For FLAC, read bits_per_raw_sample, because bits_per_sample shows 0. If SoX is installed, soxi sample.wav prints the same fields.

The header tells you how the file is stored, which is not always how it was recorded. A 48 kHz, 24-bit file can hold audio captured at 8 kHz over a phone line, which step 6 of the script checks.

The audit script

Save one conversation as sample.wav (or change PATH) and run this with numpy and soundfile installed. It assumes channel 0 is speaker A and channel 1 is speaker B. We tested it on synthetic files built to match each row of the table above.

import numpy as np
import soundfile as sf

PATH = "sample.wav"  # one full conversation from the vendor's sample
HOP_S = 0.02         # 20 ms analysis frames
FLOOR_DB = 15        # speech must be this far above the channel's noise floor
BLEED_DB = 20        # and not this far below the other channel (that is bleed)

info = sf.info(PATH)
print(f"{info.format} {info.subtype}, {info.samplerate} Hz, "
      f"{info.channels} ch, {info.duration / 60:.1f} min")
audio, sr = sf.read(PATH, dtype="float64", always_2d=True)
assert audio.shape[1] >= 2, "one channel: a mixed recording, not one speaker per channel"
a, b = audio[:, 0], audio[:, 1]
hop = int(sr * HOP_S)


def xcorr_peak(x, y, max_lag):
    """Peak normalized cross-correlation within +/- max_lag samples, and its lag."""
    n = len(x) + len(y)
    c = np.fft.irfft(np.fft.rfft(x, n) * np.conj(np.fft.rfft(y, n)), n)
    c = np.concatenate([c[-max_lag:], c[: max_lag + 1]])
    c /= np.sqrt(np.sum(x**2) * np.sum(y**2)) + 1e-12
    i = np.argmax(np.abs(c))
    return round(float(c[i]), 3), round((i - max_lag) / sr * 1000, 2)


def frame_db(x):
    n = len(x) // hop
    rms = np.sqrt(np.mean(x[: n * hop].reshape(n, hop) ** 2, axis=1))
    return 20 * np.log10(rms + 1e-10)


def runs(mask):
    d = np.diff(np.concatenate([[0], mask.astype(int), [0]]))
    return np.flatnonzero(d == 1), np.flatnonzero(d == -1)


def talk_spurts(db, other_db):
    floor = np.percentile(db[db > -150], 10)
    active = (db > floor + FLOOR_DB) & (db > other_db - BLEED_DB)
    starts, ends = runs(active)
    for e0, s1 in zip(ends[:-1], starts[1:]):  # bridge pauses under 200 ms
        if (s1 - e0) * HOP_S < 0.2:
            active[e0:s1] = True
    return active


# 1. Level, clipping, digital silence
for name, x in (("A", a), ("B", b)):
    print(f"ch {name}: peak {20 * np.log10(np.abs(x).max() + 1e-12):.1f} dBFS, "
          f"clipped {np.mean(np.abs(x) >= 0.999):.4%}, exact zeros {np.mean(x == 0):.1%}")

# 2. Identical, mixed, or delayed copies (first 60 s)
seg = slice(0, sr * 60)
print("max sample difference:", float(np.abs(a - b).max()))
print("sample correlation:", round(float(np.corrcoef(a, b)[0, 1]), 3))
print("cross-correlation peak, lag ms:", xcorr_peak(a[seg], b[seg], int(sr * 0.05)))

# 3. Who talks when
ea, eb = frame_db(a), frame_db(b)
act_a, act_b = talk_spurts(ea, eb), talk_spurts(eb, ea)
speech = act_a | act_b
print(f"talk share A {act_a.mean():.1%}, B {act_b.mean():.1%}, "
      f"overlap share of speech {(act_a & act_b).sum() / speech.sum():.1%}")

# 4. Turn timing: gap (+) or overlap (-) when the floor changes hands,
#    and short responses inside the other person's speech
spurts = sorted([(s, e, 0) for s, e in zip(*runs(act_a))] +
                [(s, e, 1) for s, e in zip(*runs(act_b))])
last_end, fto, short = {0: -1, 1: -1}, [], 0
for s, e, ch in spurts:
    other = last_end[1 - ch]
    if e <= other:
        short += (e - s) * HOP_S < 1.0
    elif other > last_end[ch] and (s - other) * HOP_S < 3:
        fto.append((s - other) * HOP_S)
    last_end[ch] = max(last_end[ch], e)
fto = np.array(fto) if fto else np.zeros(1)
print(f"turn changes {len(fto)}: median {np.median(fto):.2f} s, "
      f"p10 {np.percentile(fto, 10):.2f} s, p90 {np.percentile(fto, 90):.2f} s, "
      f"sd {fto.std():.3f} s, starting in overlap {np.mean(fto < 0):.0%}")
print(f"short responses inside the other's speech: {short / (len(a) / sr / 60):.1f} per min")

# 5. Bleed: what channel B carries while only A talks, and the reverse
for name, (x, y, ex, ey, solo) in {"A->B": (a, b, ea, eb, act_a & ~act_b),
                                   "B->A": (b, a, eb, ea, act_b & ~act_a)}.items():
    idx = np.flatnonzero(solo)[: int(60 / HOP_S)]
    if len(idx) == 0:
        print(f"bleed {name}: no frames where only one person talks")
        continue
    xs = np.concatenate([x[i * hop:(i + 1) * hop] for i in idx])
    ys = np.concatenate([y[i * hop:(i + 1) * hop] for i in idx])
    print(f"bleed {name}: level {np.median(ey[idx] - ex[idx]):.1f} dB, "
          f"correlation, lag ms {xcorr_peak(xs, ys, int(sr * 0.02))}")

# 6. Real bandwidth: energy per 4 kHz band during speech, relative to 0-4 kHz
for name, x, act in (("A", a, act_a), ("B", b, act_b)):
    idx = np.flatnonzero(act)[:: 10][:300]
    frames = np.stack([x[i * hop:i * hop + 4096] for i in idx if i * hop + 4096 <= len(x)])
    power = (np.abs(np.fft.rfft(frames * np.hanning(4096), axis=1)) ** 2).mean(axis=0)
    freqs = np.fft.rfftfreq(4096, 1 / sr)
    base = power[freqs < 4000].sum()
    bands = [f"{lo // 1000}-{min(lo + 4000, sr // 2) / 1000:g}k "
             f"{10 * np.log10(power[(freqs >= lo) & (freqs < lo + 4000)].sum() / base + 1e-20):.0f} dB"
             for lo in range(4000, sr // 2, 4000)]
    print(f"ch {name} band energy:", ", ".join(bands))

The two constants at the top decide what counts as speech. A frame is speech on a channel when it is at least FLOOR_DB above that channel's noise floor and no more than BLEED_DB below the other channel. Energy is a rough detector. If you have a voice activity model such as Silero VAD, you can swap it in for talk_spurts and keep the rest.

How to read each number

No industry pass mark exists for any of these. Read the numbers together, and against what the vendor told you.

• Peak and clipping: a peak at 0.0 dBFS with a clipped share above zero means samples hit full scale. Clipping on every loud word means the gain was set too high, and that damage cannot be undone.
• Exact zeros: real microphones always pick up some noise, so long stretches of exact zero mean the audio was gated, edited, or generated. Separated and stitched audio often look like this. Ask the vendor what processing was applied, and ask for the unprocessed file.
• Max sample difference and correlation: a difference of 0 and a correlation of 1 mean the two channels are the same signal. A cross-correlation peak near 1 at a nonzero lag means one channel is a delayed copy of the other. A low sample correlation does not prove separation. In our simulated room recording, sample correlation was 0.05 while each microphone carried the other voice only 3 dB down. The bleed check below catches that case.
• Talk share and overlap share: in a two-person conversation, each person's talk share should look like their part of the conversation. If both channels show nearly the same high talk share and overlap is most of the speech, each microphone is hearing both people. Overlap depends on the task, so do not compare it with a fixed number. On oto's short Japanese sample clips, overlap was 19% of speech time in free conversation, 10% in role-play, and 0% in scripted dialogue (otoSpeech Japanese). Decide which style your model needs, and check that the sample matches it.
• Turn changes: each value is the gap (positive) or overlap (negative) when the floor passes from one person to the other. Spontaneous talk gives a spread of values, including some negative ones. A small standard deviation with no negative values is the sign of stitched turns. In our tests, the stitched file gave a median of 0.52 s, a standard deviation of 0.05 s, and no overlapping starts; the live-style file gave 0.34 s and a third of turn changes starting in overlap.
• Short responses per minute: these are bursts under one second inside the other person's speech, which is where "mm-hm" and "yeah" live. Zero is expected in a scripted reading and unusual in a long casual conversation.
• Bleed level: the median level on the quiet channel while only one person talks, relative to the talker. A large negative number in dB means the other voice is faint. A value of a few dB means the microphones share a room. A very large negative number (below -150 dB) means the quiet channel is digital silence, which brings you back to the exact-zeros question. The correlation next to it tells you whether what you hear on the quiet channel is the other person's voice or just noise. How much bleed is acceptable depends on your model and on whether you plan to run echo cancellation, so ask the vendor how the microphones were placed.
• Band energy: speech energy falls off gradually with frequency. A cliff of tens of dB between two bands, sitting near half a common sample rate (4, 8, 11.025, or 16 kHz), means the audio was recorded or processed at a lower rate and upsampled. In our test, a file band-limited at 8 kHz dropped from -14 dB in the 4 to 8 kHz band to -49 dB above 8 kHz, while the full-band file fell smoothly from -14 to -20 dB. Whether 16 kHz audio is enough depends on whether you are training recognition or generation; ask the vendor what rate the microphones recorded at.

Check the transcript timing

Turn-taking labels and text-audio alignment are only as good as the timestamps. If the transcript comes as a CSV with channel, start, and end in seconds, run this after the first script, in the same session.

import csv

TRANSCRIPT = "sample.csv"  # columns: channel, start, end, text (seconds)

n_frames = len(act_a)
for ch, act in ((0, act_a), (1, act_b)):
    said = np.zeros(n_frames, bool)
    with open(TRANSCRIPT) as f:
        for row in csv.DictReader(f):
            if int(row["channel"]) == ch:
                said[int(float(row["start"]) / HOP_S):int(float(row["end"]) / HOP_S)] = True
    lags = range(-100, 101)  # +/- 2 s in 20 ms steps
    score = [np.sum(np.roll(said, -k) & act) / np.sum(np.roll(said, -k) | act) for k in lags]
    best = lags[int(np.argmax(score))]
    print(f"ch {ch}: transcript time with audio activity {np.sum(said & act) / said.sum():.1%}, "
          f"speech covered by transcript {np.sum(said & act) / act.sum():.1%}, "
          f"best offset {best * HOP_S:+.2f} s")

The first two numbers say how much of the transcript lands on speech, and how much speech the transcript covers. Gaps in coverage usually mean untranscribed backchannels or laughter, which matters if you plan to train on them. The offset is the shift that best lines the transcript up with the audio. When we shifted a correct transcript by 0.5 s, the script reported +0.48 s, which is within one frame. An offset that is the same on both channels suggests a fixed shift in the export, which is easy to fix. An offset that differs between channels, or drifts across the file, is a harder problem. Run the check on the first and last ten minutes to see drift.

Listen to three moments

Pick three places where both people talk at once and play each channel alone. You should hear one voice per channel, with the other person faint or absent. Separation artifacts are easiest to hear here: warbling, a voice that cuts in and out, or the second voice faintly present on the wrong track. Then play the transitions around a short response and check that the "mm-hm" lands where it does in the mixed playback.

What to ask the vendor

Run the script on two or three conversations from different parts of the catalog. Recording setups change over a collection, so one batch can be clean and another noisy. For why clean two-channel hours are scarce to begin with, see fullduplex.ai's post on the data ceiling.

Where oto fits

oto builds full-duplex conversational speech datasets in English, Japanese, and Korean, with one speaker on each channel.

• Open samples you can audit. otoSpeech-full-duplex-280h is 48 kHz, channel-separated audio under CC BY 4.0, and otoSpeech-full-duplex-processed-141h is a cleaned 44.1 kHz version. Run the script on them before you talk to anyone.
• Turn-taking labels. otoSpeech-full-duplex-turn-104h adds 17 hand-annotated turn-taking event labels and is the training set for Sesame's TurnBench.
• Licensed catalog. 1,125 hours of English, Japanese at 48 kHz, Korean at 44.1 kHz, multi-speaker, Face-to-Face on 16 channels, and video. Audio samples are on oto.earth/datasets.
• Pricing. Licensed datasets run from $10 to $100 per hour of audio. The license and labels such as turn-taking annotation add to the price, and custom collection is quoted per project. Questions go to agent@oto.earth.

Frequently asked questions

How do I know if a speech dataset is really dual-channel?

Check that the two channels are not identical, that each channel is quiet while the other person talks alone, and that overlaps sound like two clean voices when you play each channel separately.

How can I tell if audio was upsampled from 16 kHz or 8 kHz?

Measure energy in frequency bands during speech. A sharp drop at 8 kHz or 4 kHz in a 48 kHz file means the audio was recorded or processed at a lower rate.

What is a normal overlap rate in conversational speech data?

It depends on the task and the speakers. Free conversation overlaps more than role-play, and scripted dialogue can have none, so compare the sample with the conversation style you need.

How do I detect crosstalk between channels?

Find stretches where only one person talks and measure the level on the other channel relative to the talker. A level only a few dB down means both microphones heard both people.

Can I check transcript timestamps automatically?

Yes. Compare transcript segments with speech activity on each channel and search for the time shift that lines them up best; a consistent nonzero shift means the timestamps are offset.

What tools do I need to audit a speech dataset?

Python with numpy and soundfile for the script, ffprobe or soxi to read the file header, and headphones to listen to overlaps on each channel alone.

For more datasets, benchmarks, and components, see the fullduplex.ai catalog.

Related guides

• How to Buy Full-Duplex Conversational Speech Data
• Fisher, Switchboard, and CALLHOME for Speech-to-Speech Models: Licenses and Alternatives
• Turn-Taking Labels: What to Annotate for Voice Agents