Repository navigation
Chunk Kokoro text up front and fix the Nemotron parity metric - #531
Merged
Merged
Conversation
The packed-LSTM models speak at upstream Kokoro's rate, so one 5 s pass holds about 56 tokens. Trying any text up to 128 tokens whole first now overruns for most two-sentence replies, and the discarded pass delayed the first audio: a 119-token reply streamed its first chunk after 273 ms instead of 131 ms. Text is now chunked the way speech-core does it, with a short tail merged only within the 64-token hard cap of the same profile. Text within the budget is still one chunk and one call. The long-text test accepts a peak of exactly 1.0, which the output clamp can produce, and the Kokoro page gives the current budget and cap.
The test measured how many batch-transcript words the stream also had,
so every word the batch pass misheard counted as a streaming loss. On
the re-exported Kokoro audio it fell under its 0.95 bar (0.923 and
0.949) with nothing missing from the stream: the two passes only
misheard "costs declined. Our" differently ("Ledecland" against
"Ledeckland", "Our" against "are"). Both transcripts are now scored
against the paragraph Kokoro spoke, and the stream must stay within
0.05 of the batch pass, about two words of it. The batch pass must
recover 85% of the paragraph, so a broken fixture still fails.
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The packed-LSTM Kokoro models (#518) speak at upstream Kokoro's rate, so one pass of the 5 s graph holds about 56 tokens. Two things still assumed the faster models:
synthesizeChunksfirst tried any text up to 128 tokens in one pass and discarded the result on overrun (Render long Kokoro text chunk by chunk #516). Most two-sentence replies now overrun, so their first audio waited for a pass that was thrown away.E2ENemotronHarshAudioTests.testStreamingMatchesBatchOnCleanLongUtterancehas failed onmainsince the Core ML re-export (recall 0.923 and 0.949 against a 0.95 bar). CI skips the E2E suites, so it didn't show.What changed
chunkTokenHardCap), the hard cap of speech-core's 5.0 s profile (Pin Kokoro's packed-LSTM graphs and size chunks for their speech rate speech-core#160), instead of within the 128-token input. The plan lives inKokoroTTSModel.chunkPlan, and three tests pin its budget and cap.mainthe two passes only misheard "costs declined. Our" differently ("Ledecland" against "Ledeckland", "Our" against "are").testLongTextIsNotClippedToOnePassaccepts a peak of exactly 1.0, which the ±1 clamp from Size Kokoro chunks and trimming for the packed-LSTM models #518 can produce.docs/models/kokoro-tts.mdgives the current budget (56) and cap (64). It also notes that a sentence longer than one chunk is split between words.Measurements
M5 Pro, re-exported
aufklarer/Kokoro-82M-CoreML,generateStream, median of 7:mainThe Nemotron parity paragraph is now 22.8 s of audio. On this branch, batch and stream both recover 0.949 of the spoken text.
Tests
KokoroTTSTests: 102 tests, 0 failures, 1 skipped (env-gated German parity). This includes the three new plan tests and the E2E tests against the re-exported models.E2ENemotronHarshAudioTests,E2ENemotronLangTagLeakTests,E2ENemotronStreamingASRTests): 18 tests, 0 failures, 2 skipped (English-only bundle not installed), in two full runs.testNoLangTagOnOverlappedSpeechreturned an empty transcript in one of five runs. Its 4.05 s two-voice mix is the same onmainand here, and its transcript varies from run to run ("Please verify your credentials and confirm the booking time", "Sidensify your credentials"). That flake predates this change and isn't addressed here.Related: #516, #518, soniqo/speech-core#160.