Conversation
|
Thank you for investigating this and submitting the patch. I compared it with #117 against the current master using the real Kuromoji analyzer. For the reported I am not adopting whole-string romanization here because it also changes Japanese/Latin boundaries. For example, in Hepburn normal mode, Closing this PR with thanks. This is not a claim that every normal-mode cross-token case is fixed: if you have another example that fails on current master, please open a focused issue or PR with the analyzer/version and failing test. A boundary-aware fix would be welcome. |
Fixes takuyaa/kuromoji.js#53
Problem
Reported against kuromoji.js:
kuroshiro.convert('座って', { to: 'romaji', mode: 'normal' })producessuwatsuteinstead ofsuwatte.I verified the tokenization side is correct: kuromoji (IPADIC) returns
座っ(reading スワッ, 連用タ接続) +て(テ), which is the proper analysis. The romaji issue is on the kuroshiro side: in normal mode each token's pronunciation is converted independently and joined, so the trailing sokuonッofスワッis romanized standalone (suwatsu) instead of geminating the next token's initial consonant (suwatte).Fix
In normal mode, join the per-token pre-conversion strings (
pronunciation || readingfor Japanese tokens,surface_formotherwise) first, then runtoRawRomajionce on the joined string.toRawRomajialready implements correct sokuon handling (verified:toRawRomaji('スワッテ')->suwatte). Spaced mode keeps per-token conversion, since the space-separated design is intentional there.Tests
座って(normal mode, hepburn) ->suwatte座って->suwatte,行って->itte; sanity checks (つじぎり,東京タワー) unchanged勝手に買っちゃったんだ) still passes