Mati, ali je model že konvergiral?
A Slovene micro language model (~26M parameters) trained from scratch - custom BPE tokenizer, pretraining on public-domain Slovenian literature and Wikipedia, then specialized in the prose voice of Ivan Cankar (1876-1918), with a plain->Cankar style-transfer stage trained on synthetic parallel data.
Live: cankar-gpt.nextgen-solutions.xyz - samples, eval numbers, and a plain-language explainer.
Status: Phase 4 complete - CankarGPT v1 (26.3M params) specialized on Cankar, held-out BPB 1.4508 (TinyCankar 2.2227 -> base 1.5056 -> v1 1.4508). Corpus, tokenizer (v8192, 142.77M tokens), eval harness (BPB + style classifier ROC-AUC 0.993), and the public site are all live. Next: synthetic style pairs (Phase 5). See ROADMAP.md for the full plan, risk register, and budget (spoiler: the whole thing costs about one dinner in Ljubljana).
Deliberately both, in order: the from-scratch model demonstrates the full stack (tokenizer -> pretraining -> SFT -> serving) on consumer/rented hardware; a later GaMS fine-tune on the same dataset demonstrates practical delivery with modern open models. One project, two claims. Details in the roadmap.
Corpus-building scripts are published here (the merged corpus itself is not redistributed - licensing note in ROADMAP Phase 1):
uv sync
uv run cankar corpus crawl-wikivir --category "Kategorija:Ivan Cankar" \
--out data/corpus/cankar.jsonlSee CONTRIBUTING.md (setup, commit conventions) and SECURITY.md (private vulnerability reporting).
Training text comes from public sources; per-document provenance (source URL,
identifiers, retrieval metadata) is recorded in the works registry (registry/)
and shard manifests:
- Wikivir / Slovene Wikisource (sl.wikisource.org) - hand-transcribed public-domain literature, courtesy of the Wikisource contributor community.
- Digitalna knjiznica Slovenije - dLib.si (National and University Library of Slovenia) - digitized public-domain editions. Only works carrying dLib's public-domain rights mark are ingested, and dLib.si is cited here as their source, per the dLib terms of use.
- Wikipedia (Slovenian) (sl.wikipedia.org) - general-language text from the official dump; CC BY-SA 4.0, attributed to Wikipedia contributors (dump filename, sha256, and license are recorded in the shard manifest).
The merged training corpus is not redistributed (CC BY-SA share-alike vs. public-domain mixing - see ROADMAP Phase 1). Any published artifact built from this data (HF datasets, model cards, demo pages) repeats these credits.
Code is MIT. The merged training corpus is not redistributed; model weights will be licensed at Hugging Face release time.
Built by Matic Korošec - training logs and write-ups land on the site as the project progresses.
AI pastiche for educational purposes. Not affiliated with any Cankar institution, memorial, or the Vrhnika tourist board. Cankar's works are in the public domain.