Problem

The frequency lists a serious adult learner of Ukrainian needs are thin, often wordform-based, and sometimes quietly calqued from Russian, and the instruments that would measure progress (a vocabulary-size test, a text profiler, a lemmatizer) mostly did not exist as open, cited data.

Constraint

Every source carries its own license and its own headword convention. The only corpus-validated CEFR profile for Ukrainian could be redistributed with its authors' permission; a multiword resource could be read as validation but never shipped. Nothing pasted into the profiler may leave the browser.

Decision

A three-tier release: a committed, cited source snapshot, a pure builder, and frozen hashed files whose coverage curve re-derives live and in CI. Validated levels ship; predicted levels never do. The CC-BY-NC-SA resource is read one way as validation and walled out of the release by a fail-closed test.

• Predict CEFR levels with a model and ship them as data

  A validated level tracks frequency only weakly, so a predicted level would read as a proficiency level it is not. The Python package carries the validated levels and none of the predicted ones.

• A wordform list

  Ukrainian inflects. The top 1,000 lemmas cover about 61 percent of running tokens, lower than the cross-Slavic figure usually quoted, and a wordform list would have hidden that tax.

What shipped

The list, the CEFR alignment, the derussification pass, audio and paradigms, the vocabulary-size test, the text profiler, a crosswalk of two lexical minimums, a 142-expression multiword table, a hosted lemmatizer with a round-trip as its deploy gate, an installable Python package, and a Zenodo DOI.

What I'd do differently

The join to the validated CEFR profile silently failed on lemmas with apostrophes, which Ukrainian has in ordinary words, because they arrived as HTML entities. Every tier had to be rebuilt on the corrected join (#2358), and the paradigm join had the same bug a day later (#2360). Decode at the boundary, then diff the join counts against the source.

If you ask me about this

How do you handle a license you cannot ship under?

Read it one way. PARSEME is CC-BY-NC-SA, so it validates the multiword table and never enters the release, and a fail-closed test checks that on every run.

Why host a lemmatizer instead of shipping one?

The dictionary it needs cannot leave the server. The endpoint normalizes, rate-limits, caches by content hash, retains no text, and a round-trip against the list is the deploy gate.

What did you do when a figure changed?

Rebuilt every tier on it and brought the note to the profile's authors up to the shipped model (#2355). The prose tier is drift-guarded like the data tier (#2411), so a stale number fails the build.

Evidence

1. Ukrainian Frequency: a lemma-based open frequency list (research field data)

   #2054 · 2026-07-14 · feature

2. Ukrainian Frequency thread 02: CEFR alignment (honestly scoped)

   #2200 · 2026-07-22 · feature

3. ukrainian-frequency: validated CEFR overlay (thread 02) from PULS

   #2223 · 2026-07-23 · feature

4. Ukrainian Frequency: join the author-shared Borodin A1 lexical minimum (the closed seam)

   #2258 · 2026-07-27 · feature

5. Ukrainian frequency: decode the apostrophe entities in the PULS join and rebuild every tier on it

   #2358 · 2026-09-13 · feature

6. Ukrainian Frequency: the text profiler (thread 06)

   #2359 · 2026-09-13 · feature

7. Ukrainian Frequency: crosswalk the two lexical minimums against the validated CEFR profile

   #2362 · 2026-09-13 · feature

8. Ukrainian frequency: ukvocab package, Zenodo tooling, the hosted lemmatizer with its deploy path, and the upstream worklists (WO-1 to WO-5)

   #2366 · 2026-09-14 · feature

9. Ukrainian frequency: the multiword table (B1 extension, stage 3)

   #2378 · 2026-09-17 · feature

10. Ukrainian Frequency: make the prose tier drift-guarded like the data tier

    #2411 · 2026-09-19 · feature