Jake Lawrence · Corpus Linguistics / Language Learning / Open Data · Field Data theme
An open effort to close the digital-tooling gap for the Ukrainian language, built as content-as-code datasets a serious learner can actually use. Six released threads over one spine: a lemma-based, coverage-graded frequency list of the top 12,000 lemmas ranked over 1.5 billion word tokens, a validated CEFR overlay, a derussification pass, paradigms and audio, a vocabulary-size test, and a text profiler. Published as versioned, hashed CSV and JSON plus an Anki deck, CC-BY, with a Python package, a hosted lemmatizer, and the list archived on Zenodo.
The language-tooling arm of the research program. A report on learning Ukrainian named the gap plainly: the frequency lists a serious adult learner needs are thin, often wordform-based, and sometimes quietly calqued from Russian, and the instruments that would let you measure progress (a vocabulary-size test, a text profiler, a lemmatizer) mostly do not exist as open, cited data. This series builds them and gives them away. The spine is a lemma-based frequency list for modern Ukrainian, derived as aggregate counts from the UberText 2.0 lemma frequency dictionary (lang-uk) and published as a versioned, hashed CSV and JSON open-data release whose coverage curve re-derives live in the browser and in CI. The measured finding: the top 1,000 lemmas cover about 61 percent of running word tokens and 5,000 cover 82 percent, lower than the cross-Slavic extrapolation usually quoted, a direct measurement of the inflection tax. Five further threads are released on top of it: a validated per-lemma CEFR overlay joining the only corpus-validated CEFR profile for Ukrainian (Synchak et al. 2025 / PULS, redistributed with the authors' permission) onto 6,403 of the 12,000 lemmas, which finds that a validated level tracks frequency only weakly, so a frequency rank is not a proficiency level; a derussification pass fenced as an unvalidated prescriptive signal; paradigms and pronunciation audio from open Wiktionary and Wikimedia sources; a self-scoring vocabulary-size test of 60 real lemmas against 60 constructed pseudowords; and a text profiler, Ukrainian-first, that reads a pasted text against a validated level or a class's own word list without anything leaving the browser, published with the validation run against 29 level-marked course texts as it came out. Around them sit a crosswalk of the two published Borodin lexical minimums against the validated profile, a 142-expression multiword table (a lemma list cannot hold a phrase), a hosted lemmatizer matched to the list's headword convention, and an installable Python package carrying the validated levels and none of the predicted ones. The list is archived on Zenodo with a concept DOI; three further deposits are built and held pending an attribution decision, which the surface says rather than hides.
Two Ukraine field-data studies from opposite ends of the state's legibility. The Legibility Gap measures what the wartime state cannot read about its own damaged homes; Ukrainian Frequency measures what a learner has to read to reach the language itself, and builds the open list that makes the reading tractable.
Two field-data series in the same content-as-code, hashed-open-data mold, each shipping a versioned release whose numbers re-derive live. Pingree Grove publishes one village's rules as open data; Ukrainian Frequency publishes the core of a language as open data, both built to be forked.
The list is a release, not a page: a coverage-graded open frequency list that gets new versions, a CEFR crosswalk, and a profiler. Subscribe and you hear about the next version, and about the data asks when they are answered. It is the one list for the whole site, so you will get the rest of the work too. No schedule, easy out.
Back to the release table · See it on the network · Machine-readable catalog
I build the AI tools and automations a team adopts, and the full-stack apps and data pipelines under them, and ship them to production. Tell me the problem in a sentence and I'll give you an honest read on fit within a day.
Work with me →