My best number was a word list read back to me
A month of work on the Ukrainian frequency list ended with a DOI, a live lemmatizer, a correction sent to the authors of the profile I lean on, and a headline result that dropped nine points once I deleted the feature that was cheating. The improvements that mattered were the ones that let the project refuse itself.
The Ukrainian frequency list has been the longest-running thing I maintain, and this month it finally got the boring infrastructure it deserved. It has a DOI now. The lemmatizer that matches its headword convention is deployed and answering requests. The validated level map covers more words than it did. And the number I was proudest of, the one I had already put in writing to the researchers whose data I build on, turned out to be nine points too high. Deleting the feature that inflated it was the best work in the batch. Everything useful I shipped this month was some version of the same move: giving the project a way to say no to me.
The join was throwing away words it should have kept
The list carries CEFR levels by joining onto the one corpus-based, expert-validated vocabulary profile that exists for Ukrainian. The join is part-of-speech aware, because a spelling can be two words and only one of them may be A1. That caution had a cost I had not measured.
The two sources do not label parts of speech the same way. Universal Dependencies calls який, цей and той determiners. The profile calls them pronouns. It files перший and другий as numerals where UD sees ordinals, keeps an indeclinable-word class for можна and треба, and treats substantivized adjectives like військовий as nouns. My join saw the tag mismatch, decided it was looking at unrelated homographs, and dropped the row.
That was 107 words, and 34 of them sit in the top thousand. який is rank 12. цей is rank 19. Ten explicit cross-tag rules now recover them, and each recovered row records which rule caught it, so anyone who wants the strict join can filter it back out in one step. The validated overlay went from 6,284 lemmas to 6,391. Forty-seven genuine homographs are still dropped, which is the right outcome.
Then the model got worse, correctly
About 5,600 of the top 12,000 lemmas carry no validated level, so there is a gap-fill model that predicts candidate levels for review. Its job is to be honest about how little frequency and word shape can tell you. Held out on the profile's own labels under grouped folds, where a stem, an aspect link, a spelling variant and a derivational family can never straddle training and test, it was reading 49.6 percent exact and 92.3 within one level.
Then I looked at what the model was leaning on. Its strongest single weight was a flag for whether the lemma appears in two published beginner minimums, and that flag alone was worth 9.4 points of exact match. It fires on 1,499 of the validated labels. It fires on 84 of the lemmas actually being predicted.
So that flag was a label wearing a feature's clothes. It scored beautifully on the words the profile had already graded, and it did nothing for the words I needed graded. The flags are still computed and still reported, because the size of that gap is itself a finding. They are out of the shipped model.
What the honest number looks like
Retrained without the flags, the model reads 40.2 percent exact and 82.9 percent within one level, mean absolute error 0.79, against a rank-band baseline of 37.4 and 75.2. Three points of exact match over a floor that needs no model at all.
That is a worse headline and a better instrument. Of 4,210 candidate rows it will now commit to 1,750, roughly a thousand fewer than before, and the ones it abstains on are the ones it should never have been confident about. A shorter list that a reviewer can trust beats a longer list that quietly includes its own training labels.
I had already sent the inflated figures to the authors of the profile. So I wrote again with the correction, both halves of it, the coverage that grew and the accuracy that shrank. Sending a number that makes your work look worse is not fun. Leaving a wrong number in someone else's inbox is worse.
A result nobody can check is not a result
The list is now archived on Zenodo with a concept DOI, 10.5281/zenodo.22823134, which means there is a citable, version-independent identifier that resolves whether or not my site is up. The first version is v0.1.0 under CC-BY-4.0.
The part I care about is the verification. After the mint I pulled the archived checksum manifest back down from Zenodo and compared it against the files in the repo. Five files, and five matching hashes. That check matters because a DOI is a promise that a specific set of bytes is what a paper cited, and the only way to keep that promise is to confirm the bytes made it there intact rather than assume the upload worked.
The published data also installs as a Python package now, with the levels the profile validated and none of the predicted ones, because a predicted level in a package that looks authoritative is a level someone will cite as fact.
The convention had to become executable
A frequency list is only joinable if you can reproduce its headword convention. Mine was documented in prose: VESUM headwords, the apostrophe written as U+2019 so that сім’я in the list is the same string as сім’я out of a tagger. Prose only describes a convention. Anybody reimplementing mine from that description will get the apostrophe wrong, because several characters look like an apostrophe and Ukrainian happens to use one of them.
So the convention is a service now. It runs LanguageTool's Ukrainian pipeline over a pinned VESUM dictionary, normalizes whatever apostrophe you send, and returns lemmas in the list's own spelling. It keeps no text. Analyses go out and the input is discarded, with a hash prefix and a count reaching the logs.
The deploy is the load-bearing part, and it is the one thing in this whole batch I would defend hardest. The workflow will not point production at a new build until that build has round-tripped a deterministic sample of the list and matched at least 99 percent of the lemmas and resolved every apostrophe case, all inside a two-second budget for 5,000 characters. The current deploy passed at 99.8 percent, 435 of 436 lemmas, five of five apostrophes, 421 milliseconds. If a dictionary version bump ever breaks the join, the deploy fails and the old service keeps serving. A silent dictionary change is exactly the failure that would poison every downstream table, and it is now a red check instead of a mystery.
Three datasets I am deliberately not publishing
The same month, three finished deposits are sitting on hold and will keep sitting there.
Each one carries per-lemma levels derived from the profile. A co-author granted permission to redistribute those levels, and I asked, in the same note, for the exact attribution string and license they want on redistributed artifacts. That has not come back in writing yet. So the frequency list, which is mine, is deposited, and the three that lean on someone else's expert labeling are held with the reason recorded on each row.
This is the beat where I would normally tell you to ship it anyway. I am holding instead, and the mechanism is why. A DOI cannot be unminted. The deposit script refuses any row flagged as held unless it is passed an explicit override, and the workflow that mints from a button never passes it, so releasing a hold has to be a reviewed commit that a human writes on purpose. If the only thing standing between a permanent public artifact and someone else's unreviewed attribution is my own memory, then it is not standing between them at all.
What happens next, and what it is waiting on
Four things are queued and one of them is not mine to schedule.
The held deposits move when the attribution string arrives. I asked for it in the same note that carried the correction, so the ask and the erratum are in one thread, which is either good manners or a way of letting one silence block two things. If it does not come back, the three datasets stay where they are and the reason stays on the row. I am not going to guess at wording that governs how somebody else's labeling gets credited.
The usage tables owe a re-parse. The join fix added B1 verbs the corpus slice was never cut for, so those rows are empty and the README says so rather than interpolating them. That is a re-run, not a decision, and it happens before the next refresh.
The upstream worklists are written and unsent, which is the least defensible item here. There are 21 verbs where the profile, Wiktionary and VESUM disagree about aspect, 4,209 validated lemmas with no Ukrainian Wiktionary page ranked by how much each one would be worth, and 34 verb pages that never name their attested partner. Generating a worklist is not contributing. Sending it is, and I have not.
The maintenance statement is still a draft, because publishing it means naming a cadence I can then be held to. The refresh is quarterly and the next one is December. If the statement is going to commit to that, it should say so before December rather than after.
What the gates add up to
None of this month's work made the project bigger in a way you could screenshot. The coverage number went up by 107 words. The accuracy number went down by nine points. The interesting additions were four different ways for the work to stop me: a checksum comparison that would have caught a corrupted upload, a deploy gate that refuses a service whose lemmas drift from the list, a registry hold that a button cannot override, and an ablation that took my best feature away.
A dataset gets trusted because of what its own machinery will refuse to publish. That is the whole thesis, and it is why I count the erratum as this month's real deliverable.
Get the next one
An occasional note when something genuinely new ships here — essays, free tools, projects. No schedule, no filler, easy out.
Need something like this built?
I design and ship AI tools, full-stack apps, and data pipelines — end to end, to production. Tell me the problem in a sentence; I'll give you an honest read on fit within a day.
Work with me →