In my post The Big Learning Set for Big Word Helpers I explained how I planned to generate a training dataset using the 1000 most common English words from the original Up Goer Five xkcd comic, and how I had started doing so with a custom agent provided with tools to check and refine its own generations. Here I’ll go over the continuation of those efforts, up to a live demo of a language model (“large” at this point may be a misnomer) trained on this dataset!

On November 12, 2012, Randall Munroe’s famous xkcd comic published Up Goer Five, a blueprint and explanation of the Apollo V rocket written using only the 1000 most common words of the English language (as he estimated them). Later on, on November 24, 2015, came out Thing Explainer, an entire illustrated book of similar explanations for other objects and concepts. The “only the most common 1000 words” style of writing sounds sometimes stilted, sometimes a bit funny, but these texts certainly prove that it’s enough to talk virtually about anything.

In the age of LLMs, would it be possible to have a training set built only on the most common 1000 words of the English language?

Let’s try.