tinyfacts playground

The tinyfacts models are a series of models trained on the tinyfacts dataset, a synthetic dataset built only using allowed forms of the 1000 most common English words originally used by Randall Munroe in the xkcd coming Up Goer Five, and inspired by the tinystories dataset and model series. These models use a very restricted vocabulary to achieve speaking coherent, if simple, English with less than 1M parameters and very few training tokens. They are raw completion models, not finetuned for instruction following. You can experiment with them here, entirely in your browser, as inference is handled by onnxruntime-web. Blog posts:

• The Big Learning Set for Big World Helpers
• Training Not So Large Language Models on My PC