Crumbs — a tiny language model
Every large language model is, at heart, a machine for one task: given some text, predict what comes next. This is the smallest honest version of that idea — an n-gram model. Feed it text below; it counts which words tend to follow which, then predicts. The model is built entirely in your browser. Nothing leaves this page.
The model counts which tokens follow which. More text → better predictions.
Click any bar to commit that token to the context — you become the sampler. Or hit autocomplete and let the model choose, weighted by probability.
A real LLM predicts the next token from billions of learned weights and a context of thousands of tokens. Crumbs predicts from raw counts and a context of one or two words. Same shape, wildly different scale — but the core move is identical: turn “what comes next?” into a probability distribution and pick from it. Everything else AI does is built on this one trick, scaled up.