How translation actually works

If you've ever played the playground game Telephone, you know what happens. A message whispered from ear to ear arrives at the end of the line transformed, sometimes comically, sometimes chillingly. The same thing happens when you push a phrase through a chain of translators. But why?

The usual answer, “each translation loses something,” is true but uninformative. It doesn't tell you what is lost, or how, or why some phrases survive the journey remarkably well while others dissolve into paraphrase within two hops. To answer those questions we have to look at what a modern language model actually does when it translates.

It doesn't match words against a dictionary. It doesn't parse grammar into trees. It does something stranger: it turns every piece of language into a point in a high-dimensional space, and translation becomes a kind of navigation through that space.

This article explains that space, how it's built, and why (once you can see it) the drift in our game becomes almost inevitable.

The first thing to understand is that modern models don't think of words as strings. Internally, every word (every subword fragment, actually) is a vector, a list of numbers. In our demo, 1,536 of them. These vectors live in a space with one dimension per number. You can't draw that space. Nobody can. But you can project it down to two dimensions and see the shape.

Here's a projection of a few dozen English words. Hover to see each word's nearest semantic neighbours:

Notice that the model was never told “dog” and “wolf” are similar. Nobody labelled these clusters. The geometry emerged during training, as a side effect of the model learning to predict which word comes next in billions of sentences. Words that appear in similar contexts end up with similar vectors. Meaning, it turns out, leaks out of context.

This is the insight Mikolov's team made famous in 2013 with word2vec, but modern transformers take it much further: they embed not just words but entire sentences, paragraphs, and, crucially for us, passages in any language the model has ever seen.

Here's where things get properly strange. When a multilingual model is trained on text from many languages, it doesn't build a separate space for each language. It builds one shared space, and words that mean the same thing across languages end up in roughly the same region.

“Dog” sits near 犬 sits near perro sits near chien. All of them, effectively, share coordinates:

This is not a trick. Nobody trained the model to put translations near each other. It emerges because words that mean the same thing tend to appear in statistically similar contexts across languages: paired in parallel corpora, surrounded by similar neighbours, used for similar purposes. The training objective quietly discovers that dog, perro, and chien are really just three labels for the same location in meaning-space.

Once you accept that, translation stops being a dictionary lookup and starts being a kind of geometry.

If meaning is geometry, we need a way to measure distance. The standard tool is cosine similarity: the cosine of the angle between two vectors when you draw them from the origin.

It's the right choice because embedding vectors encode direction more than magnitude. Two related ideas can have very different “lengths” but point roughly the same way. Cosine asks: are these two arrows headed in the same direction?

In real embedding spaces, cosines rarely go below zero. Most pairs of English sentences land somewhere in the 0.3–1.0 range, because nearly all natural text shares some structural features. That's why Lost in Translation maps cosine 0.3–1.0 onto retention 0–100%, which is where the interesting variation actually happens.

Now we can describe what actually happens when an LLM translates. Given a source phrase and a target language, a useful simplification is:

1. Embed the source phrase. Find its point in meaning-space.
2. Generate a sequence of tokens in the target language whose embedding lands as close as possible to that point.
3. Among all valid completions, pick the one that also reads naturally to a native speaker.

Step 2 is the interesting one. The model is effectively looking for the nearest neighbour in the target-language region of embedding space:

This is also why LLMs can translate between language pairs they were never explicitly trained on. As long as both languages sit in the same embedding space, a path exists.

So if translation is a short, principled walk across embedding space, why does our game produce such comical results?

Three reasons compound:

• Quantisation error.The target language doesn't have a word at exactly the right coordinates. The model picks the nearest available one, close but not identical. That tiny error is baked into the next hop.
• Idiom collapse. Figurative phrases like “raining cats and dogs” have embeddings that encode the idiomatic meaning (heavy rain), but a literal translator may produce text that, when re-embedded, looks like the literal meaning (falling animals). The second model sees no idiom there, and the chain diverges.
• Register drift.Each language has its own conventions for formality, politeness, and rhythm. Honouring them nudges every translation a small step away from the original's emotional position, even when the literal meaning is preserved.

Visualised, a chain of translations looks less like a clean pipeline and more like a drunken walk:

This is the thing the retention score measures. Each hop gets a cosine against the original phrase. As the walker wanders further from origin, the cosine drops, and the retention percentage falls with it. The per-hop drift colours the connectors between cards: a green hop lost very little; a crimson hop took you somewhere new.

That's the theory. The tool below does all of this in real time: every translation embeds, every step gets scored, every hop colours its connector. Pick a phrase known to degrade (Shakespeare, idioms, puns), build a chain of languages, and watch the walk.

Two honest caveats, in the interest of not hand-waving:

The 2D projections are lies.They preserve cluster structure but lose the real geometry. Two points that look adjacent in 2D may be far apart in the full space, and vice versa. The intuitions still hold, but the exact distances don't transfer.

Embeddings aren't the whole translation story.Modern LLMs use attention and causal generation, not just nearest-neighbour lookup. The “translation is navigation” framing is a useful metaphor for why cross-lingual translation is possible, not a complete account of the algorithm. The real mechanics involve predicting one subword at a time, conditioned on both the source passage and every token produced so far.

But for our purposes (understanding where retention comes from, why back-translation works, and why the drift in our game compounds the way it does) the geometric picture is enough.