Lexerror

I cannot speak — Я не могу говорить, but I repeat myself.

Translation is such a commonplace quirk of the globalized era that we sometimes lose sight of the dizzying precipice over which this tightrope is strung — and tightrope it is, for a minor error in translation can send us hurtling into the abyss. This canyon is incomprehensibility, it is Babel, it is being unable to understand one another. Translation must perfectly reconstruct the meaning of the words from the source language in words from the target language. Skilled translators do not just substitute Russian words for English ones, but reach at some core meaning that lies under both languages, making a brief foray through this nonverbal wilderness before reemerging from the thickets in the target. Translation is transportation, taking ideas from one language into another. The mystical intuitions behind this process are finally being unraveled by machines, and we have some (hopefully) interesting things to tell you.

Imagine you are in a great maze constructed of many different rooms, connected by corridors (in computer science we call this maze a graph) and you begin in the absolute center of the maze, a great big room with “NULL” written on the ceiling. Thousands, possibly even hundreds of thousands of halls lead out of this room — it is really very extremely big. Not wanting to walk for hours, you make towards the nearest hall, a passage with “I” stenciled over its entrance in plain lettering. You have some destination in mind, even if you do not totally know it yet. Somewhere in this maze is a room that contains exactly what you want, and you only need to walk through the correct series of hallways to find it.

At this point I should tell you something special about this maze in particular: there are no crossing paths. Although many halls branch out of every room, there is only one way to get from “NULL” to a given room in the maze. If you find yourself on the wrong path you must backtrack until you can go down the right one. Some two rooms might be very close together, but they are not the same. In computer science we call this a tree, and “NULL” the root of the tree. All roads lead to Root, as the saying goes.

You arrive at the next room, you can see “I” written above you, like the sign on the hall you just walked through, which, incidentally, is clearly marked as the way back to the “NULL” room. There seem to be just as many halls leading out of this room as the previous one, but you are too busy to count, as you make your way eastward towards one hall in particular with “cannot” stenciled above it in the same nondescript font as before. This hall is very long, and twisted ever so slightly, so that eventually you cannot be certain you are not walking on the ceiling now, and staring up at the floor. You feel fully inverted. After an indeterminate amount of time spent walking, you emerge in a third room, which has “I cannot” written in the sky. Hopefully you are beginning to notice a pattern. You have only one more trek ahead of you, a brief jog down the hall labeled “speak” that leads you to your goal. In the room marked “I cannot speak” you finally stop, having said exactly what you want to say. The path back to “NULL” is marked — you could backtrace your steps if you wanted to, but why would you want to? You are done.

This is speaking. You have not left the maze, you have not run backward, all you have had to do is interpret the words that point towards your meaning, and know ahead of time which direction you want to be moving in. You are speaking about yourself, so you need to say “I” etc. The structure of the maze has guided you to where you need to be. The words are English, but so are the walls of the rooms and halls you have passed through. The layout of this gargantuan maze is something you subconsciously learn as you learn the language itself. You are an adept pathfinder in this place without knowing it.

Now, move toward the center of the room and hold very still. Don’t move a muscle, because I’m going to pick up the entire maze complex at once from around you, removing all the walls and ceilings and leaving you, alone. Look around — you are in a wilderness. No markers, unless you can tell where “NULL” is by the way everything seems to center around it, many miles away. Perhaps you can feel this semantic void slowly revolving around “NULL”, perhaps not. You haven’t moved, have you? Don’t worry, I won’t leave you here. Down comes another maze, with new corridors and rooms. You are covered in meaning once more.

This room is different — it smells faintly foreign, and there is no writing on the ceiling. The walls are also differently proportioned — there is one corner of the room that previously fell outside it, and the far wall is 40 feet closer than it was before. You are in the same place, technically, but you have no idea where you are in this new maze, or how to get home. You need to process this, you need to find “NULL”. Fortunately, you see one of these new passages is marked as the way backwards, and so you move through it, hoping to find your way back. In the next room, you turn around briefly to check the label of the hall you have just come from: “говорить”, it reads.

You move backward through the maze, passing in reverse through halls marked “могу” and “не”. Your final journey lands you back in the center, with the giant “NULL” etched above you in its comforting all caps. Turning around, you note the stencil of the corridor you have just emerged from. “Я”, it says. You may or may not know that this is the Russian first person pronoun, just as “I” is the English first person pronoun, but you do not need to. If you write down the halls you have passed through in reverse order — “Я не могу говорить” — you will have written the instructions for getting to where you have just come from. Every sentence is a list of instructions, a “ten paces northwest, five paces east” style treasure map to guide you through this great linguistic labyrinth towards your intended meaning.

The two sentences you have walked over are nothing alike — they lead through different rooms in different mazes, and in fact have a different number of rooms each — but they both lead you to the same physical place in the world. Follow either one, and take away the maze, and you will be in the exact same spot in the wilderness. Put down the new maze and by finding your way home you figure out how to tell speakers of that language where you have just been. There is some inherent meaning in your global position, then, whether a maze is present or not, something about where you are relative to “NULL” is itself a statement. Every step changes your meaning, maybe drastically, maybe very little. As you move through this strange wilderness, you place yourself like a marker on some idea or thought or code or whatever that means something with or without language. The maze is how we define waypoints, it is the language that we speak. It is manmade, artificial, to a certain degree arbitrary. The sentences are just history strings, keeping track of where you’ve been, how you got here, so your journey can be replicated and others can find this place.

This strange wilderness is not just a thought experiment — it actually exists. The process of tracing one sentence, changing mazes and backtracing another sentence is exactly how state of the art machine translators do their business. Of course, there are quintillions of rooms in a maze, and explicitly defining all of them would take more time and space than we have access to, so instead of drawing the walls on a map, we just measure the length of the hallways. Machines consider words as vectors: arrows with a specific direction and a specific length. It turns out that every hall marked “cannot” in the English maze is the exact same length and runs in the exact same easterly direction. Whether it is the first hall you run down or the twentieth in your sentence makes no difference — every copy of that corridor is exactly, mathematically the same. It will always wind up placing you so many miles to the east of its entrance. (Alright this is not exactly true — relative location in the sentence does matter in practical implementations, and you can’t just sum up all the words in your sentence at one go. However, this is a minor detail to the conceptual framework at play, and I won’t go into specifics of how machines define words as vectors.)

Instead of looking for a passage marked “cannot”, a machine will recall from its iron mind that “cannot” means to move 20,000 paces to the east, and will do this blind so to speak, not seeing the walls of the maze around it. Sentences then become simple mathematical problems, following a list of instructions. Translation is not much more complicated, just a matter of finding the instruction list that will lead you back out. Our imaginary maze was two dimensional, most machines map out wildernesses of around 512 dimensions, other than that the process is basically the same.

Machines learn these vectors by reading a massive corpus of text and scoring each word by its relative closeness to other words, and they do a damn good job of it, too. Take the vector for the word “king”, subtract the vector representing “man” and add the vector representing “woman” and the resulting value is the same as the vector for the word “queen”. This kind of linguistic algebra is another evidence for the existence of this semantic void, this wilderness hidden behind all languages and wherein all languages intersect.

The most gripping part of all of this for me, though, is the thought of a perfect language. A divine language with no sentences, and billions of words, each one a single vector-hallway leading towards an exact meaning. Imagine, every sentence a single vector. Imagine being able to speak this tongue! Imagine, instead of the long, winding, 1,812-step path this essay traces, a single great and terrible ARROW of a word, pointing directly at my meaning all at once.

The thought unsettles me to no end.