The Case That A.I. Is Thinking
Kanerva’s book faded from view and Hofstadter’s own star faded, except when he occasionally raised his head to criticize a new AI system. In 2018, he wrote of Google Translate and similar technologies: “There is still something deeply missing in the approach, which translates into a single word: understanding.” But GPT-4, released in 2023, produced Hofstadter’s conversion moment. “I’m amazed by some of the things the systems do,” he told me recently. “This would have been inconceivable just ten years ago.” The most fervent deflationist could no longer deflate. Here is a program that knew how to translate as well as an expert, make analogies, improvise, generalize. Who were we to say he didn’t understand? “They do things that are a lot like thinking,” he said. “You could say that they are thinking, just in a somewhat foreign way.
LLMs seem to have within them a “seeing as” machine. They represent each word by a series of numbers indicating its coordinates – its vector – in a high-dimensional space. In GPT-4, a word vector has thousands of dimensions, which describe its nuances of similarity and difference from all other words. During training, a large language model changes the coordinates of a word each time it makes a prediction error; words that appear together in texts are close together in space. This produces an incredibly dense representation of uses and meanings, in which analogy becomes a matter of geometry. In a classic example, if you take the word vector for “Paris”, subtract “France”, then add “Italy”, the other closest vector will be “Rome”. LLMs can “vectorize” an image by encoding what it contains, its mood, even the expressions on people’s faces, in enough detail to redraw it in a particular style or write a paragraph about it. When Max asked ChatGPT to help him with the park sprinkler, the model wasn’t just texting. The photograph of the plumbing was compressed, with Max’s prompting, into a vector that captured its most important features. This vector served as an address to call nearby words and concepts. These ideas, in turn, led to others as the model provided insight into the situation. He wrote his response with these ideas “in mind.”
A few months ago, I read an interview with an Anthropic researcher, Trenton Bricken, who worked with colleagues to probe the inside of Claude, the company’s series of AI models. (Their research has not been peer-reviewed or published in a scientific journal.) His team identified sets of artificial neurons, or “features,” that fire when Claude is about to say one thing or another. Features turn out to be like volume buttons for concepts; Ride them and the model will talk about little else. (In a sort of thought control experiment, the item depicting the Golden Gate Bridge was displayed; when a user asked Claude for a chocolate cake recipe, the suggested ingredients included “1/4 cup dry fog” and “1 cup warm sea water.”) In the interview, Bricken mentioned Google’s Transformer architecture, a recipe for building neural networks that underpin major AI models. (The “T” in ChatGPT stands for “Transform.”) He argued that the mathematics at the heart of the Transformer architecture closely approximated a model proposed decades earlier, by Pentti Kanerva, in “Sparse Distributed Memory.”
Should we be surprised by the correspondence between AI and our own brain? LLMs are, after all, artificial neural networks that psychologists and neuroscientists helped develop. What’s more surprising is that when the models practiced something by rote – predicting words – they began to behave in a brain-like manner. Nowadays, the fields of neuroscience and artificial intelligence are intertwined; brain experts use AI as a kind of model organism. Evelina Fedorenko, a neuroscientist at MIT, has used LLMs to study how the brain processes language. “I never thought I would be able to think about this sort of thing in my lifetime,” she told me. “I never thought we would have good enough models.”
It has become common to say that AI is a black box, but the opposite is arguably true: a scientist can probe the activity of individual artificial neurons and even modify them. “Having a functioning system that instantiates a theory of human intelligence: that’s the dream of cognitive neuroscience,” Kenneth Norman, a neuroscientist at Princeton, told me. Norman created computer models of the hippocampus, the region of the brain where episodic memories are stored, but in the past they were so simple that he could only give them rough approximations of what might fit into the human mind. “You can now give memory models the exact stimuli you give to a person,” he said.
