The Nobel Prize in Physics: An Imitation of Thought

On the 2024 Nobel Prize in Physics

A thinking machine, imagined by an artificial intelligence.

Can a machine think? The question is a vague one, not least because we have no proper definition of what “thinking” actually is. Stretch it too far and anything - even a simple thermostat - can be accused of thinking. Define it too narrowly, and even some humans might be denied the ability.

Since the emergence of computing, however, people have been asking if they might someday begin to think. Ada Lovelace, one of the first to seriously consider the possibility, thought it unlikely. Computers, she wrote in the 1830s, could not “originate anything”, and would merely do “whatever we know how to order”. Machines could only do what they were programmed to do, she argued, and no thinking was required for that.

But what, others later asked, if we programmed a computer to learn? Could it then learn to reason for itself, and perhaps even to program itself? And if we could do that, could a computer learn to be creative, and perhaps to produce some original thought? Might it then be said that a computer - a mere instruction taker - had learned to think?

The great Alan Turing took up the question in the 1940s and wrote a famous paper on the topic. An operator, he noted, could very easily inject an idea into a machine and watch it respond. But this, he said, was like hearing a piano string struck by a hammer: the response would soon die away, just as the string soon fell into silence.

Such a machine could hardly be said to think. But possibly, he went on, thought might be more like an atomic pile sitting just below the level of criticality. Left alone the pile is inert and perfectly safe. If one fires a neutron at the pile it will, briefly, trigger a flash of disturbance. But increase the size of the pile by just enough and that neutron will trigger a different response: the disturbance will propagate and multiply, and if left uncontrolled the pile will be destroyed in a great explosion of energy.

Intelligence, he mused, might function like this. Assemble a machine of sufficient complexity and an idea may act more like a neutron than a hammer. Instead of dying away, that single musical note might explode into a symphony of thought and creativity. Human brains, he reasoned, must be above this critical level, and those of animals somewhere below it.

If we could assemble a machine of sufficient complexity, Turing concluded, we could program it to learn. And once a machine could learn, it could alter its own rules of operation, adjusting them to the information it took in and thus producing quite unexpected output. Just as a child alters its behaviour in response to guidance, so might a learning machine change its own response.

Such a computer would differ greatly from the rigid, rule following type Lovelace had in mind. A learning computer would act in mysterious ways, its internal models and rules unknowable to any operator. It might be fallible, learning - just as we do - rules about the world which are incorrect. After all, Turing wrote, anything that is learnt can be unlearnt, and nothing is ever one hundred percent certain.

It might take fifty years, he thought, but such a strange machine was surely possible. After all, if our brains can learn, think and dream, why couldn’t we build a machine to at least imitate that?

---

---

The Road to ChatGPT

Turing was slightly optimistic. But it is fair to say now, seven decades after his seminal paper, that his vision has turned to reality. Modern “large language models”, of which GPT-4 is the best known example, fit closely to Turing’s vision of a learning machine. They do not follow rules set down by programmers, but rather build their own by parsing vast amounts of data. They behave in mysterious ways, hallucinate and, in conversation, produce passable imitations of human beings.

It was for contributions to the development of these machines that the 2024 Nobel Prize in Physics was awarded. It is, of course, possible to criticise this decision. Turing was no physicist, and would never have described himself as one, nor his field as a part of physics. But some of the breakthroughs that led to the development of large language models relied on tools developed by physicists, and it was thus for this reason the Nobel Committee chose to award it.

Two men shared the prize: John Hopfield and Geoffrey Hinton. Their ideas helped design ways for computers to both remember information and find patterns within it. Afterwards, with those methods in hand, others were able to build computers capable of learning and, perhaps, of thinking.

Before looking at what they did, however, we should first step back a decade or two, and look at how the human brain functions. This, we know, is based on cells known as neurons. Each neuron is capable of transmitting signals and of communicating with other neurons via a network of synapses. Alone no neuron can do much, but as part of a larger network they can conjure up speech, memory and consciousness.

To mimic this on a computer, researchers built systems called neural networks. Neurons could be represented by nodes assigned a value, and they could be connected by links called synapses. As in the brain, those links could be strengthened or weakened, allowing the network to evolve over time and this, in principle, to learn.

Although these networks were first created in the 1950s, it was not until the 1980s that researchers really worked out how to use them. One area of inquiry at that time was memory, and specifically how neurons work together to form and recreate memories. Hopfield, using equations first built to model the spin of atoms in magnets, found a way to do this mathematically, thus allowing neural networks to not just store information, but to store multiple pieces of information and then to retrieve them on demand.

Under his approach, it is almost as if the network creates a landscape of energies. Memories form low points in that landscape, kind of like valleys amidst surrounding hills. To retrieve the memory an input is required - perhaps, for example, a fragment of the original data. The network places this fragment in the landscape and then lets it “roll” down into the nearest valley. From there the network can reconstruct the original data, and so “remember” something it has seen before.

Memory alone, however, is not enough to learn. For that a machine must also be able to link memories together, and class them into broader groups. We can, for example, say an object is a cat, even if we have no memory of ever seeing this particular cat before. Our minds inherently have the idea of a “cat” - with four legs, a tail and pointy ears - based on all the other cats we have seen.

Hinton’s work, based on techniques used to study the statistics of large numbers of particles, found a way to do this in neural networks. He invented something called a Boltzmann machine, a system that can learn patterns and assign probabilities. It is thus capable, after a period of training, of recognising objects, such as cats, and of classifying them accordingly.

Both sets of work, now a few decades old, have been refined and developed. But both were important in developing ways for machines to learn, and thus to build the large language models driving so much excitement today. They were crucial steps, certainly, in creating the machines envisioned by Turing so long ago.

---

A Dangerous Road to Walk?

Machines, Turing wrote in 1950, might “eventually compete with men in all purely intellectual fields”. He pointed to chess as a then unsolved problem, but in which today machines far outstrip us humans. Today’s models, ChatGPT-4 among them, have passed bar exams, competed in mathematics Olympiads and created pieces of art. Might they soon become far more intelligent than we are?

Such a prospect might seem like a technological holy grail. Intelligent machines might accelerate technology, and thus open the way to nuclear fusion, interstellar flight and extended lifespans. Yet, as Geoffrey Hinton warned last year, they may also put us in grave danger.

Throughout much of the 2010s, Hinton worked at Google, where he helped the tech giant develop artificial intelligence systems. But in 2023 he quit that job and gave a frightening interview to the New York Times. He regretted his life’s work, he said then, and feared that it may be used for great evil in the future. It is hard, he said, “to see how you can prevent the bad actors from using it for bad things.”

One obvious problem, already seen, is the rise of false information. The Internet is already degraded, and the information it holds corrupted. It is now possible to create false images, to dress the pope in Balenciaga or depict a rising politician as a criminal. This trend is worrying: after all, when lies become so cheap, how can anything be trusted?

More frightening is the risk of creating a superintelligence we cannot restrain. Such a thing could plainly be dangerous to humanity - indeed, it was our own intelligence that allowed us to dominate the Earth. What something even more intelligent, and so perhaps unbounded in its abilities, might choose to do is unknowable.

Hinton is right. This is a dangerous road. We are creating something we do not - and cannot - fully understand. Comparisons to the atomic bomb have been made, and seem apt. It was, after all, a technology that shaped the last century, that raised the threat of self-imposed extinction, and that the responsible physicists struggled to justify.

It is said that Oppenheimer once called the bomb something “technically sweet”, a challenge of such magnitude it could not be ignored. Yet the consequences terrified him. “I have blood on my hands”, he later told President Truman, and then tried to ensure the weapon would never again be used. 

It was too late. The technology, once the physicists had proven it, passed into the hands of the politicians and the generals. Hinton fears he has done the same, and that now - this time in the hands of Silicon Valley billionaires - artificial intelligence will prove more evil than good. If so, the study of physics may once more have unleashed an existential horror on humanity.

---

The Imitation Game

Turing, in 1950, proposed a test for a thinking machine. If they could imitate humans so perfectly, he suggested, so that an interviewer could not distinguish a machine from a man, then we might conclude it can think. For decades this “imitation game” was held up as the ultimate goal for artificial intelligence.

Yet today there can be no doubt that ChatGPT has won the game. Other models, of ever increasing sophistication, are winning too. But are they really thinking, as Turing believed? Few today seem to accept they are. Are they then simply vast machines, executing algorithms of fantastic complexity? Where along the line does algorithm turn to thought, and imitation turn to reality, if it ever does?  How would we tell?

Those discussing the subject today often sound more like Lovelace than Turing. LLMs, they say, are just statistical models. They are following instructions, they cannot reason, they cannot think. But they can learn, as Turing predicted, and they can form models of the world independently, as children do. In time they may learn to reason, too.

In truth, many of the original objections to the concept of thinking machines have been far surpassed. The goal posts have thus shifted - if only, we now say, a machine could do X, Y or Z, then it might be said it is thinking. But these markers will surely be passed too. Will the posts then shift again? Are we, perhaps, afraid to admit a machine could actually think, with all the implications that would bring?

Turing’s argument, really, was that it doesn’t matter. The concept of “thought” is too nebulous, and we should instead focus on the capabilities of machines. If a machine can act like it can think, then we can say it thinks, and leave it at that. After all, we do not argue about whether submarines swim, or if cars can run. It is the result that matters, not the method.

In that the machines Turing proposed, and Hinton and Hopfield helped build, have clearly succeeded. They may not truly think or be conscious, but they already offer - at the very least - a fair imitation of it.