Beyond the Quantum review: A remarkable book on quantum mechanics reveals a really big idea

Beyond quantum
Antony Valentini, Oxford University Press

The physics, it is fair to say, did not go as planned. After decades of hopeful searches, dark matter has still not been directly detected. We found the Higgs boson, but nothing opens the way for us. And string theory, that much-vaunted theory of everything, has yet to produce a clear, testable prediction. Trust is low. Where do we go from here?

In recent years, many physicists who have written popular science have avoided this question. Where once they boldly pointed to the next great discovery, we now often see them retreating into philosophical reflection or re-explaining what we already know. This is not the case for Antony Valentini at Imperial College London. In Beyond Quantum: A quest for the origin and hidden meaning of quantum mechanics, it presents something almost quaint in its rarity: a great idea.

As the title suggests, its main focus is quantum mechanics, which has underpinned physics for a century. It depends on the wave function, a mathematical expression that can, as textbooks say, specify the complete state of any system, from a fundamental particle to a cat or even you and me.

The strange thing about the wave function is that it usually doesn’t describe ordinary, localized objects at all, but rather spread out, fuzzy, wave-like versions. Nevermind. When we look at an object – again, as the textbooks say – the wave function “collapses” into a familiar but random result, with a probability given by Born’s rule (named after the physicist Max Born). Only now we have an object with defined properties in a defined location.

Although mainstream physics tries to ignore it, the interpretation of the wave function has always been a mystery – and there are essentially only two realistic answers. The first is that the wave function actually describes reality – that electrons, cats, and people actually exist in many states at once, spread across space and possibilities. This is the many-worlds interpretation, with its vast metaphysical implications.

The other answer is that the wave function doesn’t tell the whole story. The dominant theory here, largely developed by Valentini, is the pilot wave theory, first proposed by the theorist Louis de Broglie in 1927, then revived by the physicist David Bohm.

Pilot wave theory considers the wave function to be real but incomplete. This suggests that it acts as a structure guiding individual particles, much like waves direct plastic bottles floating on the sea. The particles themselves are never spread out or indeterminate: their wave-like behavior comes from the pilot wave and where they sit on it.

It has long been known that pilot wave theory reproduces all the predictions of quantum mechanics, without any fundamental randomness. But, as Valentini points out, this agreement is based on an assumption: the particles are in balance with the wave, distributed in the right way. This hypothesis is consistent with current experimental data – the results are virtually undeniable – but it need not always be valid.

Valentini proposes that, in the early universe, particles were distributed far from quantum equilibrium, before “relaxing” into their current state, much like a cup of coffee cools to adapt to its environment. From this perspective, Born’s rule and its randomness are no longer fundamental features of nature, but historical accidents – they are by-products of cosmology.

This striking reward is not the only one. Quantum randomness also prevents any practical exploitation of nonlocality, the immediate interaction of objects separated by time and space. If Born’s rule had not been followed in the early universe, Valentini argues, instantaneous communication over vast distances would have been possible, perhaps leaving subtle imprints in the cosmic microwave background. If relics from that era still exist, such superluminal signaling could be feasible now.

Given the lack of evidence, this might seem a bit crazy without Valentini’s meticulous analysis of how orthodox quantum mechanics came to take hold (the book is worth reading for that alone). If there is a weakness, it is the absence of a clear description of the pilot wave. Yet whether this turns out to be true or not, Valentini’s work at least shows us that, in a field where trust is limited, this is what a truly great idea looks like.

Jon Cartwright is a writer based in Bristol, UK