Friday, November 18, 2011

Is the wavefunction real?

This is a nice result.

I haven't checked the calculations, but I like the logic very much. I'm kicking myself for not having tried harder to precisely formalize what the authors refer to as the "statistical interpretation" (note: this is quite a confusing terminology for most people -- see Further comments below) of the quantum state. Apparently, once you formalize this interpretation, it is easy to prove that it has to disagree with the predictions of ordinary quantum theory.

This "statistical interpretation" (e.g., that the wavefunction, or quantum formalism, only describes the knowledge state of the observer and does not correspond to physical reality) is the last shaky dodge of those who are against the reality (or correspondence to reality) of the wavefunction. The latter has always seemed to me the natural first interpretation of the formalism, subject, of course, to further analysis.

The quantum state cannot be interpreted statistically

Matthew F. Pusey, Jonathan Barrett, Terry Rudolph

http://arxiv.org/abs/1111.3328

Quantum states are the key mathematical objects in quantum theory. It is therefore surprising that physicists have been unable to agree on what a quantum state represents. There are at least two opposing schools of thought, each almost as old as quantum theory itself. One is that a pure state is a physical property of system, much like position and momentum in classical mechanics. Another is that even a pure state has only a statistical significance, akin to a probability distribution in statistical mechanics. Here we show that, given only very mild assumptions, the statistical interpretation of the quantum state is inconsistent with the predictions of quantum theory. This result holds even in the presence of small amounts of experimental noise, and is therefore amenable to experimental test using present or near-future technology. If the predictions of quantum theory are confirmed, such a test would show that distinct quantum states must correspond to physically distinct states of reality.

Here's what Nature News had to say:

Quantum theorem shakes foundations: The wavefunction is a real physical object after all, say researchers.

There seems to be widespread misunderstanding of what the authors are trying to do in this paper.

They are not trying to refute qm or the standard rules of calculation (e.g., Born rule). Perhaps their use of the term "statistical interpretation" is unfortunate because some people seem to have jumped to the conclusion that they claim to prove qm is deterministic or non-probabilistic. That is not the case.

They are addressing a particular interpretation of qm. This interpretation says: there is an underlying physical reality, but the state Psi only describes an observer's knowledge about that underlying reality. Psi is not itself a direct representation of that reality. ("Psi is not real".) I would classify this as a variant of Copenhagen; its proponents sometimes refer to it as a "Bayesian" or "Epistemic" interpretation. I prefer to call it the "Mysterian" interpretation: reality is some vast mysterious thing (never specified!), Psi only characterizes the observer's mental state; collapse of the wavefunction is simply a Bayesian update of the mental state.

Mysterian/Bayesian: "The reduction of the wavefunction takes place in the consciousness of the observer ... because the state is a construct of the observer's mind and not an objective property of the physical system."

Many Worlder: "The wavefunction is real (i.e., a direct representation of physical reality), but it does not collapse."

Note, both groups try to avoid the possibility that Psi is real and collapses. But see Weinberg's recent preprint for an attempt to understand that possibility: http://arxiv.org/abs/1109.6462

A modern proponent of the Mysterian point of view is Chris Fuchs. I would be very interested to hear his reaction to this paper. But Rob Spekkens (quoted in the Nature article) also thinks along these lines, and he seems to believe that the (lambda, q) formalization of Mysterianism captures something useful. I am still pondering it myself.

Technically, the (lambda, q) formalization describes a model in which (i) there is an underlying reality (some Mysterians apparently do not actually believe this) and (ii) the state vector Psi does not describe the underlying reality but rather an observer's knowledge about it.

The fact that a given underlying reality lambda has probability q of being consistent with two different preparations of a state, which each yield different pure states phi_0 and phi_1 (their notation), is meant to capture (i) and (ii) above. Remember that to a Mysterian the pure state is a description of a state of knowledge, not of reality. So nonzero q means that two different states of knowledge (preparations) are consistent with the same underlying state of reality.

These Fuchs slides might be of use in understanding the mysterious Mysterians: Being Bayesian in a Quantum World (I am a Bayesian, who lives in a quantum world, but not a Mysterian :-)

This blog post by Matt Leifer is very clear and gives the context for the paper in the qm foundations subfield.

Tony Pan said...

I carefully checked the simple proof on pg2.  It holds up.  I think this means we can't get around wavefunction collapse anymore?

steve hsu said...

I think it means wavefunctions don't collapse ;-)

So what do you think of Lubos' criticism?

steve hsu said...

I think Lubos misunderstands what the authors are trying to prove. They are not addressing the probabilistic rules of QM (i.e., the Born rule) or how to do basic calculations. They are addressing a particular *interpretation* ("statistical interpretation") in which it is asserted that the quantum state or wavefunction describes (for example) the state of knowledge of an observer as opposed to being a faithful representation of the state of the world ("reality"). This is an old idea, but in its modern incarnation is sometimes referred to as the Bayesian interpretation of QM (see papers by, e.g., Chris Fuchs).

If you've never been exposed to this "qm formalism describes observer's state of knowledge" interpretation (despite the fact that it is as old as qm itself), then it might be very difficult to understand what the authors are trying to do with their $\lambda$ object formulation. The authors show that it is quite difficult to avoid having the quantum state of a system ($\phi$ in their notation) determine uniquely the probabilities of all measurements. The tricky philosophical or interpretational question is whether this is enough to exclude the possibility that "reality" is more complicated than what is encoded in $\phi$ (or that $\phi$ only encodes the state of knowledge of an observer and the underlying reality is more complex).

Sabine Hossenfelder said...

I haven't read the paper, but heard John's talk some months ago http://pirsa.org/11050028/ It made a lot of sense to me. If that's what the paper is about, I am quite confused about the Nature summary, the use of the word 'real' is somewhat misleading I think.

Nano Nymous said...

In a comment at Marginal Revolution, Andrew
wrote the following. I can't evaluate any of it myself. Seems to me that if his objection is valid, it is pretty damning. Thoughts?
My take (as as someone who got a PhD in quantum measurement and control)
The paper starts by distinguishing what the results of a probabilistic
theory are from a deterministic theory using a simple two atom (or
qubit) experiment. So far so good. Then they show that a four atom
experiment does not obey this simple rule. The step they are missing is
to show that the same probabilistic/deterministic distinction holds for
a four atom experiment as for a two atom one.

This is a standard frequentist mistake in regular probability theory
and is well addressed inside the Copenhagen interpretation by a quantum
version of De Finetti’s theorem, similar to the classical De Finetti
theorem. This theorem shows how one treats multiple copies of an
experiment (or preparation) with probability.

I would be amazingly surprised if this result actually caused anyone
quantum mechanics.

steve hsu said...

See my comment below on Lubos. I think a lot of people misunderstand what the authors are trying to do. Probably their biggest mistake was to use the term "statistical interpretation". They explicitly note in the paper (for the really clueless) that they aren't trying to show that qm is really deterministic. In fact they entertain much more general classes of theories than ordinary qm, including stochastic hidden variables.

steve hsu said...

The talk you linked to is related, but (judging from the abstract) emphasizes slightly different things. The result in the paper more directly addresses the question of whether psi is "real" or merely gives information about an underlying reality.

LondonYoung said...

I was waiting for "hidden variables" to appear in the discussion.  Which does that mean:  I am clueless, or I am not clueless?  Or is that still a mystery?
I am unable to distinguish this discussion from a discussion of whether or not hidden variables exist which do describe reality, but which we cannot access, so we rely on formalisms which describe the few things we can access.

Christopher Weed said...

For those who haven't already seen it, Scott Aaronson has posted his own commentary on the paper.

The comments include one by Lubos Motl. (Remember Gilda Radner's SNL character Emily Litella?)

steve hsu said...

This paper might help: http://arxiv.org/abs/0706.2661

Robert Sykes said...

Lubos' criticism continues: http://motls.blogspot.com/2011/11/is-philosophical-babbling-on-par-with.html#more

LondonYoung said...

Yes, this looks good.  I also recall that Bohr wrote a number of first-hand accounts about his convos with other physicists on related topics.  Should make some good Christmas/New Year's reading.

Christoph Gärtner said...

As far as I can tell, the paper offers nothing of value. The statistical interpretation of QM is a minimalist interpretation which basically can be summarized like this:

"We have no idea what happens
when performing individual measurements. However, let me tell you what
you'll see when you repeat the experiment often enough."

It does not claim that the wave-function is real (which is on the same footing as claiming that the action function of classical mechanics has physical reality, btw), it does not talk about observers and collapse, many worlds or atemporal transactions: It just states that the formalism we have does not determine the outcome of a single experiment (which is the whole point of QM - you only ever get out probabilities) and therefore a single experiment is just outside the applicability of the theory.

Therefore, it is impossible to disprove the statistical interpretation from within QM, as the authors claimed to have done. What they actually did prove is that their custom probabilistic theory using hidden variables is inconsistent with QM.

steve hsu said...

Read what I wrote. What you call the "statistical interpretation" is not what the authors of the paper are talking about. Their terminology is probably confusing. No one is claiming to disprove what you call the statistical interpretation.

Tony Pan said...

Well, out of popular interpretations, I guess only Copenhagen and Many Worlds are left...   and come on, despite the niceties of decoherence, the latter is just crazy :D

MtMoru said...

What is an interpretation (of a theory) if its predictions are all the same as another interpretation's? What is a theory with predictions which depend on its interpretation?

"Note, both groups try to avoid the possibility that Psi is real and collapses. But see Weinberg's recent preprint for an attempt to understand that possibility"

What's to "understand"? What ever happened to American pragmatism?

Sounds like Sir Karl would object.

steve hsu said...

The three versions of qm that I listed have different experimental predictions -- at least in principle, if not in experiments we can currently perform. This was not appreciated for a long time, hence the original use of the term "interpretation" when what was really meant was different physical theories. Bell invented a term FAPP = For All Practical Purposes and FAPP the different interpretations agree.

Season4Reason said...

Indeed.  Infact MWI has been shown to be inconsistent: http://arxiv.org/abs/1109.6424

Guess it's time to take Weinberg's advice and rethink the entire QM

stochastic hidden variables, yes that is correct. Quantum Statistical Automata.

http://www.qsa.netne.net

Luboš Motl said...

Dear Steve, with one month of hindsight, I may return to this topic. Your further comments prove that I understood the paper 100% correctly and you, much like the authors of this nonsensical paper, 100% misunderstand modern physics.

You write very clearly that they claim that they disprove the "Mysterian" interpretation. But you also make it clear that the "Mysterian interpretation" is nothing else than the canonical interpretation of quantum mechanics due to Max Born, Niels Bohr, Werner Heisenberg, Paul Dirac, and others. It is the Copenhagen interpretation in its most general sense, it is what quantum mechanics is all about. You wrote:I prefer to call it the "Mysterian" interpretation: reality is some vast mysterious thing (never specified!), Psi only characterizes the observer's mental state; collapse of the wavefunction is simply a Bayesian update of the mental state.

LM: I repeat that whoever thinks that he has disproved the "Mysterian" interpretation is a crank because the Mysterian interpretation isn't really an interpretation. It's the set of the basic postulates underlying quantum mechanics. Any approach that denies any of these postulates is invalid. It's been the basic undergraduate stuff about QM since the late 1920s. See e.g. Dirac's 1930 textbook on Principles of QM.

Tony Zee would tell me that you were an OK physicist. I can't believe my ears today because you seem to be exactly the same kind of crackpot like the Pusey grad student and dozens of others. Sorry to say it.

tony said...