Muonic Proton Radius Discrepancy Problem Resolved

Two New Measurements Of The Proton Radius In Ordinary Hydrogen

A new measurement of the charge radius of the proton in ordinary hydrogen finds that:

Unraveling the proton puzzle 

The discrepancy between the proton size deduced from the Lamb shift in muonic hydrogen and the average, textbook value based on regular (electronic) hydrogen has puzzled physicists for nearly a decade. One possible resolution could be that electrons interact with protons in a different way than muons do, which would require “new physics.” Bezginov et al. measured the Lamb shift in electronic hydrogen, which allowed for a direct comparison to the Lamb shift measured in muonic hydrogen. The two results agreed, but the discrepancy with the averaged value remains. 

Abstract

The surprising discrepancy between results from different methods for measuring the proton charge radius is referred to as the proton radius puzzle. In particular, measurements using electrons seem to lead to a different radius compared with those using muons. Here, a direct measurement of the n = 2 Lamb shift of atomic hydrogen is presented. Our measurement determines the proton radius to be rp = 0.833 femtometers, with an uncertainty of ±0.010 femtometers. This electron-based measurement of rp agrees with that obtained from the analogous muon-based Lamb shift measurement but is not consistent with the larger radius that was obtained from the averaging of previous electron-based measurements.

N. Bezginov, et al., "A measurement of the atomic hydrogen Lamb shift and the proton charge radius" 365 (6457) Science 1007 (September 6,  2019). DOI: 10.1126/science.aau7807

See also this news article and this more meaty new article for background, and also this post at the Physics Forums. The press release based news article states:

"The level of precision required to determine the proton size made this the most difficult measurement our laboratory has ever attempted," said Distinguished Research Professor Eric Hessels, Department of Physics & Astronomy, who led the study.

"After eight years of working on this experiment, we are pleased to record such a high-precision measurement that helps to solve the elusive proton-radius puzzle," said Hessels.

In a September 19, 2018 post at this blog (about a year ago), entitled "Proton radius problem solved?", I stated:

This would solve one of the major unsolved problems of physics (I have previously put this problem in the top twelve experimental data points needed in physics), called the "proton radius problem" or "muonic hydrogen problem" in which the radius of a proton in a proton-muon system appeared to be 4% smaller than measurements of the radius of a proton in ordinary hydrogen, contrary to the Standard Model. See previous substantive posts on the topic at this blog which can be found on November 5, 2013, April 1, 2013, January 25, 2013, September 6, 2011, and August 2, 2011.

Previously most measurements in ordinary hydrogen favored a larger value for the radius of a proton (the global average of these measurement was 0.8775(51) fm), while the new high precision and more direct measurement in muonic hydrogen found a smaller value for the radius of a proton of 0.84184(67) fm. Now, a new electron measurement devised to rigorously confirm or deny the apparent discrepancy  measured a result that agrees with the muon-based measurements. 

So, it looks increasingly like something is wrong with the older electron-based results, most likely predominantly, in the process of making theoretical calculations to convert raw data to final proton charge radius value.

Basically, in yet another feud between the theoretical predictions of the Standard Model of Particle Physics and an apparent experimental discrepancy with those predictions, the Standard Model has won, a measurement error was ferreted out, and a new physics explanation has once again been rejected. With it, myriad of new physics theories devised to explain the proton radius problem will be ruled out.

If you are familiar with this area of fundamental physics research, this diagram tells the entire story. If not, read on.

What is a proton's charge radius?

The radius of the proton is the distance at which the charge density drops below a certain energy threshold. And it's possible to measure that radius by studying how the electron interacts with the proton, via either electron scattering experiments or by using electron or muon spectroscopy to look at the difference between atomic energy levels. (It's called the "Lamb shift," after Nobel laureate Wallis Lamb, who first measured the shift in 1947.) The combined fuzziness of the electron and proton means that the electron can be anywhere inside that region—including inside the proton.

From Ars Technica.

Of course, this is a notable issue not because there was a great technical and engineering need to know the precise size the the proton charge radius, although it is useful to know this fact, but because measurements of it had pointed to a possible hole in the Standard Model of Physics.

Possibly Factors In A Core Theory Analysis Of Proton Charge Radius Measurements

In theory, there should not be a material difference between (1) the radius of a proton (which is a composite particle made of quarks held together by gluons) when it has an electron rotating around it (a.k.a. "ordinary hydrogen"), and (2) the radius of a proton when it has a muon (a particle identical to an electron except that it is about 200 times more massive and decays to an electron on average in about a millionth of a second in its reference frame) rotating around it (a.k.a. "muonic hydrogen").

So far as we know, there are only four fundamental forces in Nature: the strong force, the electromagnetic force, the weak force, and gravity.

In theory, the radius of a proton should depend almost entirely upon the strong force interactions via gluons between the quarks in the proton at first order, and the electromagnetic interactions between the quarks in the proton via photons at second order. 

The weak force (which is responsible, for example, for beta decay of radioactive substances like uranium) should have almost no impact because a proton is stable with respect to the weak force, as is an electron, and the radius of a proton in muonic hydrogen is measured only during the time that the muon is a muon before it decays via the weak force.

The fourth of the four fundamental forces, gravity, turns out, numerically, to have an infinitesimal strength relative to the strong and electromagnetic forces at the scale of protons and simple light atoms.  

Any impact on the proton radius from the electron or muon which makes that proton an atom should, on average, be limited to electromagnetic interactions (since electrons and muons don't interact via the strong force) which are a weak second order adjustment to the theoretically expected proton radius, and should, on average balance out to nearly zero for symmetry reasons (an electron or muon in an atom spends an equal amount of time on average in all directions in the sphere around the proton).

Muonic Hydrogen Based Measurements And The Muonic Hydrogen Problem

Not too many years ago, the radius of muonic hydrogen was measured to high precision. And, in general, measurements made with muons are more precise than those with electrons because they have greater mass. This isn't self-evident, but turns out to be true if you look into the details of it, as explained by Ars Technica at the link above:

It's the muon spectroscopy measurements that first caused the problem back in 2010. For their experiment, physicists at the Max Planck Institute of Quantum Optics used muonic hydrogen, replacing the electron orbiting the nucleus with a muon, the electron's heavier (and very short-lived) sibling. Since it's nearly 200 times heavier than the electron, it has a much smaller orbital, and thus a much higher probability (10 million times) of being inside the proton. And that makes it ten million times more sensitive as a measurement technique, because of its closer proximity to the proton.

The measurement found that there was a quite significant discrepancy between the radius of a proton of ordinary hydrogen according to previous measurements, and the radius of a proton in muonic hydrogen according to the new measurements, although not so great as to constitute a definitive experimental discovery in physics. As explained in an pre-print from 2018 (citation references either omitted or migrated to body text), that appears to incorporate the pre-print of this paper:

The question of the proton charge radius is still the subject of current theoretical and experimental investigations. The root-mean-square charge radius, rp, has been determined by three experiments: first, by the electron-proton scattering, I. Sick, Phys. Lett. B 576, 62 (2003) and P. G. Blunden and I. Sick, Phys. Rev. C 72, 057601 (2005); second, by the precision spectroscopy of atomic hydrogen, M. Fischer and et al., Phys. Rev. Lett. 92, 230802 (2004), and, third, by pulsed laser spectroscopy measurements of the Lamb shift in muonic hydrogen, R. Pohl and et al., Nature 466, 213 (2010). The most accurate rp value with the uncertainty of 1 per cent is based mainly on atomic hydrogen experiments and calculations of bound-state quantum electrodynamics (QED). The present value given by CODATA using only electronic spectroscopy data is rp = 0.8758(77) fm. Therewith the value given by the electron-proton scattering is 0.879(8) fm. Thus, CODATA finds that the overall result rp = 0.8775(51) fm. The problem called ’proton radius puzzle’ has arised from the muonic hydrogen experiment: matching the theoretical calculations of the Lamb shift with the experimental data leads to rp = 0.84184(67) fm. This magnitude differs on 5.6 standard deviations from the CODATA value. This discrepancy constitutes one of the most attractive questions in connection with the search of ’new physics’; a lot of theoretical and experimental efforts were devoted to investigation of this problem.

The main problem of the proton charge radius determination from the hydrogenic data consists in complexity of theoretical description of such experiments, whereas the measurements in muonic hydrogen are transparent and allow the direct comparison of experiment with theory. However, the rp values obtained from electron-proton scattering and spectroscopic measurements in hydrogen are close that gives a reason for the inclusion of the overall value in CODATA. Very recently the new value of proton charge radius was reported in A. Beyer and et al., Science 358, 79 (2017): rp = 0.8335(95) femtometer. This value was extracted from the measurement of 2s − 4p transition frequency in hydrogen atom and diverges on 1% approximately from the µH data. This value was extracted from the measurement of 2s−4p transition frequency in hydrogen atom and diverges on 1% approximately from the µH data. Such satisfactory agreement was reached in the experiment accounting the quantum interference effect and hyperfine splitting of levels. The attempts to describe theoretically the spectroscopic measurements in hydrogen were performed in a series of works, where the nonresonant corrections (called quantum interference in A. Beyer and et al., Science 358, 79 (2017)) were introduced. In particular, theoretical description of the 1s − 2s transition frequency measurement in hydrogen atom was given in [another work], where the nonresonant correction to the 1s − 2p transition frequency in hydrogen atom was estimated with the account for the hyperfine splitting also. According to the results of A. Beyer and et al., Science 358, 79 (2017) the nonresonant effects should be taken into account in spectroscopic measurements and, therefore, the theoretical re-analysis of the proton charge radius determination from the hydrogenic data is required.

I'm not quite sure what I did wrong (if anything) in determining the significance of the CODATA value from the muonic hydrogen value, which I calculated directly and crudely at 6.9 sigma, rather than the 5.6 quoted (perhaps a look elsewhere effect was considered). The error relative to a refined 2013 measurement of the muonic hydrogen proton charge radius was about 7 sigma.

I calculated the difference between the old ordinary hydrogen measurement summarized by CODATA and the new one to be 4.1 sigma.

I calculated the difference between the muonic hydrogen measurement and the 2017 measurement one to be 0.9 sigma. I calculated the difference between the muonic hydrogen measurement and the 2019 measurement (see the abstract below) to be 0.9 sigma as well.

The 2017 measurement and the 2019 measurement are consistent to 0.04 sigma (far better than what would be expected given their modest precision, suggesting that some of their sources of error were overestimated).

Also, note that the Lundeen and Pipkin result illustrated in the chart is consistent at 1 sigma with the muonic hydrogen measurement and with the 2017 and 2019 ordinary hydrogen measurements, even though its central value is higher than the CODATA value, due to its low precision. This 1981 measurement is discussed in a 2018 paper that has some of the same authors as Friday's paper, Bezginov (2019). The introduction to that 2018 paper states:

The hydrogen Lamb-shift measurement has become important since it can, when compared to very precise theory, determine the charge radius of the proton. More precise determinations of this radius have now been performed using muonic hydrogen, but there is a large discrepancy between measurements made using ordinary hydrogen and muonic hydrogen. This discrepancy has become known as the proton size puzzle. 

In this work, we evaluate possible systematic effects for a microwave separated-oscillatory-fields (SOF) precision measurement of the atomic hydrogen 2S1/2-to-2P1/2 Lamb shift. This interval was measured by Lundeen and Pipkin in 1981, and is currently being remeasured by our group, with an aim to help resolve the proton size puzzle. We take advantage of the advances in computational power that have become available over the past decades to re-evaluate the measurement of Lundeen and Pipkin, and explore the implications for the proton size puzzle.

None of the latest discussion of the proton radius problem seems to discuss a refined measurement of the muonic hydrogen measurement proton radius from 2013 by the group which measured it originally, which was  0.84087(39) fm.

The 2018 result shown in the chart is from here and is 0.877(13)  fm, in line with the CODATA value, 2.8 sigma from the refined muonic hydrogen value, and 2.6 sigma from the 2017 ordinary hydrogen value.

The Theoretical Calculation Problem With Old Ordinary Hydrogen Based Measurement

My September 19, 2018 post was based upon the following paper, which argued that there had been a theoretical basis in converting the raw experimental results from ordinary hydrogen into the proton radius. Accounting for that bias made the old ordinary hydrogen based results consistent with the muonic hydrogen measurement:

We extract the proton charge radius from the elastic form factor data using a theoretical framework combining chiral effective field theory and dispersion analysis. Complex analyticity in the momentum transfer correlates the behavior of the spacelike form factor in different Q2 regions and permits the use of data up to Q2 ∼ 0.5 GeV2 in constraining the radius. The predictive theory describes the data with the same accuracy as current descriptive models (global fits). We obtain a radius of 0.844(7) fm, consistent with the high-precision muonic hydrogen results.

J. M. Alarcón, D. Higinbotham, C. Weiss, Z. Ye, "Proton charge radius extraction from electron scattering data using dispersively improved chiral effective field theory" (September 17, 2018).

This paper is 0.3 sigma from the muonic hydrogen result and is 1.6 sigma from the two new ordinary hydrogen based measurements, so the theoretically revised estimate, and these three experimental values, are all consistent with each other.

What Went Wrong In the Ordinary Hydrogen Based Measurements?

There were at least five plausible reasons that this could have been true: (1) the muonic hydrogen measurement was flawed in some way, (2) the ordinary hydrogen measurement was flawed in some way, (3) both the muonic hydrogen and ordinary hydrogen measurements were flawed, (4) the theoretical expectation of the difference between the radius of ordinary hydrogen and muonic hydrogen was flawed, or (5) "new physics" beyond the Standard Model were being revealed experimentally.

Smart money has always been on explanation number 2, flaws in the ordinary hydrogen measurements. I have been saying this since at least my August 2, 2011 post on this topic. In an April 1, 2013 post, I said:

Arguably the difference in measurements between the size of protons in muonic hydrogen and ordinary hydrogen, and the anomalous muon magnetic dipole moment are up in the air, but this could also be a result of an underestimated systemic error in either the theoretical calculation, the experimental measurement, or both. The former is in theory a 4.4 to 4.6 sigma effect (the value values differ by about 0.02 fm, which is about 2.5% of the mean value of the ordinary hydrogen and muonic hydrogen values). The 3.4 sigma discrepancy in the latter still matches the theoretical value to about nine significant digits. There is good reason to suspect that the tensions between the measurements in both cases will resolve with more precise measurements.

Muonic hydrogen proton radius measurements are inherently more precise than ordinary hydrogen radius measurements, all other things being equal, and were also new and carefully and skeptically scrutinized. 

The ordinary hydrogen proton radius measurements were old, made as a course of routine measurements for which there was no reason for skepticism, and showed considerable scatter. 

Plausible explanations with "new physics" are hard to devise in an area of physics that has been rigorously and precisely tested in a variety of circumstances that easily have the potential to over constrain the parameters being measured. You need a new force that impacts how muons interact with protons that is different from how electrons interact with protons that could impact the proton's radius, when a variety of rigorous experiment tests of other kinds have consistently demonstrated that muons and electrons behave exactly alike except that muons have a heavier mass and decay quickly to electrons.

It increasingly, it looks like the smart money was right, although part of this was due to explanation number 4, a theoretical mistake that failed to consider one material part of the calculation called a "quantum correction.".

Two new proton radius measurements with ordinary hydrogen, and a conversion of old ordinary hydrogen based measurements to factor in the theoretical factor that was neglected favors the new muonic hydrogen measurement.

The First Of More To Come?

I fully expect that the discrepancy between the current state of the art measurement of muon g-2 (the magnetic moment of the muon) and theoretical calculations of its value, will be resolved in a similar manner with better measurements. Scientists are in the process of making a new, more precise muon g-2 measurement, and efforts to refine the notoriously difficult QCD component if the muon g-2 value, which accounts for a quite small portion of the total value but the lion's share of the uncertainty in the theoretically predicted value, has been ongoing on a regular basis almost continuously since the last experimental muon g-2 measurement revealed a possible tension in the data.

I also expect that the hints of lepton decay non-universality in B meson decays to be resolved in a similar manner. Lepton non-universality, in practice, as observed, mostly means that muons aren't behaving exactly like electrons in some respect other than one that is a known function of particle mass. In many circumstances, constraints on lepton non-universality are very strict, even though in some but not all of several other circumstances involving B meson decays there is fairly significant evidence of lepton universality violations. But, one has to really contort and use seemingly very baroque beyond the Standard Model theory to produce the experimental results seen to date with new physics, without tripping over the myriad data points supporting lepton universality, and the Standard Model theoretical considerations that support lepton universality, in every other context.

I think that it is likely that both the muon g-2 discrepancy and the apparent lepton universality violations in B meson decays are really just artifacts of the way that particles of different masses impact the experimental measurement and observation process that have not been adequately controlled for in existing measurements, rather than being the consequence of some beyond the Standard Model theory in which muons have different properties (in addition to mass) from electrons.

Post-Script

Thomas Anderson has made a comment below which has a notable point to be make about social science type bias in hard science that bears reading, but contains only links to the two charts mentioned, which show historical trends in consensus values of Planck's constant (which was recently used in a redefinition of the metric system of measurements), which I post here in chart form (in the order that they appear in his post). You can follow the last to two links in his comment, to scholarly articles, yourself.