Are Modified Gravity Theories Credible?

Question

I'm a statistician with a little training in physics and would just like to know the general consensus on a few things.

I'm reading a book by John Moffat which basically tries to state how GR makes failed predictions in certain situations. I know GR is well extremely tested, but I imagine all physicist are aware it doesn't always hold up.

The book tries to put forth modified theories of gravity that make do without the need of dark matter and dark energy to make GR match real world observations. (ie speed of galaxy rotations etc)

Are modified theories of gravity credible?

Is dark energy/matter the 'ether' of the 20th/21st century? Is it likely scientists are looking for something that simply doesn't exist and there are unknown fundamental forces at work? What's the best evidence for it's existence other than observations based on the 'bullet' cluster?

(This is another abridged cross-post from Physics Stack Exchange).

Answer

Yes. Modified gravity theories are credible. While dark matter theories are one way to explain the phenomena described by dark matter and these are generally more popular ways to resolve these issues, there are deep, potentially intractable problems with a dark matter particle approach as well. The weight of the evidence as shifted as astronomers, particle physicists and theorists have provided us with more relevant evidence and with more ideas about how to solve the problem, even in the last few years in this very active area of ongoing research.

This is as it should be because dark matter phenomena constitute the most striking case in existence today where the combination of general relativity and the Standard Model of Particle Physics simply cannot explain the empirical evidence without some kind of new physics of some kind.

1. Any viable dark matter theory has to be able to explain why the distribution of luminous matter in a galaxy predicts observed dark matter phenomena so tightly and with so little scatter in multiple respects such as rotation curves and bulge sizes. These relationships persist even in cases that in a non-gravitational theory should not naturally hold. For example, planetary nebulae distantly rotating ellipical galaxies show the same dynamics of stars at the fringe of spiral galaxies do. Similarly, these relationships persist in gas rich galaxies and dwarf galaxies(which have as predicted about 0.2% ordinary matter if GR is correct in a universe that is overall 17% dark matter) despite that they are beyond the scope of the data used to formulate the theories.

One of the more successful recent efforts to reproduce the baryonic Tully-Fischer relation with CDM models is L.V. Sales, et al., "The low-mass end of the baryonic Tully-Fisher relation" (February 5, 2016). It explains:

[T]he literature is littered with failed attempts to reproduce the Tully-Fisher relation in a cold dark matter-dominated universe. Direct galaxy formation simulations,for example, have for many years consistently produced galaxies so massive and compact that their rotation curves were steeply declining and, generally, a poor match to observation. Even semi-analytic models, where galaxy masses and sizes can be adjusted to match observation, have had difficulty reproducing the Tully-Fisher relation, typically predicting velocities at given mass that are significantly higher than observed unless somewhat arbitrary adjustments are made to the response of the dark halo.

The paper manages to simulate the Tully-Fisher relation only with a model that has sixteen parameters carefully "calibrated to match the observed galaxy stellar mass function and the sizes of galaxies at z = 0" and "chosen to resemble the surroundings of the Local Group of Galaxies", however, and still struggles to reproduce the one parameter fits of the MOND toy-model from three decades ago. Any data set can be described by almost any model so long as it has enough adjustable parameters.

Much of the improvement over prior models has come from efforts to incorporate feedback between baryonic and dark matter into the models, but this has generally been done in a manner than is more ad hoc than it is firmly rooted in rigorous theory or empirical observations of the feedback processes in action.

One of the more intractable problems with simulations based upon a dark matter particle model that has been pointed out, for example, in Alyson M. Brooks, Charlotte R. Christensen, "Bulge Formation via Mergers in Cosmological Simulations" (12 Nov 2015) is that their galaxy and mass assembly model dramatically understates the proportion of spiral galaxies in the real world which are bulgeless, which is an inherent difficulty with the process by which dark matter and baryonic matter proportions are correlated in dark matter particle models which are not a problem for modified gravity models. They note that:

[W]e also demonstrate that it is very difficult for current stellar feedback models to reproduce the small bulges observed in more massive disk galaxies like the Milky Way. We argue that feedback models need to be improved, or an additional source of feedback such as AGN is necessary to generate the required outflows.

General relativity doesn't naturally supply such a feedback mechanism.

2. The fact that it is possible to explain pretty much all galactic rotation curves with a single parameter implies that any dark matter theory also can't be too complex, because otherwise it would take more parameters to fit the data. The relationships that modified gravity theories show exist are real, whether or not the proposed mechanism giving rise to those relationships is real or not. A dark matter theory shouldn't have more degrees of freedom than a toy model theory that can explain the same data. The number of degrees of freedom it takes to explain a data set is insensitive to the particular underlying nature of the correct theory to explain that data.

Also, while I don't have references to them easily at hand at the moment, early dark matter simulations quickly revealed that models with one primary kind of dark matter fit the data much better than those with multiple kinds of dark matter that significantly contribute to these phenomena.

This simplicity requirement greatly narrows the class of dark matter candidates that need to be considered, and hence, the number of viable dark matter particle theories that a modified gravity theory must compete with in a credibility contest.

3. There are fairly tight constraints from astronomy observations on the parameter space of dark matter. Alyson Brooks, "Re-Examining Astrophysical Constraints on the Dark Matter Model" (July 28, 2014). These rule out pretty much all cold dark matter models except "warm dark matter" (WDM) (at a keV scale mass that is at the bottom of the range permitted by the lamdaCDM model) and "self-interacting dark matter" (SIDM) (which escapes problems that otherwise plague cold dark matter models with a fifth force that only acts between dark matter particles requiring at least a beyond the Standard Model fermion and a beyond the Standard Model force carried by a new massive boson with a mass on the order of 1-100 MeV).

4. Direct detection experiments (especially LUX) rule out any dark matter candidates that interact via any of the three Standard Model forces (including the weak force) at masses down to 1 GeV (also here).

5. Another blow is the non-detection of annihilation and decay signatures. Promising data from the Fermi satellite's observation of the galactic center have now been largely ruled out as dark matter signatures in Samuel K. Lee, Mariangela Lisanti, Benjamin R. Safdi, Tracy R. Slatyer, and Wei Xue. "Evidence for unresolved gamma-ray point sources in the Inner Galaxy." Phys. Rev. Lett. (February 3, 2016). And, signs of what looked like a signal of warm dark matter annihilation have likewise proved to be a false alarm.

6. The CMS experiment at the LHC rules out a significant class of low mass WIMP dark matter candidates, while other LHC results exclude essentially all possible supersymmetric candidates for dark matter. If SUSY particles exist, they would be both too heavy to constitute warm dark matter (almost all types of SUSY particles are excluded up to about 40 GeV by the LHC which is too heavy) and they would also lack the right kind of self-interactions force within a SUSY context to be a SIDM candidate. This has particularly broad implications because SUSY is the low energy effective theory of almost all popular GUT theories and viable string theory vacua.

7. While MOND requires dark matter in galactic clusters, including the particularly challenging case of the bullet cluster, this defect is not shared by all modified gravity theories (see, e.g.,here and here). Many of the theories that can successfully explain the bullet cluster are able to do so mostly because the collision can be decomposed into gas and galaxy components that have independent effects from each other under the theories in question. The bullet cluster is also one of the main constraints on SIDM parameter space (which itself basically does modify gravity but just does so in the dark sector, limiting those modifications to dark matter particles only), and is tough to square with manner dark matter particle theories.

8. It is possible in a modified gravity theory but very challenging in a dark matter particle theory, to explain why the mass to luminosity ratio of ellipical galaxies varies by a factor of four, systemically based upon the degree to which they are spherical or not.

9. Many of the modified gravity proposals mature enough to receive attention to their fit to cosmological data can meet that test as well. See, e.g., here.

10. In short, while a dark matter hypothesis alone can explain the apparently missing matter in any given situation, in order to get a descriptive theory, you need to be able to describe the highly specific manner in which it is distributed in the universe relative to the baryonic matter in the universe, ideally in a manner that predicts new phenomena, rather than merely post-dicting already observed results that went into the formulation of the model.

Modified gravity theories have repeatedly been predictive, while dark matter theories have still not figured out how to distribute it properly throughout the universe without "cheating" in how the models testing them are set up, and have failed to make any correct predictions of new phenomena below the cosmic microwave background radiation scale of cosmology.

Conclusion

To be clear, I am not asserting that modified gravity is indeed to correct explanation of all or any of the phenomena attributed to dark matter, nor am I asserting that any of the modified gravity theories currently in wide circulation are actually correct descriptions of Nature.

But, the examples of modified gravity theories that we do have are sufficient to make clear that some kind of modified gravity theory is a credible possible solution to the problem of dark matter phenomena.

It is also a more credible solution than it used to be because the case for the most popular dark matter particle theories has grown steadily less compelling as various kinds of dark matter candidates have been ruled out and as more data has narrowed the parameter space available for the dark matter candidates. The "WIMP miracle" that motivated a lot of early dark matter proposals is dead.

While this post doesn't comprehensively review all possible dark matter candidates and affirmatively rule them out (which is beyond the scope of the question), it does make clear that none of the easy solutions that had been widely expected to work out in the 20th century have survived the test of time into 2016. Over the past six years or so, only a few viable dark matter particle theories have survived, while myriad new modified gravity theories have been developed and not been ruled out.