A hundred years ago, Albert Einstein came up with the theory of General Relativity that was first presented publicly at a conference in November of 1915, and was published early in 1916 in a series of three papers.
General Relativity basically describes gravity in a way that is subtly different from that of Newton's simply F=GMm/r2 law of the 1600s that is perfectly sufficient for most purposes. But, its formulations allow for a variety of phenomena that Newtonian gravity did not.
One of the most important distinctions is that energy, not just matter, generates and is subject to gravitational fields, and that energy is equivalent to matter for purposes of the conservation of matter-energy, and for gravitational purposes, according to the formula E=mc2, where m is mass, E is energy, and c is the speed of light in a vacuum. For example, since light has energy, it gravitates and is affected by gravity, giving rise to the phenomena of gravitational lensing.
Another critical distinction between General Relativity and Newtonian gravity arises in strong gravitational fields, where singularities such as Black Holes and the Big Bang can arise. Both phenomena are observed.
There are other distinctions: frame dragging, gravitomagnetic effects, and more. But, they are beyond the scope of this post.
One integration constant in Einstein's formulation of general relativity, known as the cosmological constant full describes to the limits of astronomy data such as the Planck satellite observations, a phenomena know today as "dark energy" when set to the appropriate value.
Einstein's insights come to us virtually unchanged in the leading textbook on the subject, "Gravitation", written by Charles W. Misner, Kip S. Thorne and John Archibald Wheeler in 1973 (called MTW by advanced physics students everywhere).
It is widely asserted that the behavior of a massless spin-2 boson that couple to Standard Model particles and itself with a strength equal in magnitude to the mass-energy of the particle, reproduces general relativity. There is good reason to believe that this is wrong, and I discuss one of the reasons below. But, I think that the spin-2 massless graviton model discussed in some of Feynmann's lectures, may be a more accurate description of gravity itself, than it is of General Relativity.
The Problem of Dark Matter Phenomena
But, neither General Relativity nor the Standard Model of Particle Physics, describe a set of phenomena known as "dark matter" which is necessary to model the cosmology of the universe from the Big Bang onward in a way that matches Planck data, and is also necessary to describe, for example, the disconnect between the observed rotation curves of galaxies which do not match the naive predictions of simplified versions of General Relativity applied to the observed luminous matter in those galaxies.
There are two ways to reconcile these effects to General Relativity and the Standard Model.
One is to hypothesize the existence of "dark matter" that is massive, nearly collisionless with ordinary matter, made up of something other than the protons and neutrons that make up ordinary "baryonic" matter, and makes up the lion's share of matter in the universe.
The simplest model has just a single dark matter fermion particle, but many dark matter theories imagine the existence of dark forces that led to self-interaction of dark matter particles with each other by means other than gravity, or additional kinds of dark matter particles in a complex "dark sector" similar to that of the sector of ordinary matter described by the Standard Model.
At first it was hoped that this problem would be solved by Supersymmetry (SUSY) or string theory models, which would provide dark matter candidates.
Astronomers are no slouches and have worked hard on several fronts to infer the properties of the dark sector from observation.
One approach has been to use analytical and numerical models to determine what the universe should look like if it has a particularly kind and quantity of dark matter and then to compare the predictions of those models to what we actually observe. The dark matter hypothesis, set forth vaguely, does a very good job of fitting observed cosmic background radiation patterns (which give rise to radio static among other things) to very precise observations, but have proved less successful at predicting the amount of structure observed in the universe (e.g. how many dwarf galaxies surround the Milky Way galaxy and other galaxies of similar size) and halo distribution shape.
One approach has been to infer the distribution and mass of dark matter halos around galaxies from their rotation curves and known luminous matter. Similar inferences have been made in systems like the Bullet Cluster where a colliding galaxy provides a means by which to discriminate between theories, an in examinations of RAVE stars in the Milky Way galaxy that are outside the galactic plane. We have some idea regarding what shape dark matter halos must have to fit observations. Unfortunately, the shape of the halos observed are not a great fit to the NFW dark matter halo shapes we would expect from analytical simulations to see if they universe were made entirely of a single kind of "cold" (i.e. GeV to TeV mass) dark matter particles. Simulations match experiment between when gravitational interactions between baryons and dark matter are considered, but still show far more scatter in dark halo shapes than is observed. Simulations also tend to favor "warm dark matter" (in the keV particle mass range) over "cold dark matter" in the GeV to TeV particle mass range that lack self-interactions.
A third approach has been to try to directly detect dark matter particles at high energy colliders. This has not revealed any evidence of dark matter. But, they have made clear that dark matter must be made up of one or more types of particles not found in the Standard Model, if it exists, and these effects strongly constrain the parameter space of potential dark matter particles.
A fourth approach has been to in build direct dark matter detection experiments. So far, these have not revealed any convincing evidence of dark matter. There have been a few potential "hits" but those have been contradicted by more accurate measurements, or confirmed by experiments of similar accuracy.
A fifth approach has been to look at cosmic rays to see if any could be produced by a hypothetical dark matter annihilation interaction and have no other known source. Several candidates have been identified as potential signals of dark matter by these means, although there is not a consensus on how to interpret this data.
Modifications of Gravity
The other is to tweak General Relativity in weak gravitational fields of large objects such as galaxy and galactic clusters, in a way that reproduces phenomena attributed to dark matter. There are about half a dozen to a dozen ways of doing this in a way that reproduces the dark matter phenomena seen in galaxies with a fair degree of accuracy that have been published and compared to the data, the most famous of which is MOND, a toy model theory proposed in 1983 by Mordehai (Moti) Milgrom.
MOND itself is not the best of those theories. It is strictly a phenomenological relationship, it underestimates dark matter effects in galactic clusters, and it isn't great at predicting galactic dynamics outside the plane of a galaxy. But, it also has an impressive record of making firm predictions about unobserved phenomena that were later confirmed by observation, with only one new experimentally measured physical constant, and has a general relativistic generalization, TeVeS, devised by one of Milgrom's colleagues, Jacob D. Bekenstein.
MOG and similar theories proposed by John M. Moffat as the University of Toronto, has the distinction of having a broader range of applicability that describes phenomena in galactic clusters as well as galaxies.
A number of theories known as F(R) theories, which add a term that is a function of the Ricci scalar to the equations of general relativity, have also had some success in describing dark matter, dark energy and cosmological inflation.
If General Relativity Is Wrong, In What Way Is It Wrong?
General Relativity is a very tightly formulated set of equations based on a handful of mathematical first principles. But, perhaps quantum gravity or simply an omitted term in General Relativity could do what modified gravity theories that account for dark matter do.
To go beyond purely phenomenological models like MOND to a full fledged competitor to General Relativity, however, one needs a good theoretical justification for the modifications to the equations of General Relativity.
Deur has been at the forefront of demonstrating that the real key problem could be that conventional general relativity theory is wrong about the real world effects of gravitational self-interactions.
Section 20.4 of MTW at 467 is emphatic about this question:
To ask for the amount of electromagnetic energy and momen tum in an element of 3-volume make sense. First, there is one and only one forula for this quantity. Second, and more important, this energy-momentum in principle "has weight." It curves space. It serves as a source term on the righthand side of Einstein's field equations. It produces a relative geodesic deviation of two nearby world lines that pass through the region of space in question. It is observable. Not one of these properties does "local gravitational energy-momentum" possess. There is no unique formula for it, but a multitude of quite distinct formulas. The two cited are only two among an infinity. Moreover, "local gravitational energy-momentum" has no weight. It does not curve space. It does not serve as a source term on the righthand side of Einstein's equations. It does not produce any relative geodesic deviation of two nearby world lines that pass through the region of space in question. It is not observable.
Anybody who looks for a magic formula for "local gravitational energy-momentum" is looking forthe right answer to the wrong question. unhappily, enormous time and effort were devoted in the past to trying to "answer this question" before investigators realized the futility of the enterprise. Toward the end, above all mathematical arguments, one came to appreciate the quiet but rock-like strength of Einstein's equivalence principle. One can always find in any given locality a frame of reference in which all local gravitational fields" (all Christoffel symbols . . .) disappear. No [Christoffel symbols] means no "gravitational field" and no local gravitational field means no "local gravitational energy-momentum."
Nobody can deny or wants to deny that gravitational forces make a contribution to the mass-energy of a gravitationally interacting system. The mass-energy of the Earth-moon system is less than the mass-energy that system would have if the two objects were at infinite separation. The mass-energy of a neutron star is less than the mass-energy of the same number of baryons at infinite separation. Surround a region of empty space where there is a concentration of gravitational waves, there is a net attraction, betokening a positive net mass-energy in that region of space (see Chapter 35). At issue is not the existence of gravitational energy, but the localizability of gravitational energy. It is not localizable. The equivalence principle forbids it.Of course, in 1915, and even in 1973, the analogy of QCD, in which a force is carried by particles that are self-interacting (gluons in that case) was not known. But, QCD without self-interacting gluons would produce a very different effect.
A graviton, of course, is the very epitome of localized gravitational energy, which is why conventional General Relativity as espoused in MTW is fundamentally inconsistent with quantum gravity theories.
Deur argues, by analogy to QCD, that self-interacting gravitons do indeed have observable effects and gravitons curve space just like any other carrier boson would. To the extent that Einstein's equations do not reflect this fact, they are wrong. This, he argues with back of napkin estimates, produces dark matter phenomena of approximately the right amount in galaxies and galactic clusters, and accurately reflects the pattern seen in which more spherically symmetric systems have less apparent dark matter than those that have (in the original sense of the word) more pretzelosity.
Gravity is weak, and so, the gravitational self-interactions of gravity in low mass systems are modest. but, gravity is also cumulative, because it is always attractive, so in immense systems, gravitational self-interactions have material observable effects that probably give rise to substantially all dark matter phenomena, and by weakening gravitational fields in directions from which gravitons are diverted to give rise to dark matter phenomena effects elsewhere, also some or all dark energy phenomena.
I am quite convinced that the failure of the Einstein equations to reflect a contribution of gravitational self-energy is the most likely by far reason for dark matter phenomena that we observe and most of the dark energy phenomena that we observe, and that correcting this error will cause theory and observation to match exquisitely without the need for any beyond the Standard Model particles other than the massless spin-2 graviton that couples with a strength equal to the mass-energy of a particle.
I am confident that sooner or later, probably within ten to forty years, such a theory will be well articulated and tested against the data and will become the scientific consensus, and that dark matter theories will be discarded.
Thus, we will be left with the Standard Model and a very simple quantum gravity, with no dark matter or dark energy. Thus, the six quarks, three charged leptons, three neutrinos, photon, three weak force bosons, eight gluons and Higgs boson of the Standard Model, plus the graviton and their interactions according to four coupling constants, will prove to be the only particles needed to account for everything in the universe. All other proposed theories of fundamental physics will end up on the scrap heap of intellectual history. Maybe somebody will come up with a way to unify these pieces and explain the source of all of their constants, and maybe they won't. But, for practical purposes, it doesn't really matter one way or the other. The results will be the same.