The Dark Matter Hypothesis
Dark matter theory proposes that one or more nearly collisionless particles give rise to the phenomena attributed to dark matter in the lambda CDM standard model of cosmology, in models of large scale structure formation in the universe, in the flat rotation curves of essentially all observed galaxies, and in the large discrepancies between the inferred mass of galaxies and galactic clusters and the mass of the luminous matter in these galaxies and galactic structure observed in the kinematics and in relativistic lensing measurements of their mass.
In principle, dark matter theory does not necessarily require new fundamental particles to give rise to dark matter, which the six parameter lambda CDM parameter measurements from experiments such as the Planck satellite measurements of cosmic background radiation suggest constitute about three-quarters of all matter in the universe.
But, ordinary matter made of baryons or of any of the Standard Model leptons or bosons (such as interstellar gas made of hydrogen and helium atoms, and interstellar dust) has largely been ruled out by astronomy observations, or because all known fundamental and composite particles lack the necessary particle properties.
So dark matter theory seems to require new, beyond the Standard Model physics that gives rise to new kinds of particles that have thus far not been detected experimentally.
The State of the Effort To Detect Dark Matter Particles
Scientists have not directly observed such dark matter particles, but they have ruled out large swaths of the parameter space for such particles that seem to be inconsistent with the empirical data. There have recently been false alarms in which dedicated direct dark matter detection experiments (recapped below), the Fermi experiment, and more recently observations of a 3.55-3.57 keV monochromatic X-ray emission line (recapped below) have observed something that seemed to be direct evidence of new physics dark matter particles. But, each time, these false alarms have been subsequently discredited.
This post recounts the most recent false alarm, and some of those dark matter parameter space exclusions.
The Impending Existential Crisis For Dark Matter
The non-detection of dark matter is not for want of a large community of physicists and astronomers devoted immense efforts in often reasonably well funded experiments to look for it.
The experimental quest for new physics dark matter particles is barreling towards an existential crisis point at which we either (1) identify and characterized dark matter fairly precisely given the myriad experimental and observational bounds on its existence, using direct or indirect evidence, or (2) we find that dark matter theory is overconstrained by the data and disproven, because all possible dark matter candidates can be ruled out.
The Problem With Gravitational Equation Based Understandings Of Dark Matter Phenomena
The trouble is that compelling data points like the Bullet Cluster observations also rule out many versions of the main competitors to new physics dark matter particles to explain dark matter phenomena, which involve modifications to gravity, or alternatively, to refined understandings of the non-Newtonian aspects of general relativity. Many kinds of gravitational force effects are likewise ruled out as an explanation of dark matter phenomena.
The non-Newtonian implications of general relativity have so far only been explored in selective simplifications of the equations of general relativity themselves, because these equations have so far provided too intractable mathematically to apply directly to complex real world systems.
But, it is possible, for example, that assumptions involved in simplifying the equations of general relativity to apply them to real world complex systems assume way non-Newtonian implications of these equations that would be revealed if other assumptions were made, or that the equations of general relativity are incorrect in some subtle way not revealed by experimental tests of General Relativity conducted to date. So far, experimental tests of the non-Newtonian features of general relativity have all resoundingly confirmed the real world validity of these equations, although not at anything approaching the levels of precision found in experimental confirmations of the Standard Model.
The New Data
* Jester at Resonances has noted indications that the 3.55-3.57 keV monochromatic X-ray emission from galactic clusters and Andromeda, is probably just excited potassium and chlorine atom emissions and not actually dark matter. In support of this conclusion, he cited the pre-print Tesla E. Jeltema and Stefano Profumo, "Dark matter searches going bananas: the contribution of Potassium (and Chlorine) to the 3.5 keV line" (August 7, 2014). The abstract for this paper states that:
We examine the claimed excess X-ray line emission near 3.5 keV with a new analysis of XMM-Newton observations of the Milky Way center and with a re-analysis of the data on M31 and clusters. In no case do we find conclusive evidence for an excess.
We show that known plasma lines, including in particular K XVIII lines at 3.48 and 3.52 keV, provide a satisfactory fit to the XMM data from the Galactic center. We assess the expected flux for the K XVIII lines and find that the measured line flux falls squarely within the predicted range based on the brightness of other well-measured lines in the energy range of interest.
We then re-evaluate the evidence for excess emission from clusters of galaxies, including a previously unaccounted for Cl XVII line at 3.51 keV, and allowing for systematic uncertainty in the expected flux from known plasma lines and for additional uncertainty due to potential variation in the abundances of different elements. We find that no conclusive excess line emission is present within the systematic uncertainties in Perseus or in other clusters.
Finally, we re-analyze XMM data for M31 and find no statistically significant line emission near 3.5 keV to a level greater than one sigma.A response has been posted here and in the comments at Resonances.
There had previously been strong claims that the 3.5 keV X-ray emission line detected by Bulbul and others was a warm dark matter annihilation signal.
Another pre-print just released August 20, 2014, casts further doubt on this signal because it is not seen in observations of 170 other galaxies.
We conduct a comprehensive search for X-ray emission lines from sterile neutrino dark matter, motivated by recent claims of unidentified emission lines in the stacked X-ray spectra of galaxy clusters and the centers of the Milky Way and M31. Since the claimed emission lines lie around 3.5 keV, we focus on galaxies and galaxy groups (masking the central regions), since these objects emit very little radiation above ~2 keV and offer a clean background against which to detect emission lines. We develop a formalism for maximizing the signal-to-noise of sterile neutrino emission lines by weighing each X-ray event according to the expected dark matter profile.
In total, we examine 81 and 89 galaxies with Chandra and XMM-Newton respectively, totaling 15.0 and 14.6 Ms of integration time. We find no significant evidence of any emission lines, placing strong constraints on the mixing angle of sterile neutrinos with masses between 4.8-12.4 keV. In particular, if the 3.57 keV feature from Bulbul et al. (2014) were due to 7.1 keV sterile neutrino emission, we would have detected it at 4.4 sigma and 11.8 sigma in our two samples. Unlike previous constraints, our measurements do not depend on the model of the X-ray background or on the assumed logarithmic slope of the center of the dark matter profile.From Michael E. Anderson, Eugene Churazov, and Joel N. Bregman, "Non-Detection of X-Ray Emission From Sterile Neutrinos in Stacked Galaxy Spectra" (18 Aug 2014) (emphasis added).
The bottom line is that even if warm dark matter sterile neutrinos do exist, they do not annihilate in X-ray emitting events of they kinds that would have been produced by the "Bulbulon" dark matter candidate.
Given the ordinary "fertile" neutrinos and anti-neutrinos do not directly annihilate in a manner that produces photons, in the way that collisions of charged matter-antimatter particle pairs do, this isn't too surprising. (Neutrino and anti-neutrino pairs could "annihilate" an give rise to a Z boson with which both neutral leptons can couple via the weak force. But, the neutrinos would need a combined 90.1 GeV of kinetic energy to make this possible, which would require that an ultra-relativistic neutrino and antineutrino collide, and such energetic neutrinos are largely restricted to rare cosmic ray neutrinos such as those recently detected by the IceCube experiment that are probably emitted by blazars. Alternately, neutrino and antineutrino pair could annihilate into particle-antiparticle decay products of a Z boson (including low energy photons which would be the only Z boson decay products that are energetically permitted in many cases) with less combined mass-energy than the neutrino-antineutrino pair via a virtual Z boson. But, while this quantum tunneling effect is possible, it is suppressed in frequency due to the large amount of mass-energy that must be "borrowed" to cross the 90.1 GeV Z boson mass threshold, if the colliding neutrinos are less energetic.)
But, the strongest direct evidence for the detection of warm dark matter now seems to be discredited.
* Meanwhile, Tomasso Dorigo reports on dark matter exclusions from the CMS experiment at the Large Hadron Collider which when added to the exclusions from the LUX direct dark matter experiment, rule out a huge part of the dark matter parameter space.
Between CMS and LUX scalar dark matter particles are ruled out in the mass range of about 1 GeV to 1 TeV, down to cross sections of interaction of 10-44 cm2, and down to cross sections of interaction of 10-45 cm2 for dark matter particles in the mass range of 1 GeV to 200 GeV. For vector dark matter particles, the corresponding cross section of interaction exclusions are 10-38 cm2 and 10-40 cm2, respectively.
The new LHC data exclude more of the parameter space mostly in the dark matter boson mass ranges from 1 GeV to 10 GeV.
These direct dark matter exclusions add another nail in the coffins of the CDM (cold dark matter) and WIMP (weakly interacting massive particle) dark matter paradigms. All but the heaviest forms of CDM are excluded and the cross sections of interactions are too slight for any particle that has the same weak force coupling constant as a neutrino and not other Standard Model interactions.
Neither the LHC, nor the various direct dark matter detection experiments are sensitive enough at low masses to rule out WDM (warm dark matter, generically, in the mass vicinity of a keV) with the same certainty, and astronomy data tend to prefer WDM models over CDM models, or other very light dark matter particle candidates such as hypothetical particles called axions which are in the "hot dark matter" mass range that has been experimentally ruled out, but fit within an exception to that exclusion because axions would not be produced as thermal relics, unlike most proposed forms of dark matter But, precision electroweak data from LEP does exclude the existence of any new weakly interacting particles with masses of less than 45 GeV that are heavier than the three Standard Model massive neutrinos.
Supersymmetry (SUSY) models, generically, predict the existence of beyond the Standard Model particles that are likewise too heavy, given current experimental bounds.
Other Constraints On New Dark Matter Sectors
As I noted in a post in January of this year, the astronomy data, in addition to the direct detection and collider data, severely constrain the dark matter parameter space. Some key conclusions from that post:
The purest form of sterile neutrino, with a particular mass and no non-gravitational interactions at all, is ruled out by observational evidence from the shape of dark matter halos.
* No particles that could produce the right kind of dark matter halo are produced in the decays of W and Z bosons, ruling out, for example, any neutrino-like particle with a mass of 45 GeV or less. In other words, no light dark matter candidate can be "weakly interacting".
* Direct detection of dark matter experiments such as XENON and LUX rule out essentially all of the cold dark matter mass parameter space (below 10 GeV to several hundreds of GeVs with the exclusion most definitive at 50 GeV) through cross-sections of interaction on the order of 10^-43 to 10^-45 which is a far weaker cross section of interaction than the neutrino has via the weak force.
The data rule out any kind of interaction between between cold dark matter and ordinary matter via any recognizable version of the three Standard Model forces (electromagnetic, weak and strong). Of course, by hypothesis, dark matter and ordinary matter interact via gravity just like any other massive particles. Thus, interactions between dark matter and ordinary matter other than via gravity are strongly disfavored.
* Dark matter has zero net electric charge (if dark matter is composite and confined, in principle, its components might still have electric charge) and is not produced or inferred in any strong force interactions observed to date in collider experiments.
* XENON also places strong limits on interactions between ordinary photons and "dark photons" found in some self-interacting dark matter theories.
To explain dark matter phenoma, one needs at least a new dark matter fermion and a new massive boson carrying a new force, because purely collisionless dark matter models don't fit the data.
* Purely collisionless dark matter (i.e. dark matter that interacts with other dark matter only via the gravitational force), that has a particular mass anywhere from the keV range to the TeV+ range produces cuspy halos inconsistent with observational evidence. (But, quantum mechanical effects when dark matter halos become dense, and gravitational interactions between ordinary baryonic matter and dark matter could mitigate these problems. Arguments that Fermi pressure in the case of fermionic dark matter can solve the "cuspy core problem" were discussed at a recent conference on Warm Dark Matter for example, in this power point presentation by de Vega citing work supporting the WDM paradigm). These papers also argued that angular momentum can discourage, but not sufficiently prevent clumpiness, and that collisionless GeV mass cold dark matter simulations always over predict the abundance of dark matter in the central of galaxies.)
* Models with multiple kinds of collisionless dark matter simultaneously present in the universe at the same time produce worse fits to the data than single variety of collisionless dark matter models.
* Collisionless bosonic dark matter, as well as fermionic collisionless dark matter, is likewise excluded over a wide range of parameters.
* Self-interactions between dark matter particles with each other with cross-sections of interaction on the order of 10^-23 to 10^-24 greatly improve the fit to the halo models observed (self-interactions on the order of 10^-22 or less, or of 10^25 or more, clearly don't produce the inferred dark matter halos that are observed). Notably, this cross section of self-interaction is fairly similar to the cross-section of interaction of ordinary matter (e.g. helium atoms) with each other. So, if dark matter halos are explained by self-interaction, the strength of that self-interaction ought to be on the same order of magnitude as electromagnetic interactions. But, our observations and simulations are now sufficiently precise that we can determine that ultimately, a simple constant coupling constant between dark matter particles, or even a velocity dependent coupling constant between dark matter particles, fails to fit the inferred pseudo iso-thermal ellipsoid shaped dark matter halos that are observed. Generically, these simple self-interacting dark matter models generate shallow spherically symmetric halos which are inconsistent with the comparatively dense and ellipsoidal halos that are observed.
* Experimental evidence has not yet ruled out next generation self-interacting dark matter models look at more a general Yukawa potential generated by dark matter to dark matter forces with massive force carriers (often called "dark photons") that have masses which empirically need to be on the order of 1 MeV to 100 MeV (i.e. between the mass of an electron and a muon, but less than the lightest hadron, the pion, which has a mass on the order of 135-140 MeV) to produce dark halos that are a better fit to the dark matter halos that are observed. Sean Carroll was a co-author on one of the early dark photon papers in 2008.
* Rapidly accumulating evidence regarding the properties of the Higgs boson disfavors new heavy particles that gain their mass via the Higgs mechanism. But, these measurements are not so precise that they could disfavor new light particles that gain their mass via the Higgs mechanism. Current experimental uncertainties in this equivalence could accommodate both a new massive boson of 100 MeV and a new massive fundamental fermion of up to about 3 GeV, so both particles could couple to the Higgs boson and obtain their mass entirely from interactions with it, even though they don't couple to the other Standard Model forces. But, reduced margins of error in measurements of the Higgs boson mass and top quark mass could tighten this constraint.
Two Plausible Corners of Dark Matter Parameter Space Remain In Play
The case to rule out a simple warm dark matter scenario with a 2-3 keV dark matter particle that interacts only via gravity and Fermi contact forces is not yet ruled out. It may avoid, via Fermi contact forces, cuspy core problems that are generically a problem in heavier cold dark matter scenarios.
Another part of the dark matter parameter space (also here) generates the appropriate halos with a fairly light dark matter candidate with a dark photon of a mass ca. 1-100 MeV and a fermionic dark matter particle under 3 GeV, neither of which has any meaningful non-gravitational interaction with ordinary matter. [Update August 27, 2014 - the linked papers actually suggests a 1 TeV mass dark matter particle and manages to make a complicated mixed dark matter spectrum with additional sterile neutrinos that couple to dark photons as well work.]
De Vega, in the power point presentation linked above, however, makes a pretty convincing argument that axion dark matter is also a poor fit to the data.
On the other hand, if both of the small valid remaining corners of the dark matter parameter space are ruled out, then the experimental data is close to ruling out dark matter models entirely.
In either case, dark matter was clearly be almost entirely outside the domain of Standard Model physics. There must be a dark sector that has almost no interactions with it.
Gravitational Approaches Aren't As Definitively Ruled Out As They Seemed To Have Been
Physicist Alexandre Deur claims to have identified a non-linear, non-Newtonian aspect of the canonical equations of General Relativity interpreted in the context of a graviton field theory involving the non-Abelian self-interactions of gravitons that could explain all or more dark matter phenomena in a manner that evades the limitations arising from observations of the Bullet Cluster that have dealt a serious blow to gravity modification theories, because his gravitational effect is suppressed in spherically symmetric systems (an assumption found in many attempts to analyze the non-Newtonian effects of general relativity).
Even if Deur is wrong in concluding that these effects are present in the general relativity equations themselves, which have not been replicated by any other the many other general relativity theorists in the last hundred years, he demonstrates that the Bullet Cluster data is not necessarily an insurmountable barrier to explanations of dark matter phenomena via the equations of gravity. So, the primary alternative to the dark matter hypothesis is not entirely dead.
Despite Deur's obscurity, and the lack of consensus regarding his findings (although I am not aware of any categorical refutation of his conclusions either), as the parameter space of potential dark matter candidates grows more narrow and requires more elaborate new physics to explain it, and as potential signals of direct dark matter detection continue to be false alarms, Occam's razor is beginning to favor gravitational alternatives to dark matter.