Empirically, no one has ever observed anything with more mass per volume than a neutron star, which is just slightly more dense than atomic nuclei.
The conventional argument for an absence of superdense objects.
The reason that there is nothing with a greater volume that is more dense is fairly straightforward. Anything with a volume much greater than a neutron star and the same mass per volume would be a black hole. And, once something is a black hole, its event horizon which defines its volume, is purely a function of its mass. And, the more massive a black hole happens to be, the lower its mass per volume becomes.
The reason that there is nothing with a lesser volume than a neutron star that has more mass per volume is less obvious. The conventional account goes something like this:
Non-black hole mass that existed after the Big Bang had cooled down a bit congeals over time into bigger and bigger lumps through gravity that sometime form bigger and bigger starts that, if they are large enough, give rise to stellar black holes caused by the collapse of big stars.
But, until enough mass congeals in this way, gravity isn't strong enough to make the glob of matter collapse in upon itself, and so a new black hole doesn't form.
Maybe there were smaller black holes once upon a time, but Hawking radiation from these smaller black holes would have caused them to gradually lose mass over the more than thirteen billion years since the Big Bang, so now any remaining ones are tiny or have ceased to exist.
The ansatz supporting a maximum density conjecture.
In this post, I'd like to consider an alternative explanation. Perhaps it is not possible for matter that is stable for more than a moment to have density greater than a neutron star.
The maximum density conjecture explored in this point is one that I suggested in a previous post at this blog.
This post expands on the hypothesis by suggesting a mechanism arising from the asymptotic freedom demostrated by the strong force and the fact that most of the ordinary mass in the universe comes from the gluons in protons and neutrons, to explain how a theoretical maximum density might arise from ordinary standard model physics.
If this is true, it follows that there are not now and never have been at any time in the last thirteen billion years or so, black holes with the same volume as neutron star, or a smaller volume.
There are a few key observations that suggest this alternative explanation:
1. Dark energy appears to have a uniform density throughout the universe, which is infinitesimally small, or may not be a substance at all and may instead be simply a cosmological constant which is one part of the equations of general relativity. So, mass-energy attributable to dark energy can be ignored in this analysis.
2. Virtually all of the ordinary matter in the universe whose composition we understand comes from baryonic matter in the form of protons and neutrons. The proportion of the mass in the universe that comes from electrons orbiting atomic nuclei or in free space, and from neutrinos appears to be negligible by comparison. There is also no evidence whatsoever that there is any place in the universe outside of a black hole in which there exist stable baryons or mesons other than protons and neutrons that make up a meaningful percentage of matter.
There is also no evidence that dark matter plays an important role in giving rise to highly compact objects in space, and that the amount of ordinary matter relative to dark matter is partially due to an undercounting of ordinary matter. The role of dark matter is one of the weaker links in this analysis, however, and it limits its rigor and generality. If dark matter is a neutrino condensate, for example, that type of matter might or might not escape the maximum density limitations considered here. There would have to be a completely separate analysis to determine the impact of dark matter on this conjecture. The analysis that follows is really only a weak version of the conjecture applicable to cases in which dark matter does not play a material role.
3. Within a proton or neutron, the vast majority of the composite particle's rest mass is not attributable to the three quarks that make up the particle. Instead, the bulk of the rest mass of these baryons is attributable to the energy in the strong force interactions of the three component quarks which come quantized in gluons which are the strong force equivalent of photons. (While not obviously relevant to this particular result, it is also worth noting as an aside that most of the strong force field is localized a the center of the three quark system, rather than at the edges.)
4. The strength of the strong force between two quarks is a function of their distance. When two quarks are sufficiently close the strength of the strong force between them declines and they become what is known as asymptotically free. Thus, if you reduce the average distance between a group of quarks, the amount of energy in the strong force field declines. A proton whose quarks are momentarily closer to each other with a given combined kinetic energy should weigh less than a proton whose quarks are at the usual average distance from each other with the same combined kinetic energy.
5. Thus, it would seem that, the rest mass of a group of tightly spaced quarks bound to each other by gluons is lower than the group of a less tightly spaced quarks bound to each other by gluons. Unlike the substances we are familiar with if you could squeeze quarks close enough they would get less dense not more dense like all ordinary matter does.
6. The quark spacing at which the mass per volume of the quark-gluon system is greatest is that found in a neutron star. If the same number of quarks in a gluon star were squished together more tightly and had the same average kinetic energy as they did within a neutron star, they would have less mass per volume.
7. But, if they had more kinetic energy per quark on average than in a neutron star, they would rapidly spread out until the asymptotic freedom that the strong force affords to closely spaced quarks faded away and the strong force got stronger.
8. A black hole smaller than a neutron star in volume must be much more dense than a neutron star, but because most of a neutron star's mass comes from the strong force fields between its quarks, and because momentary conversions of strong force sourced mass into kinetic energy can't reach this threshold in a large enough volume with such a large number of particles that the law of averages becomes overwhelming even for very short time frames, systems as large as a neutron star can't "tunnel" into a density high enough to create a black hole by converting strong force energy into kinetic energy in a manner that randomly happens to materially reduce the volume of the system by an amount sufficient to create a black hole.
Is the loophole for asymtoptically free quark masses real?
Now, there is a conceivable loophole here, because is you get enough quarks close to each other (packing them ten to a hundred times more tightly than in an ordinary proton or neutron), the increase in mass from squishing the quarks together would exceed the decrease in mass from the virtual disappearance of the strong force field between these quarks.
But, the problem with this loophole is that there is nothing in the strong force or weak force or gravitational force that should squish so many quarks together so tightly over any substantial volume for any meaningful length of time, and presumably at some point the repulsive effects of electromagnetic charges between same charged quarks will kick in as well. And, at the scale of a simple proton or neutron's volume this concentrated bunch of asymptotically free quarks must be dense indeed to give rise to a black hole. Even if this could theoretically happen, the time frame for it to happen once on average may be much longer than the age of the universe.
Also, the fact that some quarks would be a full strength strong force distances from each other in such a system could be problematic. It isn't clear what happens with the strong force in such closely spaced quark circumstances involving large numbers of quarks.
A maximum density conjecture could resolve the theoretical inconsistency betwen quantum mechanics and general relativity by rendering the inconsistent possibilities unphysical.
In addition to explaining the absence of any observed small black holes, and removing any concern that something like the LHC could ever create a mini-black hole, this analysis could also have another useful theoretical benefit.
A maximum density conjecture cordons off many, if not all, of the situations in which there is conflict between general relativity's assumptions about space-time being continuous, and quantum mechanic's assumptions about fundamental particles being point-like, as unphysical. The conflicts are deepest when there is a general relativity singularity and this seems to be avoided in most if not all systems small enough to exhibit strongly quantum mechanical behavior with a maximum density conjecture.
Also, it may be that if one attributes reality for the purposes of general relativity calculations to the notion that point-like quantum mechanical particles are actually smeared over the entire volume of the probability distribution for their location at any given point in time, rather than being truly point-like, that the problem associated with anything that has mass and is truly a point giving rise to a singularity (i.e. a black hole) of infinite density in general relativity could be resolved (and indeed, this principle might even provide a theoretical limit on the maximum mass of any fundamental particle). I have to assume that someone has looked at this issue before but I don't know what the analysis showed and have never seen any secondary source provide an answer one way or the other even though the calculations would seem to be pretty elementary in most cases. The probability distribution needs to be roughly on the order of 10^-41 meters for a fundamental particle (also here) to escape this inconsistency, which is much smaller the the Planck length, so intuitively it seems like this smearing principle should suffice to resolve the infinite mass of a point-like particle singularity.
In effect, this maximum density conjecture, if correct, might imply that it is not necessary to modify general relativity at all in order to make it theoretically consistent in every physically possible circumstance with quantum mechanics, allowing a sort of unification without unification of the two sides of fundamental physics. It has long been known that the quantum mechanical assumptions that are at odds with general relativity theoretically generally aren't a problem in real world situations where we apply GR and visa versa, but we've never had any good way to draw a boundary between the two, which would provide theoretists to way to explore the very limits of physically possible circumstances where they can overlap. A maximum density conjecture might resolve that point.
Implications of a maximum density conjecture for inflation in cosmology
A maximum density conjecture also suggests that it may not be sound to extrapolate the Big Bang back any further than the point at which it reached a volume at which it had maximum density. Indeed, the aggregate mass of the universe would give rise to a black hole singularity at a volume at which it would have a mass per volume of much much less than a neutron star. The notion of inflation in Big Bang cosmology could simple be a mathematical artifact of extrapolating back to a point in time of more than the maximum density and hence assuming an unphysical state for the infant universe. Arguably, it really doesn't make any sense to extrapolate further back than the first point at which the universe's mass per volume would be insufficient to give rise to a black hole which would be much later in time than the point at which a maximum density threshold would be reached. Unless inflation happens after this point in time, inflation itself may be unphysical.
This wouldn't resolve the why was there something rather than nothing at the moment of the Big Bang, but the current GR without maximum density formulation doesn't answer that question either. The only real difference would be in the initial conditions of the two models.
Implications for causality
A theoretical and practical maximum density would also act to preserve causality and to even strengthen it beyond the limitations that would apply in the absence of this limitation by putting a cap upon the maximum extent to which gravity slows down time outside a black hole, with the cap being particularly strict outside the vicinity of large maximum density objects like neutron stars. This effect is tiny in the vicinity of atomic nuclei compared to similar density black holes because they have a much smaller volume.
Why is there a gap of medium sized maximum density objects?
A maximum density conjecture doesn't itself solve another empirical question. Why are there no intermediate sized maximum density objects, and for that answer to that one might need to resort to dynamical arguments.
There do not appear to be any objects even approaching maximum density for volumes greater than large atomic nuclei, which the weak force and limited range of the strong force tends to shatter and in which gravity is negligible in effect, and collapsed stars on the same order of magnitude of mass as the sun, countless orders of magnitude more massive. The strong force, the weak force, the electromagnetic force, and gravity all seem to lack the capacity to compact matter to that degree at those scales.