The density of an atomic nucleus is roughly the same as the density of a neutron star, and both are made of basically the same stuff: neutrons and in atoms, protons of nearly identical mass and volume.
The heaviest possible neutron star has a mass of roughly 2-3 solar masses.
The density of a black hole (dividing its total mass by the volume enclosed by its event horizon) is proportional to 1/M^2, which M is the aggregate mass of the black hole. The smallest black holes have a density that is at most slightly greater than a neutron star and a mass of about 3 solar masses. The larger a black hole gets, the lower its density. The supermassive black hole at the center of the Milky Way galaxy had a density roughly the same as water, and larger galaxies have even larger, and less dense supermassive black holes.
All known black holes have masses in excess of the mass of a neutron star.
One of the great mysteries of the universe is why there are not any lighter black holes.
Black holes of any mass are theoretically permitted by general relativity, so long as the event horizon is small enough, although black holes with a mass of less than the moon emit Hawking radiation at rates greater than the cosmic background radiation of the universe and thus would tend to evaporate over time. But, no similar barrier exists with regard to black holes with masses greater than the moon and smaller than two or three times the mass of the sun.
Heuristically, the reason seems to be that even if they would be stable or grow slowly once brought into being, there is apparently no natural process that can commence the formation of a black hole of that size. A black hole smaller than a neutron star would have to have a density greater than a tight packed mass of neutrons (whose minimum volume outside a black hole is a product of the strong force and their status as fermions) without relying solely on gravity to compress them further. It takes two or three stellar masses of gravitational pull to generate that kind of gravitational pull, so any excess compression would have to come from electromagnetism (since the strong force and weak force do not operate beyond short ranges), and there is apparently no natural process that generates a charge distribution shaped in a manner that would generate that kind of additional compression.
Protons and neutrons are the only stable hadrons. Roughly speaking, all baryons take up the same amount of space and all mesons take up the same amount of space, regardless of their quark content.
The mass of a hadron is equal to the mass of the quarks that go into it (at least in the case of baryons, most pseudoscalar mesons and the vector mesons, where the quark content is well defined) plus the mass attributable to the gluon field binding the hadron together. As a first order approximation, the mass of the gluon field is the same for all hadrons with the same JPC quantum numbers.
Protons and neutrons have masses of about 0.94 Gev of which more than 98% comes from the gluon field with the rest coming from the up and down quarks inside them. The heaviest hadrons made of two or three quarks (apart from highly theoretical hadrons with hadronized top quarks which have never been observed in real life) would be a triple bottom omega baryon with a mass of roughly 15.9 neutron masses (rounded to 16 for easier math).
In theory, assuming that all baryons have roughly the same volume, a black hole with the density of a triple bottom omega baryon would have a mass of roughly 1/2 to 3/4 of the mass of the sun (i.e. about a quarter of the mass of the heaviest possible neutron star), and would have a smaller volume than either a white dwarf or a neutron star by almost an order of magnitude.
Of course, if a black hole that size really did exist, we would expect it to be stable and growing, while a triple bottom omega baryon would have a lifetime shorter than the almost 10^-10 second lifetime of an omega baryon made of three strange quarks, but probably longer than the 5*10^-24 second lifetime of the delta baryons and rho mesons which are the shortest lived hadrons whose lifetimes have been measured.
All black holes simultaneously emit Hawking radiation and absorb the cosmic background radiation that they are enveloped in, even if they are otherwise stranded too far from fermionic matter and photon sources to absorb any meaningful quantity of either relative to the cosmic background radiation that it absorbs. This input and output reach equilibrium at a black hole mass equal to about the mass of the moon, and an event horizon with a roughly 0.0001 meter diameter (about the size of a small grain of sand).
Larger black holes grow over time, because they absorb more mass-energy than they emit. Smaller black holes shrink over time (sometimes evaporating to nothing in fractions of a second), because they emit so much Hawking radiation relative to their mass. The expected lifetimes of smallest of the black holes small enough to shrink over time can reach time frames grossly similar in scale (or smaller than, sometimes by many orders of magnitude) the lifetimes of unstable hadrons, top quarks, muons, tau leptons, Higgs boson and weak force bosons (10^-6 seconds to 10^-25 seconds) as they reach masses of the TeV scale or so.
Now, in reality, there are no observed black holes with masses of less than about 3 stellar masses (i.e. lighter than the heaviest neutron star). So, in reality, the most dense composite system in the universe is probably a triple bottom omega baryon.
The proton and neutron are by far the least dense of the baryons. A number of mesons have less mass than a proton (the pions, the rhos, the kaons, the eta meson, and the omega meson), put mesons presumable take up less volume than baryons.
If mesons have two-thirds the volume of a baryon (i.e. if volume is proportional to the number of quarks), then pions, pseudo-scalar kaons, and the eta meson (as well as the scalar f(500) meson) have less density than a neutron, while rhos, vector kaons, and omega mesons would have more density than a neutron. If volume of a hadron is proportional to square or cube of the number of quarks, then pions are the only mesons that have a density lower than neutrons.
Then again, because two mesons can be in the same place at the same time because they are mesons, perhaps the volume of a meson is not well defined (something that could also be true of fundamental particles including the Higgs boson, W boson, Z boson, photons and gluons).
But, most unstable hadrons have densities that are greater than a neutron star.
One can feel somewhat comfortable ignoring point particle singularities of individual quarks within a hadron. After all, a radius of less than 10^-41 meters, which is one ten millionth of a Planck length, is all that would be necessary to prevent these point-like particles from becoming black hole singularities and no current technology or experiment is capable of distinguishing between a particle so tiny and a point particle.
Point particle singularities are harder to disregard in the case of electrons, muons, tau leptons, Higgs bosons, top quarks, W bosons and Z bosons that are not contained in composite particles and hence have much less well defined volumes (and as noted above, there may also be definitional problems with applying common sense notions of volume to bosons). They have a wide range of mean lifetimes.
Top quarks (which weigh about 173.4 GeV), W bosons (which weigh about 80.4 GeV) and Z bosons (which weigh about 91.1 GeV) all have mean lifetimes on the order of 10^-25 seconds with the lifetime of the top quark being slightly longer than the W and Z boson lifetimes. The characteristic time frame of hadronization is on the order of 10^-24 seconds. Higgs bosons (which weigh about 125.9 GeV) have a mean lifetime on the order of 10^-22 seconds. The tau leptons (which weigh about 1777 MeV) has a mean lifetime of about 2.9*10^-13 seconds. Muons (which weigh about 105.7 MeV) have a mean lifetime of about 2*10^-6 seconds. Electrons (which weigh about 0.511 MeV) are stable, of course, as are neutrinos (although they oscillate) and photons. Gluons are, in principle, stable as well, but in practice are emitted and absorbed in time periods comparable to characteristic hadronization times and while they have zero rest mass, acquire mass in the hundreds of MeV dynamically at low energies.
Now, if Compton wavelengths are appropriate to use in lieu of the point particle approximation, then muons and electrons (and strange quarks and up and down quarks) have lower densities than neutrons, tau leptons (and charm quarks) have densities similar to neutrons, while top quarks and the electroweak bosons are the most dense objects in the universe by a huge margin. But, at these quantum gravity scales, it isn't at all obvious how to generalize the concept of density to seemingly point-like particles. Also, even the heaviest of these is much longer lived than a micro-black hole of comparable mass given the impact of Hawking radiation to evaporate them.
Of course, while we don't know for sure that fundamental Standard Model particles are true point particles, or if general relativity still works at such tiny distance scales, we do know that massive fundamental Standard Model particles like quarks and electrons are (at a minimum) much smaller in radius than a proton or neutron.