Free Parameters

The paper below has a basic problem. 

It looks at how well nine different models fit observed galaxy rotation curves using a Chi-squared test. But it doesn't fully adjust for the fact that some models have more free parameters than others, even though it does used a reduced Chi-square test. It also fails to consider the the dark matter models aren't predictive.

When you compare to different models to see which is a better description of the data, while the Chi-square goodness of fit is important, you are supposed to penalize models with more free fitting parameters, usually using Akaike’s Information Criterion. Why?

In estimating the amount of information lost by a model, AIC deals with the trade-off between the goodness of fit of the model and the simplicity of the model. In other words, AIC deals with both the risk of overfitting and the risk of underfitting.

Another test used for the same purpose that is subtly different is Bayesian Information Criterion which is designed for Bayesian statistical analysis as opposed to frequentist statistical analysis like that in the AIC (other criteria exist too, such as the Hannan-Quinn Information Criterion, but are similar and less often used).

In other words, if you have enough free parameters you can make even a bad model fit the data, but this doesn't necessarily make the model better. Some of it is just smoothing out noise without adding value on a case by case basis.

The money chart is this one (which makes dark matter models, in general look much better than modified gravity models, and makes some of the dark matter models look a bit better than others, although the large uncertainty in their estimation of the modified gravity theories also exaggerates the differences, each of which have fit values within two sigma of the best fits that are below the mean values of the dark matter models, and hence are also not statistically significant differences):

But this breaks down when you see that the dark matter models each have three or for parameters adjusted on a galaxy by galaxy basis to maximize the quality of the fit, while the modified gravity models have just one (which is shared with the dark matter models) in addition to a parameter that is fixed for all galaxies considered:

The paper's comparison of the fits of the dark matter models which have either three or four parameters each, is more fair since it is more comparable (and has a much lower uncertainty as well, which also makes it more valid). The differences between the dark matter models aren't statistically significant. But, the data do weakly disfavor the NFW halo shape that standard sterile cold dark matter theory predicts that dark matter halos should have, as analytical calculations can establish.

The preprint and abstract are as follows: 

We use the galaxy rotation curves in the SPARC database to compare 9 different dark matter and modified gravity models on an equal footing, paying special attention to the stellar mass-to-light ratios. We compare three non-interacting dark matter models, a self interacting DM (SIDM) model, two hadronically interacting DM (HIDM) models, and three modified Newtonian dynamics type models: MOND, Radial Acceleration Relation (RAR) and a maximal-disk model. The models with Gas-DM interactions generate a disky component in the dark matter, which significantly improves the fits to the rotation curves compared to all other models except an ad-hoc Einasto halo; the MOND-type models give significantly worse fits.

Nicolas Loizeau, Glennys R. Farrar, "Galaxy rotation curves disfavor traditional and self-interacting dark matter halos, preferring a disk component or ad-hoc Einasto function" arXiv 2105:00119 (April 30, 2021).