Dilution Or Selection?

A new paper from Nature focuses on 51 ancient genomes from the Upper Paleolithic.

One notable observation is that Neanderthal admixture falls from 3%-6% in the early Upper Paleolithic to current levels of about 2%. This is attributed to selection, although dilution with less admixed populations could produce the same top line result. (Oase1 is an outlier at 10% from 40kya).

The paper's analysis suggests also that most Neanderthal admixture is quite old (long before cohabitation of Europe) and decayed slowly and steadily through slight natural selection, presumably in West Asia or SW Asia, rather than Europe. Indeed, the model is consistent with zero admixture of modern humans and Neanderthals in Europe itself.

Eurogenes captures many observations from the comments.

Broad brush, modern populations are really only in continuity with historic populations in Europe back to the Epipaleolithic era ca. 14,000 years ago when Europe's population was replaced following the Last Glacial Maximum when the Western Hunter Gatherer (WGH) autosomal population starts to gel.  Earlier individuals loosely cluster around MA1 from 24,000 years ago with one individual from ca. 19,000 years ago looking like a transitional figure.

This is one Epipaleolithic individual from Northern Italy ca. 14,000 year ago with Y-DNA R1b.  There are stronger Asia affinities in European individuals than would be expected for most of the Upper Paleolithic.

The picture form uniparental ancient DNA until now has been one of a very narrow, homogeneous gene pool with a small effective population size.  To the extent that this is true, it is a post-LGM phenomena as the ancient autosomal DNA over the tens of thousands of years and thousands of miles spanned by the sample shows only fairly loose affinity.

Two out of 21 pre-LGM samples are mtDNA-M (now found almost exclusively in East Eurasia).  Of 13 pre-LGM Y-DNA samples, three of C (now found almost exclusively in East Eurasia) and four are haplogroups that precede the East-West divide in Y-DNA haplogroups.