Razib Khan's latest piece on human evolution in his Unsupervised Learning series takes a step back and looks at how the big picture has changed with new discoveries made in the last several decades.
His main point is that archaic hominin introgression may have originally been a much higher percentage of ancestry, with most archaic hominin introgression expelled in the roughly 1,000 years after the initial introgression due to natural selection and incompatibilities between archaic hominin sourced genes and modern human genes. I suspect that the 10%-20% percentage that he cites is quite a bit too high, but surely this did happen to some extent.
As usual, he educated synthesis of the research and evidence is pretty solid, even if not every assertion he makes in laying out a single coherent narrative (like the Population A and Population B that hybridize to form modern humans, with Neanderthals, Denisovans, and "ghost African" populations all derived from Population A hypothesis in the diagram below) has reached the level of academic consensus yet, and even if there is some room to quibble over fine details.
Some highlights of his piece (but please, click on the link and read the whole thing) are quoted below:
A generation ago, we imagined that Homo sapiens, “thinking man,” emerged fully formed in Africa over 100,000 years ago and swept away all our monstrous kin before us through dint of our sheer genius. Today, it seems more likely that it was we who were the monsters out of the dark, the demons about which Neanderthal mothers would tell their little-ones.In 2002’s The Dawn of Human Culture, Stanford paleoanthropologist Richard Klein presented what at that time was the standard model of the recent origins of humanity. Some time before 50,000 years ago, a new form of human arose through some sort of mutational jump. Klein posited that our genius, our superiority, was because of a macromutation that enabled us to generate fully articulate language. Before the emergence of this new species of human, there were many varieties of human, or, more precisely, hominin. Neanderthals in Europe and Central Asia, various “archaic” lineages in eastern Eurasia, and also descendants of other Homo forms in Africa.
Ω humans, whom many popular slogans would refer to as “Africans,” rapidly swept away all these varieties of humanity after 50,000 years.And so it was for a decade. There were dissents; in 2006 Jeffrey Wall and Michael Hammer published Archaic admixture in the human genome. This paper reflected suspicion among many evolutionary geneticists that the orthodoxy promulgated was too pat and simplistic (see also Magnus Nordborg’s 1998 On the Probability of Neanderthal Ancestry). But despite an underground counter-consensus, very few evolutionary geneticists were vocal on this issue in public. . . . This all changed in 2010, when Svante Paabo and colleagues reported that Neanderthal whole genomes yielded strong evidence of several percent admixture into non-Africans, as well as the discovery of a new human lineage in eastern Eurasia, the Denisovans, who also contributed about 5% of the ancestry of Papuans.Previous work had almost entirely been a matter of inference derived from contemporary populations. You looked at genetic variation in people alive today and worked backward to plausible models of how that variation could have arisen. In the 1980s, geneticists examined mitochondrial lineages, which represent the direct matrilineal genealogy. Geneticists noticed that all non-African populations nested within African genetic variation. They soon replicated this result with the Y chromosome, passed only through males, and the autosomes (markers on chromosomes 1 through 22) representing the whole genetic heritage. These results neatly dovetailed with the findings of paleoanthropologists like Chris Stringer of the British Museum of Natural History, who argued that modern human morphology, mostly exemplified by traits in human skulls, reflected continuity with African Homo, and not Neanderthals. Following the molecular genetic results, researchers applied similar phylogenetic methods to morphometric traits and discovered the same pattern of non-Africans nesting within African variation.It was an immaculate and tidy story. . . . And that still seems to be much of the story. But not the entire story.
Over the last few years, geneticists have concluded that a much higher fraction of non-African DNA was originally Neanderthal. In an interview with Dwarkesh Patel Harvard’s David Reich asserted that as much as 10-20% of the overall heritage of early non-Africans, just as they were expanding out of the Near East 50,000 years ago, may have been Neanderthal.
How then is that today we detect only about 2% Neanderthal genes outside of Africa?
The genome rapidly sheds genetically incompatible segments within a few thousand years via purifying selection. Because Neanderthals diverged from our predominantly African ancestors 600 to 700,000 years ago, their overall genetic makeup exhibited much more striking incompatibilities with the expanding Africans than occurs when different branches of our own species mix (the deepest division in our own lineage dates to about 200,000 years, when Khoisan ancestors diverged from everyone else). No doubt the same phenomenon applied to Denisovan admixture, which today in some Oceanian populations, like those in New Guinea or the indigenous populations of the Philippines, approaches 5% or so.What does this all mean?
Because natural selection changes allele frequencies in ways that are out of step with the overall genome, the signatures that we get from modern and ancient DNA are deceptive as to the demographic dynamics of early anatomically modern humans and the Neanderthals (and Denisovans) whom they encountered. All the evidence, both ancient and modern, points to a tiny non-African ancestry population between 50 and 60,000 years ago, a few thousand individuals at most (some models posit a bottleneck of 200 breeding individuals!). If 10-20% of the ancestry of the early modern human expansion, also known as the Initial Upper Paleolithic (IUP), was Neanderthal, that implies the integration of hundreds of Neanderthals, as opposed to ten or twenty.