From here citing Stephen Shennan et al., Regional population collapse followed initial agriculture booms in mid-Holocene Europe. Nature Communications (2013). (Open access).
The raw number of archaeological finds in Western and Northern Europe, aggregated suggest a deep (more than 50% from peak to trough), roughly 500 year long slump between the surge in population associated with the arrival of farming in the Atlantic area (the Atlantic Neolithic) and the immediate emergence of Megalithism from the Atlantic Neolithic, and the advent of the parallel Copper Age cultures known as "Corded Ware" (associated with Indo-Europeans) and "Bell Beaker" (associated with a non-Indo-European substrate in the region).
This slump rivals or exceeds subsequent major slumps in the region around 2000 BCE, around 1200 BCE, and in the early Middle Age and again about 800 years later. Climate events and documents plagues are reasonable well correlated with these slumps.
We understand far less well what caused the first one. There are some indications that this may be a case of primitive early Neolithic farming techniques leading to a boom-bust cycle as fertile land is depleted that moved like an echoing wave following the initial wave of the Neolithic. But, at this point, merely establishing that there was a major slump between the initial Neolithic and the Copper Age almost everywhere in West Eurasia, represents major progress relatively to previous models that assumed steady growth more or less continuously from the early Neolithic to the present. Climate could also be a suspect, but it if was we would expect the impact to be fairly uniform across West Eurasian, rather than being staggered, and it isn't at all clear that this is the case.
As further charts in the linked post indicate, however, the clear picture for the region as a whole is rather less clear when broken down by sub-region. The Atlantic Neolithic surge is pretty much universal. But, the subsequent Copper Age patterns vary considerably by region. Corded Ware was pretty much absent from the modern British Isles, Ireland and France, for example. Very different sample sizes in different regions (which may reflect genuine differences in population by region rather than simply differences in sampling strategy, or not) also influence how the regional results feed into the overall result.
The big surge in population with the Atlantic Neolithic is associated with a dramatic change in population genetics between the preceding Mesolithic era (i.e. late hunter-gather) and the Neolithic. The second Copper Age peak is associated with another significant change in population genetics in the region. Not later than about 3000 BCE (and possibly sooner), the population genetics of this part of Europe are substantially similar to those found in Europe today.
The results exclude Southern Europe and Eastern Europe where Corded Ware played a prominent role and Bell Beaker played a much more secondary role.
Pre-Neolithic Climate Conditions
Data from another fairly recent open access paper in PNAS by Samuel Bowles (of the interdisciplinary Sante Fe Institute) and Jung-Kyoo Choi (a Korean academic economist) entitled "Coevolution of farming and private property during the early Holocene" has pretty iffy analysis but good data. It indicates that during the Upper Paleolithic era immediately prior to the Neolithic, that climate was subject to dramatic shifts in temperature. The key paleoclimate data are presented in the chart below.
Ignoring their simulated data in light gray bar graph form (which flows from a problematic model):
Estimated dates of some well-studied cases of the initial emergence of cultivation are on the horizontal axis (8, 54, 55). Climate variability (Left) is an indicator of the 100-y maximum difference in surface temperature measured by levels of δ18O from Greenland ice cores (SI Appendix). A value of 4 on the vertical axis indicates a difference in average temperature over a 100-y period equal to about 5 °C.Pre-Holocene temperatures were also much colder, on average.
The Supplemental materials section linked above states:
Differences in temperature (Centigrade) are about 1.2 times the difference in the δ18O signal shown in Fig. S1 (2). The data indicate that changes in mean temperature as great as 8 degrees (C) occurred over time spans as short as two centuries. By way of comparison, the Little Ice Age that devastated parts of early modern Europe experienced a fall in average temperatures of one or two degrees, and the dramatic warming of the last century raised average temperatures by one degree, comparing the unprecedentedly hot 1990s with a century earlier (3, 4). The variability of climate during the late Pleistocene required high levels of geographical mobility, which was an impediment to any substantial investments in tree crops or field preparation or even stores and storage facilities. The scale and pace of climate change is truly extra ordinary: for example δ18O signals from sea cores indicate that between 25 and 60 ka, variations in sea surface temperature of 3–5 degrees Centigrade occurred over periods of 70 years or less in the Santa Barbara Basin, California (5) (sea surface temperatures today are about this different between the Santa Barbara Basin and northern Vancouver Island). Think about the frequency of moves and the distances that early humans may have traveled. A change of 9 degrees Centigrade in the course of a millennium appears to have been common prior to the Holocene. That's the difference in the average daily temperature in Cape Town and Mombasa 4 thousand kilometers to the north. While humans and the wild species on which they depended could of course adapt to a few degrees change in temperature, we infer that the distances covered and frequency of moves must discouraged the kinds of investment that farming requires.
As the chart indicates, intermittent periods of wild temperature variation over the span of just a few generations was the norm for the entire Upper Paleolithic era (from about 40,000-50,000 years ago until about 10,000 years ago), after which temperatures became much more stable starting at the beginning of the Holocene era about 10,000 years ago when farming first emerged in the Fertile Crescent and China and the middle latitudes of the Americas (farming arose independently at later times in the New Guinea Highlands, Sub-Saharan Africa and the Eastern United States).
So, a key part of the answer to the question of why farming emerged when it did is that the climate was too unpredictable for farming to be a viable means of food production during almost the entire Upper Paleolithic era. Farming didn't emerge before the Holocene because it couldn't in the climate conditions at the time.
Thus, a last post-Neolithic hurrah of Upper Paleolithic temperature variability illustrated in the second chart, that would have wrecked havoc on the crops of early farmers in Europe, could explain the slump that the experienced after the Atlantic Neolithic peaked.
Also, what has been interpreted by some as a behavioral revolution in the Upper Paleolithic that distinguishes anatomically modern humans from behaviorally modern humans, could simply have been a necessary cultural reaction to much more adverse climate conditions.