Lousy Experts: Looking Back at the Ex Ante Estimates of the Costs of Brexit

Yves here. It’s revealing to see the amount of economist hand-wringing in the wake of the Leave vote. And this article, while it makes some good points, makes a strong claim at the very top of the article that was contradicted in an article we ran yesterday, that there was a great deal of agreement among economists regarding the cost of a Brexit. The Institute for New Economic Thinking said the same thing in an earlier post we featured, that there was considerable divergence on the potential impact.

As reader fosforos said:

Speaking as an economist, it is totally clear to me that no economically rational voter would pay a moment’s attention to any “Brexit” prognostications by any economist. Why? Firstly, because the question to be voted on offers two choices “Stay or Leave” without any clue whatsoever about how either choice would be implemented. “Stay” might seem a simple default choice, but its consequences would depend on the future evolution of an EU already in crisis and heading for major changes with quite unpredictable effects. And “leave,” as we all now can see, has so many possible variants that no consensus either on which variant would be chosen or on its effects would be conceivable. Therefore, any pronouncement by an economist being inherently irrational and probably contradicted by the estimate of every other economist, the only economically rational choice for a rational voter was to disregard all the economists. But even that level of analysis leaves out the elephant defecating over the whole thing: the voters were being asked to participate in a purely advisory, legally meaningless, exercise whose only possible direct effects (besides settling a backroom knife fight among several gangs of Tories) would be purely psychological. So actually the rational voters were those who saw through the whole fraud and abstained.

By Nauro Campos, Professor of Economics and Finance, Brunel University. Originally published as a chapter in the VoxEU ebook, Brexit Beckons: Thinking ahead by leading economists, available to download free of charge

On 23 June 2016, 52% of British voters decided that being the first country ever to leave the EU was a price worth paying for “taking back control”, despite advice from economists clearly showing that Brexit would make the UK “permanently poorer” (HM Treasury 2016).

The extent of agreement among economists on the costs of Brexit was extraordinary: forecast after forecast supported similar conclusions (which have so far proved accurate in the aftermath of the Brexit vote). Yet the publication of each one of these estimates was followed instantaneously by acerbic criticism which culminated, days before the vote, with the claim that economic experts warning about leaving the EU were like the Nazis who denounced Einstein in the 1930s (Cowburn 2016). Institutions were not immune, with the Treasury, Bank of England, IMF, OECD, and IFS receiving similar treatment. What went wrong? Were economists not ready? Were our forecasts technically poor? Were economic studies fundamentally incomplete and thus flawed? Are we to blame? This column addresses these questions.

How Come Brexit?

In the years to come, there will undoubtedly be many PhD dissertations dissecting Brexit. Economists have been blamed for it, but I don’t think we even make the top three cuplrits. The three main culprits for Brexit, in my opinion, are political elites, economic elites, and the media. These are the three Cs – Cameron, the City, and coverage – with a number linked to each: 11, 17.5, and 41.

Former prime minister David Cameron’s referendum pledge was a reaction to UKIP’s performance in the 2014 European Parliament election. Voter dissatisfaction with the economic policies implemented by the coalition government since 2010 meant severe losses for the two coalition parties. The Conservatives lost seven seats (of 26), while the Liberal Democrats lost 10 (of 11). UKIP gained 11 seats to become the largest UK party.

Economic elites were complacent because they thought common sense would deliver a win for ‘Remain’. Those with more at stake, like the City, did not feel the urge to back up the Remain campaign. Hence, the final fundraising total for ‘Leave’ was bigger than that for Remain by about £3.3 million: £17.5 against £14.2 million (Electoral Commission 2016).  Coincidentally, 17.5m was also the number of pro-Brexit votes.

My third main reason is media coverage. Levy et al. (2016) use a sample of 1,558 articles across nine major UK newspapers to show that 41% were in favour of Brexit, while only 27% were pro-Remain. These are absolute numbers, not weighted by circulation. The authors call the remaining 30% “mixed, undecided or no position”.

It is absurd to blame Brexit on economists, especially in light of the three reasons above. Yet economists may have not been fully prepared. The breadth of our knowledge was inadequate. Some examples: two years ago, we were still struggling with the fragility (i.e. lack of robustness) of our estimates of the benefits of EU membership;1 two years ago, we did not have answers to key questions such as how much EU membership increase FDI flows into the UK;2 and, to this day, we have not yet seen time-series data on how the UK financial sector grew after 1973.3

Such gaps are important because the estimates of the costs of leaving the EU are a function of the estimates of the benefits from EU membership adjusted by the size (and time profile) of the entry/exit shock. The latter can be thought of as a turning point, a structural break (Campos and Coricelli 2015), or also as varying across countries with some more capable than others of absorbing the benefits of integration.

The bottom line is that economists cannot be listed among the main Brexit culprits. Yet gaps in knowledge may well have hindered the quality of our advice. Did this happen? In other words, how good were the ex ante estimates of the costs of Brexit?

Looking Back at the Ex Ante Estimates of the Costs of Brexit

Between the outright victory of the Conservative party in May 2015 and the Brexit vote, there was a stream of medium- and long-term forecasts. We can identify three types of estimates.

  • Type 1 is the one showing gains: Economists for Brexit (2016 ) predict that Brexit will increase UK incomes by about 4% by 2030 (see Dhingra et al 2016b for a thorough assessment).
  • Type 2 are older (pre-May 2015), mostly done by pro-Leave think tanks and often reporting a zero effect.
  • Type 3 includes the vast majority of estimates, which show significant medium- and long-term losses from Brexit. To be specific, the Treasury (2016), CEP/LSE (Dhingra et al 2016a), the OECD (2016), and the NIESR (Ebell and Warren 2016) predict short-term income losses of about 3.8%, 2.6%, 3.3% and 2.3%, respectively, and long-term losses of about 6.2%, 7.5%, 5.1% and 7.8%, respectively.4

Figure 1 Recent estimates of the long-term impact of leaving the EU on UK GDP

Source: Chadha (2016).

These are central estimates. Type 3 studies often presented three scenarios: the EEA/Norway model, the Swiss model, and WTO rules.5 Although losses from the EEA option are the smallest in per capita GDP terms, there is evidence of productivity losses of 6-9% for Norway vis-a-vis EU membership (Campos et al. 2015). Note that the heated UK ‘productivity puzzle’ debate is over a similar productivity loss of 6% to 9% (Yueh 2015).

How can we differentiate between these types of estimates? I argue they differ in at least two fundamental ways: in methodological transparency, and in the quality of the assumptions. On the first count, it is abundantly clear that Type 3 studies are superior to the others. They all provide extensive details that make their estimates entirely replicable. The same can simply not be said of Types 1 and 2.6

The three groups also vary significantly regarding their underlying assumptions. An illustrative example refers to EU regulation costs, which are closely associated with the sovereignty debate. Types 2 and 3 correctly assumes these to be small. Type 1 assumes costs of regulation that are about 6% of GDP, which is large and unsupported by international evidence. Clearly, the larger the costs assigned to EU regulation, the better the Brexit option looks.

There is something insidious about reports arguing that the long-run effect is small or zero. They not only cloud the debate, but present Brexit (and, by extension, UK membership of the EU) as immaterial, irrelevant, or even inconsequential.

Brexit May Mean Brexit, but What Will “Success” Mean?

If one silver lining is needed, the referendum focused our minds and pushed us to generate a lot of knowledge that we didnít have before. Almost without exception, the plethora of studies produced in the wake of Brexit will be useful for understanding, reforming, and hopefully improving the EU. A month on, the Type 3 forecasts are sadly proving accurate. We need to wait until 2030 to assess the long-run estimates, but one thing is clear: being ëoutí turns those still ëiní into natural comparators or counterfactuals. In 2030, there will be less need for sophisticated counterfactual estimation because just analysing the economic performance of Germany, France, the Netherlands, Poland and the other EU members after 23 June 2016, may well be instructive enough.

See original post for references

Print Friendly, PDF & Email


  1. PlutoniumKun

    I would add to fosforus’s comment about uncertainty above that the economic impact on Brexit depends on the nature and make-up of future governments. Release from EU Directives would mean a right wing government has far more freedom to weaken labour protections and (in particular) environmental protections (the latter would have a major impact on the direction of both agriculture and oil and gas extraction). A left wing government, conversely would have more leeway in bringing more sectors under direct control (by avoiding competition rules) and in engaging in a more proactive industrial policy.

    Ultimately, calculating the impact of Brexit is almost impossible even at this stage because we simply don’t know what form it will take, and we will not know for several years.

    1. vlade

      While I fundamentally agree with the high path-dependency of the cost/benefits of Brexit, I do believe that there are few cases where it is easier to say something.

      First case is implementing exactly what Brexit voters wanted – no immigration. This means no single market access (EU has been pretty unequivocal on that, and I doubt they will change their tune). If we assume this case, then pretty much all EU negotiations (in the 2 year horizon) are meaningful only on the technical Brexit, because the trade negotiations (and not just with EU) will take longer. UK based companies should pretty much start either moving out (like banks), or looking at the worst case scenario (WTO option with nothing else). The economic impact of that is going to be massive, and negative.

      Second option is a muddle-through option (EEA style stuff). In this case the impact can be pretty much minimal, at least until 2020, when new elections happen. The question then is who gets elected – would voters punish Tories that that basically Brexit means “More EU and less control”?

      That assumes that any option is actually implemented before 2020 (which depends on when A50 gets invoked)

      1. PlutoniumKun

        I’m not even sure a ‘no immigration’ option would be easy to define. What would that mean exactly? Expelling existing non UK citizens? Visa requirements at every entry point? Or something more moderate like ‘no net immigration’, which would of course be monumentally difficult to organise, even if the EU was not determined to prevent it. Although presumably advocates of a more modest ‘no net immigration’ platform would make dubious but not impossible assumptions about being able to gain allies such as Switzerland or even current EU members in Eastern Europe – or even Denmark (which maintains quite a strict system that only barely works within the EU context).

        I think there is an overwhelming desire and belief across the UK establishment that a ‘muddle through EEA type’ option will be relatively neutral economically and achievable. It is, I think, entirely possible that an EU under stress would decide that facilitating this would be the least worst option. Possible, but fairly unlikely given what seems a real desire to punish the UK for its vote. Much more likely is that ‘muddle through’ becomes ‘5 years of limbo, uncertainty and barely concealed chaos’ with all sorts of unforeseen negative impacts.

        1. vlade

          For me it was setting out the worst and best (economically speaking) short/medium term options. There’s a universe of possibilities between them, although I believe that the “no immigration” universe is severely limited (at least as far as EU immigration is concerned).

          If your ‘5 years of limbo, uncertainty and barely concealed chaos’ option comes to pass, it will be interesting 2020 elections..

      2. a different chris

        >exactly what Brexit voters wanted – no immigration.

        Is that really true? Or maybe they just wanted a chance to catch their breath? Absolutes don’t do us any good here.

        1. vlade

          Ok, “what a majority of Brexit voters wanted”.

          Poll wise immigration was THE deciding factor for Brexit voters. If you have better data (not anecdotes) suggesting otherwise, please supply.

        2. PlutoniumKun

          I think its fair to say that the ‘average Brexit voter on the ground’ was motivated by a general discomfort about immigration, and had learned to be deeply cynical about Conservative claims to be restricting it. But even the term ‘no immigration’ is so loose as to be undefinable given the history of free movement into and out of the UK. Only the most irredeemably racist (and they are, thankfully, a small minority) would I think advocate a complete stop on all permanent movement.

          In all the discussion, btw., we forget that the UK actually has a lot of control over non-EU immigration which it has chosen not to use to any great extent. But so much discontent at ground level was against eastern Europeans, seen (almost certainly correctly) as driving down wages in poorer parts of the UK. Of course, one of the ironies is that the Conservatives were big fans of extending the EU to eastern Europe, as they saw this as a way of weakening the core of the EU (they weren’t wrong there). But then the business lobby ensured that the UK under both Blair and Cameron did not take up the option that other EU countries wisely took of accepting a time limit on incomers. Only the UK and Ireland allowed open immigration from new members from the start. That was great for business, but we can see it was politically catastrophic in many other ways.

          1. vlade

            Hmm, seems my comment disappeared (Yves/Lambert, apologies if it’s in moderation), so a short version

            Polish Plumber is now pretty much a myth. Majority of EU immigrants from 2011 onwards were from Spain/Portugal/France/Italy and Greece. Lot of them ending in low-skill jobs – baristas, shop assistants, waiters, kitchen help/low level chefs.

            But Polish Plumber is an easier target than Spanish Barista or French Chef. Even if they make a fraction of what a real Polish plumber can make in the UK, and still undercut – in my experience – unreliable and expensive English one.

            1. PlutoniumKun

              I’m not sure that ‘in real terms’ the Polish plumber is a myth. While for those living in bigger, prosperous cities, the ‘face’ of EU immigration is that nice Spanish girl in your local Costa, in smaller towns and backwater cities the ‘face’ for working people are the East Europeans working in factories and construction sites. On a purely anecdotal basis, when I was a student looking for summer work as a labourer in London in the late 1980’s the going rate was around £8-10 an hour. I hear its common now to be paid £5 an hour (interestingly, half the figure a typical office cleaner will get). I don’t know any Irish people without a higher qualification (engineering or specific skill) working in an English building site now, its just not worth the money – you can get more working in a bar or coffee shop.

              I’m also somewhat suspicious of official figures. I can’t give examples from the UK, but here in Ireland, there is a general rule that you can double the number of East Europeans compared to official (census and tax) information. A typical Polish immigrant is often working on a building site for a relative or friend who has a small business, maybe painting or carpentry. He has no need to bother with any form of registration as he’s not paying tax and does not intend to apply for social welfare. As a matter of course, they don’t fill out census forms (having once done census work I can confirm that personally). I think this greatly distorted figures for those working in ‘official’ jobs (such as for big retail chains) from those in the agriculture and building industry, where so many ‘shadow’ workers can be found.

              This is a long winded way of saying that while the actual figures give a picture of diversity, the actual lived reality of many working and lower middle class people around the UK of immigration is very different. Its not the nice Italian kid in your shop, its the gruff Polish guys who are doing the work you used to do for half the price.

              1. vlade

                Well, CEE definitely dominates the seasonal works in agri/fishing etc. But this was the case since at least 1990s – pretty much always done w/o any worries about tax or whatever. Entry into EU just legalised it.

                My experience with CEE outside of larger cities is that unless it’s a seasonal job, they want the same salaries as locals (especially the Polish community is very good at sharing the info on what are the going rates I’m told), so not (much) undercutting. The “will work for half” is definitely a myth. The exception being black (no tax, no NI, no nothing) short term works. There’s plenty of that going on, but then the employer doesn’t often care whether its (technically) legal EU migrant, or a Somali illegal one (estimate of UK illegals is at least one mil, so about a third of all EU migrants). And, in my opinion, illegal work is different beast from legal immigration (although very much tied to illegal one)

                I had talked to a few builders (admittedly in London/East Anglia, so not comparable to North), and one reason they preferred CEE even at the same pay that English ones was that CEE workers were (in general) more open to work long/weird hours (paid, according to the builders) and be more skilled than a comparable English worker (a comment from one builder was “their English is better than the local lads I can get, they can do percents w/o me explaining it to them – and they drink only on Fridays” – no data, just anecdote. He could be biased for reasons unknown to me. He was English FWIW). That said, the official data shows that the immigrants are substantially higher educated than UK workforce. Say, in 2014 45% (!) of all UK male workforce quit education by the time they were 16, but it was true of only 17% of (all) immigrants.

                The “problem” with job-driven immigrants is that you get self-selection, as people who are willing to take the risks of going for a job in a foreign country tend to be much more driven, ambitious etc. than you average local (and that’s true anywhere, not just in the UK). So even for the same job, with the same pay, employer may end up preferring immigrants (because ultimately he’s paying same for more skills).

                Few years back I was hiring some developers. All the people we ended up offering the positions to were immigrants (non EU)- because when I gave the Oxbridge home-grown PhDs a very simple problem (but outside of their immediate range of experiences), they just were lost. The various out-of-UK (not necessarily EU) candidates were order of magnitude more driven and clued up on problem solving (which is what I was looking for). The sample was about 50-50 UK/immigrants, so really a recurring problem.

                So, if you want my brutal, honest to god opinion, UK must first fix its lower/medium (and even higher in some cases) education system. Because right now it’s shambles, unfit for any purpose. Because w/o that stopping immigration or whatever else will not really make any difference. And yes, I hear that the immigration puts a strain on the system – but FFS, if increase of about 2% of the population in 5 years breaks the system, the system was broken before, and this just showed it.

  2. a different chris

    The problem with economists looking at short run effects is that they are really at the Heisenberg Uncertainty level here. If I’m thinking about my cash, I’m listening to at least financial people that are listening to economists. So the economists get nervous, that trickles down to me, I react by doing something that they were nervous I was going to do, now they are no longer just vaguely nervous but have my specific behavior to point to (but not admit that they were the root cause) and the whole thing starts ratcheting up.

  3. DanDaniel

    Economists could not forecast the exchange rate next year, or the oil price in 7 months, or the stock price in 43 days! Long-term? hahaha

  4. Patrick

    Calculating “GDP per household” is deceptive. Gains from EU were not distributed evenly. Losses will not be either. And even if we assume that the losses were distributed evenly, the median loss will be far less than the mean. Median household income in the UK is ~23,000 pounds. If GDP is permanently reduced by 2%, median income will be reduced by no more than ~460 pounds per year, not ~1400.

    I think Brexit voters knew this, intuitively. Most economists speak the language of comfort to the rich.

  5. marku52

    If GDP never trickles down to me, I don’t really care much what happens to it, now do I?

    1. paul

      You get, in exchange for foregone GDP share, membership of the highly valued ‘hard working’ sector of the economy. Wear this classification with pride.
      That is until your job is taken away, then, as jj hunsecker told sidney falco, “you’re dead son, go get yourself buried”.

  6. Roland

    Let’s suppose that Brexit were guaranteed to permanently reduce UK living standards.

    A rational voter might still choose, “Leave!”

    It depends on what sorts of things one wants to pay for. Everybody knows that independence can be expensive. Sometimes, it’s too expensive.

    But compared to the price that some peoples in the world have paid for clear-cut sovereingty, even a 10-15% drop in real per capita GDP wouldn’t be all that much.

    People are not under some sort of sacred, inalterable, obligation to eternally maximize the production of goods and services.

    It’s the distribution that matters most in any case. That’s why the UK Left has screwed up its Brexit stance. The buggers should have been jumping all over the idea of a Parliament that can really legislate the hell out of everything.e

Comments are closed.