Most Long-Term Growth Forecasts by Economics Researchers Biased Upwards

Lambert here: Now they tell us.

By Masayuki Morikawa, Vice President, Research Institute of Economy, Trade and Industry (RIETI). Originally published at VoxEU.

Although long-term macroeconomic forecasts substantially affect the sustainability of government debt and the social security system, they cannot avoid significant uncertainty. This column assesses whether academic researchers in economics make accurate long-term growth forecasts, comparing ten-year growth forecasts made by Japanese economists in 2006–2007 with the realised figures. Even excluding the years affected by the Global Crisis, the results show that forecasts tend to be biased upwards and involve significant uncertainty, even for economics researchers specialising in macroeconomics or economic growth.

The accuracy of long-term macroeconomic forecasts substantially affects the sustainability of government debt and social security systems. However, long-term forecasts cannot avoid significant uncertainty because the future developments of new innovations and their diffusion are difficult to assess quantitatively. In addition, many unforeseeable events such as financial crises, international conflicts, and major natural disasters occur.

Upward Bias in Growth Forecasts by Governments

Many studies have indicated that economic growth forecasts by government agencies tend to be biased upward. In the case of Japan since the fiscal year 2000, the government’s GDP growth forecast for the next fiscal year has, on average, had an upward bias of 0.5 percentage points and 1.4 percentage points for real and nominal GDP growth rates, respectively. The upward bias in economic growth forecasts often causes optimistic bias in the projections of fiscal balance and debt level. However, these studies have mostly focused on short-term forecasts of one or two years.

Frankel (2011) and Frankel and Schreger (2013), which analyse three-year forecasts of government agencies, are two of the very few studies evaluating medium-term forecasts. These studies indicate that the optimistic bias tends to increase with the time horizon of the forecasts. However, studies on the accuracy of long-term forecasts, such as those involving a ten-year time horizon, have been scarce. In addition, as far as the author is aware, there has never been a study evaluating macroeconomic forecasts of academic researchers in economics.

Against this background, in recent work (Morikawa 2019) I empirically assess whether academic researchers in economics make accurate long-term growth forecasts. Specifically, I use unique survey data on macroeconomic forecasts on real/nominal GDP growth made in the mid-2000s to evaluate the ex-post performance of the 10-year forecasts.1

A Survey for Economics Researchers

The survey data are taken from the Survey of Long-Term Outlooks of the Japanese Economy, designed by the author and conducted by the Ministry of Economy, Trade and Industry in 2006 and 2007. The survey questionnaire was sent to all economics researchers belonging to the Japanese Economic Association (JEA) and to about 100 professional forecasters in private economic institutes who are not members of the association. The Japanese Economic Association is the counterpart of the American Economic Association in the US and the European Economic Association in Europe. A total of 608 responses were obtained; 94% of the respondents were association members and 77% were professors/researchers belonging to universities.

The survey asked respondents to report their long-term forecasts of real/nominal GDP growth rates for the next ten years. Specifically, the survey asked for the expected annualised growth rates (up to the first decimal point). Since realised figures of the variables are now available, ex-post forecast errors or the accuracy of the ten-year forecasts can be evaluated.

It should be noted that the survey asked for forecasts based on an assumption of no significant exogenous shocks. Given the effect of the 2008–2009 Global Crisis on the Japanese economy, we calculate the mean realised growth rates by removing figures for the fiscal years 2008 and 2009 and use these rates to calculate the forecast errors by subtracting realised rates from the forecasted rates. Hence, positive figures mean upward bias in the forecasts, and negative figures denote downward bias.

Ex-post Evaluation of the Accuracy of Forecasts

Figure 1 shows the means and medians of forecast errors. Forecasts by the researchers in economics have an optimistic bias, similar to the findings for the forecasts by the government agencies. Even after removing the two years affected by the Global Crisis to calculate annual growth rates, a non-negligible upward bias remains: about 0.5-0.6 percentage points for real GDP growth and about 1.3-1.4 percentage points for nominal GDP growth.

Figure 1 Means and medians of forecast errors

However, the bias in economics researchers’ growth forecasts is smaller than those of the government. According to the New Economic Growth Strategy drawn up by the Japanese government in 2006, the real GDP growth forecast was 2.2% per year for fiscal years 2005–2015.2 The realised real GDP growth rate for this period, after removing the two years affected by the Global Crisis, was 1.3%, meaning that the government forecast had an upward bias of 0.9 percentage points.

Why are economic researchers’ growth forecasts, which are unlikely to be affected by political pressure, upward biased? Although it is difficult to determine from the data, a possible interpretation is that the respondents tend to extrapolate past growth performance at the time of making forecasts. Another possibility is that the actual economic policies conducted by the government were inappropriate relative to the respondents’ expectations.

Size of the Bias by Individual Characteristics

The survey collected information about the respondents’ gender, age, affiliation, and research field (19 JEL classifications).3 Figure 2 shows the regression coefficients for the individual characteristics to explain their forecasting errors.

Figure 2 Forecast errors: Japanese Economic Association members versus macroeconomics researchers

Notes: The figures indicate estimated coefficients for respondents’ characteristics. Researchers in macroeconomics is a dummy variable that represents those who chose the JEL classification ‘Macroeconomics and Monetary Economics’ and/or ‘Economic Development, Technological Change, and Growth’ as their research field.

The coefficients for the Japanese Economic Association member dummy are negative and significant at the 1% level in real and nominal GDP growth forecast errors. Although the size of the coefficients is not large, the upward bias is about 0.2–0.3 percentage points smaller for forecasts by academic researchers than by professional forecasters from private institutes who are not members of the Japanese Economic Association.

In contrast, the coefficients for the dummy for researchers specialising in macroeconomics are positive and significant in real and nominal GDP growth forecast errors. The long-term economic forecasts by researchers specialising in macroeconomics and/or economic growth are somewhat less accurate, although the differences are quantitatively small (about 0.2 percentage points).

Why are their forecasts relatively inaccurate? One possible reason is that they were frequently in contact with information about the statistics of high growth rates and the government’s economic outlook at the time of forecasting.

These results show that long-term growth forecasts involve significant uncertainty, even for economics researchers specialising in macroeconomics or economic growth.

References available at the original.

Print Friendly, PDF & Email
This entry was posted in Economic fundamentals, The dismal science on by .

About Lambert Strether

Readers, I have had a correspondent characterize my views as realistic cynical. Let me briefly explain them. I believe in universal programs that provide concrete material benefits, especially to the working class. Medicare for All is the prime example, but tuition-free college and a Post Office Bank also fall under this heading. So do a Jobs Guarantee and a Debt Jubilee. Clearly, neither liberal Democrats nor conservative Republicans can deliver on such programs, because the two are different flavors of neoliberalism (“Because markets”). I don’t much care about the “ism” that delivers the benefits, although whichever one does have to put common humanity first, as opposed to markets. Could be a second FDR saving capitalism, democratic socialism leashing and collaring it, or communism razing it. I don’t much care, as long as the benefits are delivered. To me, the key issue — and this is why Medicare for All is always first with me — is the tens of thousands of excess “deaths from despair,” as described by the Case-Deaton study, and other recent studies. That enormous body count makes Medicare for All, at the very least, a moral and strategic imperative. And that level of suffering and organic damage makes the concerns of identity politics — even the worthy fight to help the refugees Bush, Obama, and Clinton’s wars created — bright shiny objects by comparison. Hence my frustration with the news flow — currently in my view the swirling intersection of two, separate Shock Doctrine campaigns, one by the Administration, and the other by out-of-power liberals and their allies in the State and in the press — a news flow that constantly forces me to focus on matters that I regard as of secondary importance to the excess deaths. What kind of political economy is it that halts or even reverses the increases in life expectancy that civilized societies have achieved? I am also very hopeful that the continuing destruction of both party establishments will open the space for voices supporting programs similar to those I have listed; let’s call such voices “the left.” Volatility creates opportunity, especially if the Democrat establishment, which puts markets first and opposes all such programs, isn’t allowed to get back into the saddle. Eyes on the prize! I love the tactical level, and secretly love even the horse race, since I’ve been blogging about it daily for fourteen years, but everything I write has this perspective at the back of it.

6 comments

  1. Another Amateur Economist

    Well, for one thing, mainstream economists mostly assume that ‘the system,’ capitalism, the global economy, or whatever, is just going to go and grow forever. So much so that they haven’t even asked the question: “What might the collapse of a capitalist economy look like?” So they could be in the middle of a collapse, and just wouldn’t know it, because they would be interpreting all economic phenomena through the rosy lens of unending growth. And what was that phrase? “Creative destruction.” Yeah.

    So that’s a serious unconsciously positive spin economists are going to put on everything economic. Probably something around positive 3% is likely to be the true baseline in their thinking, although that may have gone down lately.

    Anyway, they still haven’t produced a working model of the Great Depression, or even the Great Recession, and they lived through that.

    Reply
  2. Sound of the Suburbs

    The BoE forecasts that it will able to raise interest rates next year.
    https://blogs.spectator.co.uk/2017/08/mark-carney-should-stop-blaming-brexit-for-rock-bottom-interest-rates/
    When next year arrives, they can’t.
    It has never occurred to them to think about what might be wrong with their models.

    The economics of globalisation has always had an Achilles’ heel.
    In the US, the 1920s roared with debt based consumption and speculation until it all tipped over into the debt deflation of the Great Depression. No one realised the problems that were building up in the economy as they used an economics that doesn’t look at private debt, neoclassical economics.
    Not considering private debt is the Achilles’ heel of neoclassical economics.

    Can you guess what the problem with the BoE models is?
    It’s not hard (hint – look at first two paras. and give it 5 seconds thought).
    https://www.housepricecrash.co.uk/forum/uploads/monthly_2018_02/Screen-Shot-2017-04-21-at-13_53_09.png.e32e8fee4ffd68b566ed5235dc1266c2.png
    Look at all that debt in the economy, that’s what the problem is.
    Debt repayments to banks destroy money and this is dragging the economy down, it’s called a balance sheet recession.

    Reply
    1. Sound of the Suburbs

      The FED does the best it can with an economics that doesn’t consider debt, which is not very good at all.
      Greenspan and Bernanke can’t see the problems building before 2008.
      https://www.youtube.com/watch?v=vAStZJCKmbU&list=PLmtuEaMvhDZZQLxg24CAiFgZYldtoCR-R&index=6
      At 18 mins.
      No one can work out what caused 2008, and afterwards and they attribute it to a “black swan”.

      Janet Yellen is not going to be looking at that debt overhang after 2008 and so she can’t work out why inflation isn’t coming back.
      It’s called a balance sheet recession Janet, you know, like Japan since the 1990s.

      Jerome Powell is not looking at the debt overhang after 2008 and so thinks the US economy is fixed and raises interest rates. Raising interest rates with all that debt in the economy will soon cause a downturn and there is no way he will get anywhere near normalising rates.

      Reply
  3. Synoia

    Why are economic researchers’ growth forecasts, which are unlikely to be affected by political pressure, upward biased?

    Because they are affected by by political pressure? It appears much of economists continued employment or funding is based on telling people what they want to hear.

    Reply
    1. Jeremy Grimm

      This post starts off with another troubling statement: “long-term macroeconomic forecasts substantially affects the sustainability of government debt and social security systems”. I am also a little uncomfortable with ‘nominal’ opposed to ‘real’ growth. I suppose some corrections for inflation were made in the ‘nominal’ to obtain the ‘real’?

      How does Japan calculate GDP numbers? I think that is a most glaring omission in this post. If they do so in a way similar to way the U.S. calculates GDP an upward bias is built-in. The U.S. treats the bank penalties and interest rate hikes consumers pay when they have trouble paying debts as an income item in the GDP accounting. The U.S. counts the imputed rents a homeowners enjoy as a consequence of owning a home as an income item in the GDP.

      Growth in the U.S. GDP primarily goes to the extremely wealthy, with a small slice for the merely wealthy, and a few crumbs for the upper levels of the rest of the population. Whether the GDP grows or not hardly matters to most of us and the ‘real’ inflation opposed to the ‘nominal’ inflation is a very bad joke. I am not sure why it matters that fudgy numbers GDP numbers fudge high. If high fudges help suggest the “sustainability of government debt and social security systems” — I suppose that is a good thing.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *