Guest Post: Is the Problem the Models or the Modelers?

Submitted by Richard Alford, a former economist at the New York Fed. Since them, he has worked in the financial industry as a trading floor economist and strategist on both the sell side and the buy side.

The fault, dear Brutus, is not in our model, but in ourselves–apologies to W Shakespeare

The economics profession is in disarray. Internecine warfare has broken out as proponents of various models/schools of thought are attacking and being attacked by each other. The battle between the different camps has been increasingly fought in the open e.g. The Economist and blogosphere. Despite the credentials of the combatants, there is a definite “My model can beat up your model’ air to the contretemps. The best outcome would be for policymakers to avail themselves of the models while losing the modelers.

Given the complexity of the financial markets and the economy, macro policymakers must operate on the basis of either implicit or explicit model(s). Furthermore, an explicit and transparent model subject to outside examination continuous testing is preferable to implicit poorly specified models that can serve as quasi-intellectual cover for policies based on solely belief systems.

However, macroeconomic models have been oversold again, the great moderation was fleeting. While economic models are necessary, both the recent experience and well-known limitations of the models themselves indicate that models alone will not be sufficiently robust to perform as well as many including investors and the general public have been led to expect. The underlying markets and the real economy are not stable. The models by definitions are incomplete. The responses of economic agents in times of stress and crisis may differ dramatically from the behavior reflected in parameters estimated on the basis of data collected in normal times. Goodhart’s law suggests that policy changes will induce changes in the statistical relationships on which the policy is based.

In defense of the current class of models (DSGE), some economists have spoken out saying that the models evolved as they did in response to questions raised by economists. But isn’t that the problem? Macro-economic models have become more formal and mathematically elegant, but all the while the modelers both ignored important drivers of economic performance, e.g. the workings of the financial markets, and failed to communicate the limitations of the models. The models evolved reflecting the economists‘desire for elegance and tractability, but with insufficient weight given to concerns such as financial stability or external balance.

Macro-economic policy is now being executed by the same people who devised the models. At times it has appeared that some of them sought policy making positions in order to be able to demonstrate the value of the model that they created or to which they made a contribution. These model builders were quick to take credit for the great moderation, but slow to see/ blind to the risk building up in the financial system and the imbalances. This type of behavior is to be expected. There is no reason to assume that the next set of policymaker/model builders will react any more positively to “warnings” that risks not central to the model pose real threats.

The role of model builders and policymakers should be split. Given the necessity of models in the policy formation process, it would be best, if the policymakers’ judgments about which model to use or when diverge from it are not clouded by previous involvement in the development of one or another of the models that might be employed.

Print Friendly, PDF & Email


  1. Anonymous

    I was a Russian student in the 1970s. I recall our teacher, himself a recent Russian emigrant, telling us that in the USSR every factory, school, ship, and military brigade had what was called a 'political officer'. This person was trained in the principles of Marxism-Leninism and a true believer therein. Whenever there was a problem, ranging from the broadest policy decision to the most personal matters, the political officer gave advice based on his interpretation of communism; he was a kind of priest.

    It seems that our modern economists occupy somewhat the same role. They are mostly government or academic employees, and as such must advocate policies that line up with their employers' interest or with the academic mainstream.

    Like the Soviet political officers, economists have a model of how the world works, a model which takes too little account of observed reality. Imbalances and contradictions are explained away until they crawl out of the swamp and terrorize the townfolk.

    Governments don't really want economic insight as much as they want intellectual cover for what they intended to do anyway. (This was a big selling point for Keynes in the 1930s.) In that respect, and perhaps no other, mainstream economics has delivered the goods.

  2. Anonymous

    Yes, I think the first comment is correct. Neo-classical/neo-liberal economics is nothing more than ideology, and for the past twenty year economists have been the ideological police defining what is legitimate discussion and what is not.

    However when we examine the policies of the past twenty years we find (with a few notable exceptions) economic policies have overwhelmingly benefited the very wealthiest section of society.

    The whole discipline is hopelessly compromised. The answer will not be found in new models or in new theories, but in a new political settlement. We are standing on the brink of new reformation – Friedman, Mises, Keynes, Samuelson even Krugman and Stiglizt will go the same way as Filmer’s “Patriarcha”. They are the artifacts of a dying age.

    Now is not the time for theory, now is the time for action.

    1. Our first objective should be to prevent the bonuses from being paid. The office of JPM and GS should be occupied and the bankers expelled
    2. All bonus paid over the period 2000-2009 should be repaid – a special economic reconstruction tax should be levied on bankers to this end
    3. A wealth tax should be levied on the assets of the richest 10% of society
    4. The banks should be socialized. Banks should be run like public utilities

    This land is our land, and we will take it back!

  3. David Habakkuk

    One of the key things which the models seem to leave out — as does economic theorising in general — is herd behaviour. Some remarks from an interesting short paper by the University of Warwick economist Andrew Oswald:

    "Why do herds form? They happen when relative position matters. People paid extraordinarily high prices for houses, even though not justified by fundamentals, because they felt they were trailing behind the Joneses. Brokers sold unsound mortgages because they had to keep up with rival brokers. Money managers — remunerated on their relative performance against other managers — traded shares with the same motive.

    "Yet conventional economics contains no recognition of such action. The word 'herd' does not appear in leading textbooks. In consequence, those texts do not offer an intellectual framework that could have predicted, or can help policy-makers in, our current dilemma. The research journals are little different. Since 1970, on an electronic count, only 3 out of the last 8000 articles in the Economic Journal discuss herd behaviour; 2 out of 2000 articles in the Quarterly Journal of Economics; and 4 out of 1500 articles in the Journal of Financial Economics. It would be difficult to prove that this is why the world is in a mess. But common sense suggests that the lacuna is a powerful contributing factor. Just as before the Great Depression, economists and central bankers have been using the wrong model of human behaviour.

    "It will be necessary to rewrite standard economics. We must bring the idea of relative comparisons and herd behaviour into the centre of our thinking. A good place to start is William Hamilton’s article on defensive herding in the 1971 Journal of Theoretical Biology, and work by Andrew Clark and others in, for example, the 1998 Journal of Public Economics.

    "In a world with herd behaviour, there exists a natural intellectual case for government intervention to internalize the externalities created. The coolheaded individuals of our unrealistic traditional textbooks do not need to be regulated. Herds do."


  4. jimcaserta

    One thing that seems to be inherently overlooked in economic modeling is time-lag from event to response. The basic principle of economics is that you take some action and there is a consequence, good or bad. If you are counting on negative feedback to regulate bad behavior, system stability analysis should be addressed.

    With any feedback loop, the phase shift – time lag, can cause overshooting the target steady-state level, or can become unstable.

    During step-function type events, you get either too slow a response, or too fast a response with overshoot and ringing. No feedback system instantaneously reaches steady-state. I have not seen any economic models that address that source of error.

    Most models are also piled on with assumptions, and lack: correction for additional information, and basis on smaller units. Why build a model for a MBS, when you can build a model for an individual mortgage, and then build a much better model of an MBS as an aggregation of the individual mortgages? The idea that that approach is too complex is complete BS, as that is how real engineering models are created.

    If you don't take the time to understand your models, you might as well just make up your results as you will have no insight into whether something is slightly off in the model pushing you to some boundary condition. The problem is with the modelers, and those using the models.

  5. Anonymous

    "The role of model builders and policymakers should be split"

    that's the only idea Richard Alford can contibute for the future?

    that is all? the models don't work but the rest is ok?

    i think that is not much…but it goes somehow in line with the development of …

    YVES this page has been full of fantastic ideas … (sad to say)… some time ago. maybe it becomes better again when your book is in print.

  6. Anonymous Jones

    Even the near future is uncertain. Any system with billions of variables and obvious feedback loops is chaotic to the point that no model can be dependable. The economy is far worse than weather in this regard because many of the variables are self-aware and are trying to get ahead of the next turn of events. Only huge amounts of data over long periods of time can give us broad ranges of cause-effect links with which we might try to make things as best we can, throwing out the ideas that clearly make things worse. The idea that we will ever be able to fine tune is hubris, truly idiotic and delusional hubris.

  7. Eric L. Prentis

    Economists model the financial system as efficient using the following assumptions: 1) all investors make financial decisions using rational expectations; 2) market prices incorporate all available information, i.e., all trades are zero net present value transactions; 3) markets are always in balance, and when random information moves prices, markets are self equilibrating; and as a result of the prior three assumptions 4) markets cannot be beaten (assumption #4 explains the reverence that markets are held in academe and by extension, Wall Street and politicians).

    As important, however, is what is not mentioned above. For example, markets are necessary but they have to be fairly regulated, i.e., self dealing, front-running, corruption and racketeering by the political and financial ruling class can subvert markets for their own benefit, to the detriment of the economy.

    Models map territories and are never as complete, consequently, current research questions the above efficient market assumptions, i.e., 1) behavioral finance looks at irrational and herd behavior in markets: 2) the speed with which information affects prices is extensively investigated; 3) it is assumed that random information moves prices randomly and markets have always come back so markets are self equilibrating which is the worst assumption and is rarely tested in the literature; and 4) research tests risk and return of individual stocks or a specific portfolio to a benchmark such as the S&P 500 Index to conclude the stock market cannot be beaten on a risk adjusted basis.

    My empirical research into issues numbered 3 and 4 above shows inconvertibly that rather than stock prices moving randomly, the stock market is a discounting mechanism where professional traders look ahead, anticipate and make allowance of expected corporate earnings and economic data before they are announced in the media. By stripping out the random noise in stock prices, serial correlations are plainly evident in the data which means that statistics is not the best method for investigating the stock market.

    In my research using calculus and an algorithm, on market data from 1928 to 2008, S&P trading-methodology portfolio B, due to price data serial correlations, is +146 percent economically superior and -34 percent less risky than S&P buy-and-hold portfolio A. Consequently, markets can be beaten if you know what you are doing and therefore, assumption #4 is clearly proven invalid.

    I’ve sent my empirical research to ten financial academic journals and each time the response is “no,” with no explanation as to why. I conclude that along with government regulatory capture by Wall Street, financial academe has been captured by their banking corporate sponsors.

  8. dragoneyes

    As someone who majored in geology in colege before getting a degree in math, I am appalled at the politicization of climate issues from both major parties. Since so much of the climate change hype is based on the results of models, and we all know how well important judgements based on models have worked for the financial system.

    Here is an excellent article on the current global cooling sub-cycle we're in, by someone with excellent credentials.

    Global Warming or Global Cooling? A New Trend in Climate Alarmism

  9. Anonymous

    I had an engineering professor who described this real life problem: a crane with a supposed safety factor of 3 kept falling down. The third time, someone died. The company asked for his analysis, but when he ran the numbers, he got the same result.

    Finally, he took the output of where the crane was bending under its load and fed the new positions back in. The equations assumed that it didn't matter, but he did it anyway. Turns out it did matter — a lot. The "safety factor 3" crane was really operating within 3% of its buckling load, and the standard linear equations weren't valid.

    Models are useful, but it you don't know when they break down, they will literally kill someone.

  10. Hugh

    Models and modelers are discredited for the same reason that the wider field of economics is. They not only missed predicting the biggest economic events of our times but helped cause them. What is the point of having a model or an economic theory if it can't see an $8 trillion housing bubble, if it doesn't see the relationship between derivatives, risk, and moral hazard, if it can't distinguish between the real economy and the paper one, if it blindly promotes globalization and fails to see the dangers of de-industrialization, if it sees wage gains as inflationary but sees no problem with surplus investment going into bubbles (until it is too late), that ignores capitalism's history of boom and bust and posits theories based on equilibria. Modern economics failed and in rather spectacular fashion. Rather than sticking with those who got it wrong and keeping them in their positions, economics to regain its credibility needs to turn them out and turn instead to those who got it right. I don't see this happening. The legitimacy of elites is based on their knowing more and better. They can't admit error, especially one as big as this, without losing authority so they will simply not admit it. Wrongheaded econmists will remain in their positions in government, academia, and business until time removes them, and they will continue to train future economists in their errors.

  11. alex

    Try Steve Keen – an Australian economist who's spent his career ripping a new one in the neoclassical models.

    And yes, he did predict the current crisis. He's frighteningly pessimistic about recovery though, so I hope he's not too accurate.

  12. joebhed

    On splitting the roles.
    Hey, guess what?
    How about this?
    Do what Friedman said in his epic piece on "A Fiscal and Monetary Framework for Economic Stability".

    What we need to separate is the money-creation function from the banking function.
    It is so simple that it is through our own myopia that we fail to see its wisdom.
    The creation of the nation's money is properly a governmental role, according to not only Friedman, but ALL of the architects of the Chicago Plan for Monetary Reform back in the '30s.

    If you want economic stability, DO NOT LET THE BANKERS CREATE THE NATON'S MONEY.
    Sorry for shouting, but, yes, it is that simple.
    Monetary reform.
    A public mmoney system.
    A private banking system.

Comments are closed.