How Imperfect Knowledge Shapes Financial Markets

Yves here. The post starts out on a sour note by treating investment as coming from savings. That is the loanable funds fallacy! in fact, bank lending creates deposits. But the discussion of how capital markets allocate investments to various opportunities doesn’t hinge on that notion.

By Nevin Cavusoglua, Associate Professor, Economics and International Business, James Madison University, Michael Goldberg, Todd H. Crockett Professor of Economics , University of New Hampshire and Josh Stillwagon, Assistant Professor, Babson College. Originally published at the Institute for New Economic Thinking website

Asset markets are indispensable in harnessing society’s diverse views and insights about future business performance. But those views are shaped as much by emotion and crowd mentality as by rational expectations.

Every society must decide how to allocate its scarce savings among the many companies (and households) seeking to finance investment projects. Ideally, the limited savings should be channeled to companies with the highest future returns and lowest risk. Companies with poor prospects should not receive financing at all. The problem, of course, is that there are a great many views about companies’ prospects and no objective way to determine that one view is necessarily better than any other. Financial markets address this problem by placing weight on the views of all its participants in setting prices. Prices fluctuate as news about prospects becomes available. Better prospects lead to price increases and greater access to financial capital. Deteriorating prospects lead to the converse. Once we recognize that no one possesses the truth about the future, we think it is difficult to imagine a better means of allocating society’s savings across companies.

This view of financial markets stands at odds with both the efficient markets hypothesis (EMH) and behavioral finance. EMH rests on the rational expectations hypothesis (REH), which supposes that rational market participants know the true probability distribution governing outcomes and use it to interpret news about companies’ prospects. If true, financial markets would be nearly perfect in setting prices and allocating capital. Behavioral finance also uses REH to characterize rational forecasting. However, behavioral models suppose that some or all market participants lack access to the true probability distribution. In these models, price fluctuations are driven by emotions, crowd psychology, and momentum trading, rather than news about company prospects. According to this view, financial markets often set prices and allocate capital poorly.

The efficient markets view’s embrace of rational expectations is striking. It assumes away the very reason society needs financial markets: to harness the great diversity of views in society about companies’ prospects. Behavioral finance’s reliance on REH is also odd given its emphasis on psychological realism.

Our paper (Cavusoglu, Goldberg, and Stillwagon 2019) on currency returns is part of INET’s imperfect knowledge economics (IKE) research program. This program’s premise is that no one can know the true probability distribution governing future market outcomes. The reason is simple: historical developments that are to some extent novel (e.g., shifts in fiscal and monetary policy, changes in institutions, and technological progress) trigger structural change in the process underpinning market outcomes. The novelty of these developments implies that they are to some extent non-repetitive and that their impact on the data generating process “deal[s] with situations which are far too unique…[to rely solely on] statistical tabulations” (Knight 1921. p. 198). Consequently, no amount of past data and tabulation can deliver the true probability distribution governing future market outcomes. How could one have anticipated, years in advance, the probability and implications of, for example, Brexit, the US government shutdown over a border wall, or Draghi’s “Whatever it takes” speech?

Ignoring the primacy of imperfect knowledge has led to many puzzles in financial markets. A longstanding anomaly is the inability of conventional risk premium models to account for excess returns in asset markets (Fama 2013, and Cochrane 2011). Conventional models rely on expected utility theory. They relate the market’s expected return to individuals’ degree of risk aversion and fundamentals such as asset supplies or consumption growth. The market’s expected return is not observable. Researchers have thus used data on actual, ex post returns. To draw inferences about expected returns, they invoke the rational expectations hypothesis, which implies that expected and ex post returns differ by a white noise – purely random – forecast error. As such, existing empirical studies of risk premium models provide tests of the joint hypothesis of predictions concerning expected excess returns and REH’s prediction of white noise forecast errors. The rejection of this joint hypothesis in study after study has led to the view that returns are not driven by risk or fundamentals.

In our paper on currency returns, we provide evidence that the anomaly stems from a failure of both expected utility theory and REH. Our investigation considers two competing portfolio balance models, one based on Dornbusch’s (1983) international CAPM (ICAPM) and the other developed by Frydman and Goldberg (2007, 2013). The latter model follows Dornbusch (1983) in its basic setup, but replaces expected utility theory (EUT) with Kahneman and Tversky’s (1979) prospect theory (PT). Researchers have found that alternatives to EUT improve the consumption CAPM’s empirical performance. (See Barberis, Huang, and Santos 2001, Backus et al. 2010, and Bansal and Shaliastovich 2013). Such alternatives may also improve the empirical performance of portfolio balance models.

We also sidestep the joint hypothesis problem by using survey data on exchange rate expectations, which provide a proxy for the market’s expected return. These data enable us to test directly the risk premium models’ predictions concerning expected returns, without conflating them with REH.

The Expected Utility approach- and Prospect Theory-based models imply different risk factors. Both models relate the risk premium on foreign currency to the country’s bilateral international debt position (IDP). With EUT, the risk premium also depends on the conditional volatility of returns. But, with prospect theory, the premium depends positively on the gap between the exchange rate and market participants’ assessments of its benchmark value. The “gap effect” is intuitive: the more over- or undervalued a currency becomes, the more risky it is for market participants who speculate on a further over- or undervaluation. The two models’ predictions for sign reversals also differ. Both models relate the risk premium’s sign to the sign of the country’s international debt position. But with the PT model, the sign of the risk premium also depends on the risk assessments of the bulls (who take long positions in foreign exchange) relative to those of the bears (who take short positions). This additional factor gives the model greater potential to explain sign reversals.

A key issue for the empirical analysis is whether excess returns are stationary. Most studies make the assumption that they are. But, there is considerable evidence that excess returns are highly persistent and possibly nonstationary—that is they change over time in unpredictable ways, as opposed to returning to some average value. (See Johansen et al. 2010, Juselius 2014, and Juselius and Assenmacher 2017). Frydman and Goldberg (2007) address this problem with single-equation error correction models. In this paper, we rely on the I(1) cointegrated vector autoregression (CVAR) framework (Johansen 1996, Juselius 2006), which is better suited for handling persistent variables. We find that expected excess returns and other variables in the information set are best characterized as random walks, implying that they are nonstationary.

We also extend Frydman and Goldberg’s (2007) analysis by including measures of exchange rate volatility and bilateral IDP in our information set. This enables us to consider all of the PT model’s predictions and those of the international CAPM. We also consider a hybrid model that includes both the gap and ICAPM risk factors. This is the first paper to compare or combine these two specifications of risk.

To preview our results, we find little support for the ICAPM. By contrast, the PT model’s prediction of a positive gap effect is borne out in all three currency markets examined. In two of these markets, IDP enters the cointegrating relationship as predicted. The PT model also accounts for sign reversals better than the international CAPM. Interestingly, our CVAR results show the strongest evidence for the hybrid model in which the gap, conditional volatility, and IDP drive expected excess returns.

The results suggest that financial markets do a better job in setting prices than implied by behavioral finance. Risk and fundamentals do matter, but to see their importance, we need to move away from EUT and REH.

Behavioral finance has uncovered much evidence that psychological factors such as emotion and confidence underpin market participants’ forecasting. Its reliance on REH, however, leads then to interpret such underpinning as evidence of less than full rationality. After all, if rational participants have access to the true probability distribution, there would be no need to rely on anything else. Doing so would be a sign of irrationality and poorly functioning markets.

IKE models, by contrast, recognize that rational forecasting must rely on psychological factors, as well as fundamentals and calculation (Frydman and Goldberg 2011 and Frydman and Stillwagon 2018). Rational individuals understand that they do not know the truth about future returns and risk. They therefore consider a range of possible scenarios that are underpinned by reason and past data. There is no objective way to determine ex ante which scenario is better than any other. Rational individuals therefore must rely on their emotion, confidence, and intuition in choosing a scenario on which to base their forecasting.

IKE’s recognition that we all must cope with imperfect knowledge leads to an intermediate view of financial markets. They are indispensable in harnessing the diverse views and insights about the future that exist in society. But, because trading is based on imperfect knowledge, markets are not perfect in setting prices. Asset prices undergo persistent swings away from and back toward benchmark values in their normal course of functioning. Short-termism can at times lead to departures that are excessive. It is during these times that Frydman and Goldberg (2011) call for excess dampening macroprudential policies.

See original post for references

Print Friendly, PDF & Email


  1. Susan the Other

    Aaaargh. This one gave me such a headache. I wish they had put this little insight in the first paragraph, not at the end, “After all, if rational participants have access to the true probability distribution there would be no need to rely on anything else.” aka there would be no need for a market. There would be no price fluctuation or manipulation. There would be no profit in currencies, only in progress and social and environmental value. Trying to predict random variables or fools is too random for economists to mess with? It’s gambling at best; socializing losses usually. I think we can safely say that the market is a godawful research design for determining value. It’s like determining the orange by looking at the apple. And after all, intuition is a good thing if it comes from a good person. Otherwise it can get a little sinister. The thing that should be harnessed is the perversity and prevalence of bad information. Capital markets are an inefficient way of promoting an economy based on value and progress. Or did I miss the point altogether?

  2. diptherio

    “Ignoring the primacy of imperfect knowledge has led to many puzzles in financial markets.” I.e. denying the obvious makes it difficult to understand things.

  3. Sound of the Suburbs

    The overall value depends on the amount of money chasing a fairly fixed number of financial assets.

    The central banks demonstrated how they could inflate the markets by pouring more QE money in (aka liquidity) to create a “wealth effect”.

    Then there is relative pricing within those markets.

  4. Synoia

    EMH rests on the rational expectations hypothesis (REH), which supposes that rational market participants know the true probability distribution governing outcomes and use it to interpret news about companies’ prospects.

    If true, financial markets would not be driven by fear of loss. nor opportunity (greed) for gain.

    Such emotions are not “rational,” because they are emotions, and cause Non Linear Feedback, Irrational Behavior, leading to Chaos…and so on.

    I wonder if AI’s have fear and Greed, and that coud be proven…

  5. Chauncey Gardiner

    If returns are not driven by risk or fundamentals, that raises the question of whether financial markets themselves are useful as allocators of capital. If alternative systems or structures have been established and are quietly being used to set market valuations that do not reflect capital allocations consistent with risk and fundamental considerations, then should we not explore alternatives to markets in allocating capital?

  6. animalogic

    The REH has always struck me as completely bizarre, if not perverse. How intelligent persons could actually assert that something as mind bogglingly complex as markets could be in some essential way “rational” has always stuck me as (equally) bizarre.
    The whole thing has a religious smell to it. Given that religion itself is absent, I can only conclude that ideology (& perhaps intellectual inertia) is the chief motive behind REH

    1. bruce wilder

      It is often explained in ways that make it sound more bizarre than it is. And, it has been wielded ideologically in completely unjustified ways — as an assertion, for example. The authors of the OP here did not do a credible job of outlining the intuitions of either rational expectations (which is an analytic modeling convention) or the efficient markets hypothesis, which is technically a way to form a null hypothesis for research on historic transaction price data for financial markets.

      The practical problem of assessing the efficient functioning of financial markets and relating that assessment to the institutional design or management of the market as an institution has not been much advanced by criticism of rational expectations or efficient markets.

      The problem of practical assessment would seem to be answerable as a matter of quantitative degree, perhaps in a way analogous to the way an engineer might measure the heat and power efficiency of a steam engine. But, economists have gotten hung up on the problems posed by uncertainty, as the OP indicates.

      I think Yves is wrong about the loanable funds error at the beginning. The only way this works is as operational modeling and the authors do not see the need for operational knowledge. No one with operational knowledge of finance would lead with loanable funds.

  7. dutch

    The future resembles the past until it doesn’t. The behavior of a mass of non-cooperating individuals is as predictable as the movements of a flock of starlings, or a school of fish. For short durations their group’s relative movements appear coordinated and predictable until a perturbation abruptly and randomly alters the direction of flow. Within the group the actions of individuals are coordinated only to the extent that each attempts to stay within the group without colliding with its immediate neighbors. A market is a flock. Predictions are hard.


Leave a Reply

Your email address will not be published. Required fields are marked *