Revealed: New Insight into What Really Drives the Stock Market

Yves here. The headline overstates the conclusions from new research on the stock markets. However, the study serves to confirm that economists’ need to depict consumers and investors as cool and rational is all wet. But the assumption of strong-form rationality was essential to toy models that showed markets as producing virtuous outcomes. And do not forget, the need to shore up the pretense that market-based activity is at least efficient and better yet optimal (in terms of resource allocation) has become fundamental to the defense of capitalism. In other words, consumer/investor rationality is central to the economic dogma that defends capitalism.

We’ll get to the argument in the new INET paper soon, but basically, its authors are chuffed to establish that stock market activity is driven by stories. That is old hat to anyone on Wall Street. It is also old hat that decision-making revolves around stories; criminal attorneys know well that their success depends on getting the jury to put together the evidence and legal interpretations into a story they find convincing as the basis for their decision.

But back to the importance of the fiction of rational decision-making to economic dogma. For more detail, read ECONNED. Some key parts:

The scientific pretenses of economics got a considerable boost in 1953, with the publication of what is arguably the most influential work in the economics literature, a paper by Kenneth Arrow and Gérard Debreu (both later Nobel Prize winners), the so-called Arrow-Debreu theorem. Many see this proof as confirmation of Adam Smith’s invisible hand. It demonstrates what Walras sought through his successive auction process of tâtonnement, that there is a set of prices at which all goods can be bought and sold at a particular point in time.42 Recall that the shorthand for this outcome is that “markets clear,” or that there is a “market clearing price,” leaving no buyers with unfilled orders or vendors with unsold goods.

However, the conditions of the Arrow-Debreu theorem are highly restrictive. For instance, Arrow and Debreu assume perfectly competitive markets (all buyers and sellers have perfect information, no buyer or seller is big enough to influence prices), and separate markets for different locations (butter in Chicago is a different market than butter in Sydney). So far, this isn’t all that unusual a set of requirements in econ-land.

But then we get to the doozies. The authors further assume forward markets (meaning you can not only buy butter now, but contract to buy or sell butter in Singapore for two and a half years from now) for every commodity and every contingent market for every time period in all places, meaning till the end of time! In other words, you could hedge anything, such as the odds you will be ten minutes late to your 4:00 P.M.meeting three weeks from Tuesday. And everyone has perfect foreknowledge of all future periods.
In other words, you know everything your unborn descendants six generations from now will be up to.

In other words, the model bears perilous little resemblance to any world of commerce we will ever see. What follows from Arrow-Debreu is absolutely nothing: Arrow-Debreu leaves you just as in the dark about whether markets clear in real life as you were before reading Arrow-Debreu.

And remember, this paper is celebrated as one of the crowning achievements of economics.

And the notion that financial markets worked the way financial economics said was also exposed early on as bunk, but the theoreticians preferred to keep selling it than admit they didn’t have much to offer. Elite mathematician Beniot Mandelbrot got his hands on what was then the largest set of trading data and found that the distribution was not “normal” (aka Gaussian or “bell curve”). Their randomness was much wilder. He found that trading data from other markets showed the same behavior.

Here’s what happened next:

Mandelbrot and his ideas began to circulate in the financial economics community. At first, the reception was positive. The European polymath became an informal thesis advisor to University of Chicago economist Eugene Fama, who had found that the prices of the members of the Dow Jones Industrial Average were indeed not “normal” but were what statisticians called “leptokurtic,” with high peaks, meaning they had more observations close to the mean than in a normal distribution, but also much fatter tails. In lay terms, that means dayto- day variability is low, but when unusual events occur, variability both is more extreme and occurs more often than would occur with a normal distribution. MIT’s Paul Samuelson and other economists started looking into Lévy distributions and their implications.

The problem with Mandelbrot’s work, however, was it threatened the entire edifice of not simply financial economics, but also the broader efforts to use formulas to describe economic phenomena. Lévy distributions didn’t merely have difficult math; that might have been an intriguing challenge. There wasn’t even a way to calculate Lévy’s “alpha” reliably, although Fama’s efforts with market data did show that it was less than two, which confirmed the fear that the distributions were not normal.

The backlash was predictable. MIT professor Paul Cootner, who later published a book of essays on the random-walk hypothesis, tore into Mandelbrot at a winter 1962 meeting of the Econometric Society:

Mandelbrot, like Winston Churchill before him, promises us not utopia but
blood, sweat, toil, and tears. If he is right, almost all of our statistical tools are
obsolete. . . . Almost without exception, past econometric work is meaningless.

Surely, before consigning centuries of work to the ash pile we should like to have some assurance that all our work is not truly useless. If we have permitted ourselves to be fooled this long into thinking the Gaussian assumption is valid, is it not possible that the [Lévy] revolution is similarly illusory? At any rate, it would seem desirable not only to have more precise and unambiguous evidence in favor of Mandelbrot’s hypothesis as it stands, but also to have some tests with greater power against alternatives that are less destructive of what we know.

But Churchill had been right. The British prime minister had advocated a
difficult, perilous, and ultimately successful course of action, yet Cootner perversely
invoked him to argue instead for a failed status quo. He wanted assurances that exploring new terrain would be successful, but that isn’t the way a paradigm shift works. Indeed, Lévy distributions might not provide a comprehensive solution, but the point is to move toward better approximations, particularly when the existing ones have serious shortcomings.

Now to the current post. Note it suggests at the end not being under-diversified…but the notion that sufficient diversification (whatever that is) is good comes out of the Capital Asset Pricing Model. As we also explained in ECONNED, CAPM suffers from fatal internal contradictions, as even its creator, Bill Sharpe, conceded. But he reversed himself, falling in with the widespread view that a bad model was better than none.

By Lynn Parramore, Senior Research Analyst at the Institute for New Economic Thinking. Originally published at the Institute for New Economic Thinking website

If watching the stock market is giving you that old sinking feeling, you’re not alone. Inquiring minds want to know if the funds they’ve so carefully invested in 401(k)s are going to be there for them down the road. Investors of all kinds are worried about wild swings and lingering perils.

Here’s the unnerving thing: No matter how much market enthusiasts hype the stock market’s “efficiency” and hail it for incorporating all possible information, you’re always in for a surprise. That’s because just like death and taxes, you can count on stuff you never saw coming to throw off even the most careful and detailed planning.

Sometimes these unforeseeable events are big ones with massive market consequences. The “black swans,” as they are called, come in the form of terrorist attacks, financial crashes, and yes, pandemics. If an alien delegation lands on the White House lawn, that’s a black swan. These events ask our brains to process a lot of new and complex information. Suddenly old assumptions become irrelevant. New worlds open up.

When we are trying to make sense of things we don’t understand and can’t be certain about, our brains naturally turn to stories. For a long time, scholars in many disciplines, like history, psychology, and anthropology, have understood that narratives are extremely important to people and all that concerns them. Economists, however, have been late to the party. That’s why practitioners like Nobel laureates George Akerlof and Robert Shiller sought to change this by creating a subfield known as “narrative economics” which takes stories seriously and recognizes that they are major drivers of the economy.

The Story-Driven Stock Market

In his new contribution to narrative economics, Georgia Southern University economist Nicholas Mangee delves into the link between stories and unrepeatable, unforeseen changes and how they impact the stock market. In his new book from the Institute for New Economic Thinking’s book series with Cambridge University Press, Mangee tracks evidence of narrative dynamics through millions of daily stock market news stories from the last two decades of data, presenting an analysis of nonrepetitive events and associated story elements and how they relate to stock market changes.

Mangee did graduate work under maverick economists Roman Frydman and Michael Goldberg, who helped him to see that 1) unforeseeable change is always happening in financial markets and 2) mechanical ways of thinking that dominated traditional economics were inadequate to understand it — namely the assumption that the future follows mechanically from the past.

He knew that radical uncertainty has to be taken seriously.

Mangee saw that when it comes to the stock market, uncertainty and ambiguity are at the very core, which is why investors are all about stories. His dissertation work on financial news analytics convinced him that the factors driving stock prices are always changing over time, and nobody really knows which of them will matter the most to investors.

In a recent discussion, Mangee gave an example of just-released jobs numbers: “They blasted past expectations, but is that really bullish? Or is it potentially bearish because it confirms for some people that the Fed will tighten and rates are going to come up by more than they expected? Even when you have news that is scheduled, like jobs numbers released by the BLS, it’s not clear how the market is going to interpret that. Context matters.”

The core question in Mangee’s work concerns the degree to which the story threads that investors rely on and perpetuate over time help them deal with unforeseeable change and the uncertainty it engenders.

As he explains in his book, black swans draw people to stories and produce emotional reactions. Our emotions then get mixed into the kind of thinking Mangee refers to as “cold calculation” as we try make sense of what’s going on. Our brains buzz with questions: How will people behave differently? How will companies do business? What government policies can we expect? How we will live during and after the event? Rational sentiment helps reasonably shape answers to these questions in terms of current information and the reality that change is unfolding in real-time.

Mangee points out that just a few weeks after the coronavirus took off, the stock market lost a giant chunk of its total value, erasing the gains of 2019 and bringing a decade-long bull market to a screeching halt. Stories also quickly came into play, fueled by comparisons to history, personal judgment and prior experience, and scary emotions like panic and anxiety. People opened their inboxes to media stories like, “Panicked Shoppers Empty Shelves as Coronavirus Anxiety Rises” (NYT), which reflected, and helped generate, the stories that they latched onto.

“Narratives are the currency of uncertainty,” Mangee writes, and key to how we view the world in uncertain, changing, times. With this insight in mind, he created a new model of stock market instability under situations where we can’t measure the odds of a certain outcome (what economists call “Knightian uncertainty” after influential economist Frank Knight). His “Novelty-Narrative Hypothesis” (NNH) offers a way to understand the stock market that standard economics has missed.

Stories for Survival

Stories emerge not only in response to the big black swans but also to smaller unforeseen events like supply chain snarls and corporate legal issues. Even a product recall or a bankruptcy can create enough uncertainty and ambiguity to spark stories. These “novel corporate events,” as Mangee calls them, can impact the bigger trends and vice versa, in a processes too complex for our brains to deal with. So investors turn to narratives as a survival tactic, a defense mechanism, and a tool for making a satisfying decision when faced with limited information and Knightian uncertainty. Stories fill in the gaps, and, as Mangee points out, turn our emotions into cognitive guideposts and anchors when we’re dealing with the meaningof novel events for future stock returns.

We’re continuously dealing with novel events. Mangee notes, for example, that two-thirds of events identified in news reports as important to corporate prospects and share prices are “unscheduled,” as are four-fifths of big events in the U.S. economy. Instability and uncertainty are everywhere. All the time. Every day.

Some stories are flexible. Others are more rigid. Some are based on pretty good information. Others are not. They reflect our prior beliefs about how things work, and they reflect our culture, societal norms, and the popular zeitgeist. Some economists, like Shiller, have looked at the negative side of stories and how they drive us to do illogical things. Shiller scrutinized “narrative epidemics” and how stories are connected to cognitive disorders and conspiracy thinking.

Mangee has a different take. He explains that unlike Shiller, he doesn’t treat stories as stuff outside the stock market emanating within a few people’s minds, but rather a fundamental part of what the stock market is all about. He emphasizes that it’s perfectly rational, normal, and actually necessary for people to engage with stories when there’s uncertainty and ambiguity. Just because emotion is involved doesn’t mean that the stories will steer us wrong, Mangee stresses.

As Mangee sees it, traditional economists have tended to misunderstand how humans make decisions, holding the view that emotion and cold calculation are necessarily at odds. But he (like scholars from many other disciplines) holds that both are both perfectly rational and necessarily interdependent.

Mangee points out that capitalism produces profit –- and loss — through unanticipated change. “You can have knowledge and be skillful and roughly right,” says Mangee, “but you cannot know what precisely right looks like ex ante.” He says that when looking at the stock market rollercoaster currently underway, we can think of each moment in time as a node of intersecting highways of narratives concerning events that are not routine. “So you have a stock market correction of 10%. What does it mean? The Novelty-Narrative Hypothesis says that we have to take seriously that each and every day events are happening that are not perfect replicas of the past. We’re still learning about what omicron did and didn’t do to the economy, for example. The past doesn’t predict the future in a mechanical way.”

Mangee’s work suggests that ordinary investors need to diversify much more than they’re already allowing. Just holding the broad stock market doesn’t cut it. “There are too many macro shocks, so you need to be represented in more asset classes than just the S&P 500,” he says. “You need high and low yield bonds; high-cap, mid-cap, low-cap value stocks; high-cap, mid-cap, low-cap growth stocks; international, and so on. The S&P500 is dominated by high-cap. That’s not enough diversification.” Mangee’s work reminds us of how volatile financial markets can be.

His work also makes it clear that when we hear anyone making economic and financial forecasts, we need to keep in mind the limitations of those forecasts. “The recognition of those limitations should almost outweigh what the modelers are saying in their predictions,” says Mangee. “We have to give up precision but we can allow for more nuanced relationships and, importantly, for the novelty of real-world change. People don’t like that tradeoff, scientifically. They don’t like the idea that they can’t apply an objective probability distribution to something before the fact.”

Lastly, Mangee’s emphasis on the importance of narratives steers us to scrutinize our own and those of our communities. “Reality for an individual can depend on contextualized meaning of the world around us,” says Mangee. “You interact with this meaning, and the reality you shape depends on your training, your experience, your culture, your generation, your family — it’s all related.” It might behoove us to remind ourselves, especially when we’re pointing the finger at someone and saying “Oh, look, that silly person over there is making an emotional decision,” that we all make decisions based on emotion, whether we admit it or not. And that’s not a bad thing – we couldn’t deal with the unforeseeable future otherwise.

Perhaps the best way to approach the future in the face of uncertainty is with a strong dose humility. And beware of under-diversification!