naked capitalism

A Bloomberg story today points out that the snowballing credit market crisis is an indictment of the use of quantitative measures of risk, particularly one of the longer established and still widely used approaches, value at risk. VAR uses historical trading patterns to determine the probability of loss to a certain percentage of certainty. Firms will set certain risk thresholds for certain types of risk, say 95% or 99% certainty of no loss over a certain time frame.

While VAR is popular, no firm relies solely on VAR. But a big problem is that it is the measure of risk that regulators understand best and thus tends to dominate conversations between regulators and their charges. Some have even claimed that securities firms look to VAR more than they would otherwise due to the role it plays in industry oversight.

What is wrong with VAR? There are three big shortcomings. First is that it relies on historical norms. When you have new instruments with limited trading history like mezzanine CDOs that have never been tested in either a down market or a weak economy, the past is often not a reliable guide to future performance. Second is that even instruments that appear to be the same over time, such as subprime loans, may not in fact be the same instrument in terms of economic performance and therefore trading risk. Subprime mortgages nominally have a ten-year history, but the product in its early years was issued in small volumes and consisted primarily of manufactured housing. And as we now know all too well, vintage 2004 subprimes were vastly better credit risks than the 2007 edition.

The third problem with VAR is that the underlying pricing models assume that securities prices have a normal (as in bell curve) distribution. But that just isn’t true. Securities prices exhibit fat tails (extreme moves are more probable than the models assume) and are not symmetrically distributed (stock prices, for example, exhibit negative skewness, meaning the negative portion of the distribution extends further from the mean than the positive end). This says that VAR needs to be used with a handful of salt in those extreme 95% to 99% measures, since that is where the model is most likely to break down (the way of compensating presumably would be to set your risk parameters considerably higher than you would otherwise). But from what I can tell at my remote range, most (all?) players have tended to assume that a very high degree of certainty is good enough.

Now from a practical perspective, since VAR is not the only risk metric in use, firms may compensate for the shortcomings listed above by other means (although recent results would say whatever tools they used were also flawed). But the most serious implication is that this failing is that it shows the regulators were emperors with no clothes. And it appears that none of them plans to hire a tailor.

From Bloomberg:

The risk-taking model that emboldened Wall Street to trade with impunity is broken and everyone from Merrill Lynch & Co. Chief Executive Officer John Thain to Morgan Stanley Chief Financial Officer Colm Kelleher is coming to the realization that no algorithm or triple-A rating can substitute for old-fashioned due diligence.

Value at risk, the measure banks use to calculate the maximum their trades can lose each day, failed to detect the scope of the U.S. subprime mortgage market’s collapse as it triggered more than $130 billion of losses since June for the biggest securities firms led by Citigroup Inc., Merrill, Morgan Stanley and UBS AG.

The past six months have exposed the flaws of a financial measure based on historical prices that securities firms use idiosyncratically and that doesn’t anticipate every potential disaster, such as the mistaken credit ratings on defaulted subprime debt…

Executives at Merrill, Morgan Stanley and UBS took steps in the past six weeks to overhaul their risk-management groups after internal models failed to foresee the first annual decline in house prices since the Great Depression that eroded five years of trading gains.

Goldman Sachs Group Inc., the firm with the highest nominal VaR, was the sole investment bank to report record earnings in the fourth quarter, while New York-based Merrill, which had the second-lowest nominal VaR of the five biggest U.S. securities firms, posted a $9.8 billion loss for the last three months of 2007, the biggest in its 94-year history…

Hiring risk managers and giving them more power won’t alter the mistake that led to last year’s slump and that was Wall Street’s dependence on statistics to quantify risks, [Nassim] Taleb {a research professor at London Business School and former options trader} said.

“We have had dismal failures in quantitative finance in measuring these risks, yet people hire quants and hire risk managers simply to back up their desire to take these risks,” he said. “There are some probabilities that you cannot compute.”…

All the New York-based firms base their calculations at a confidence level of 95 percent, meaning they don’t expect one- day drops to exceed the reported amount more than 5 percent of the time.

The amounts differ in part because every firm uses their own methodology and data. For instance, Lehman uses four years of historical data to calculate VaR, with a higher weighting given to more recent time periods, while Morgan Stanley provides VaR calculations using both four years and one year of market data.

“If you compare what peoples’ values at risk are versus what their losses were in the third quarter or fourth quarter, the numbers are astounding,” said David Einhorn, president and co-founder of hedge fund Greenlight Capital LLC in New York. “There are a lot of things that probably the value-at-risk model said would have trivial losses 95 percent of the time or 99 percent of the time but are now having a huge loss.”….

All of the risk-measurement tools failed to prepare Merrill for the unforeseen declines on triple-A rated securities backed by subprime mortgages, according to the company’s third-quarter filing with the U.S. Securities and Exchange Commission. The firm’s writedowns related to the highest-rated portions of CDOs backed by pools of home loans, which plunged in value as defaults on the underlying mortgages soared.

“VaR, stress tests and other risk measures significantly underestimated the magnitude of actual loss from the unprecedented credit market environment,” Merrill’s filing said. “In the past, these AAA ABS CDO securities had never experienced a significant loss in value.”

Securities firms developed statistical models during the early 1990s to better quantify risks as the trading of bonds, stocks, currencies and derivatives increased. J.P. Morgan & Co., now part of JPMorgan Chase & Co., helped popularize the use of value at risk as the primary measurement tool in 1994 when it published its so-called RiskMetrics system.

Four years later, two events helped demonstrate the drawbacks in using statistical analysis based on historical market movements to measure risk. Russia’s bond default sent fixed-income markets into a tailspin and Long-Term Capital Management LP, the Greenwich, Connecticut-based hedge fund run by former Salomon Brothers trader John W. Meriwether, had to be bailed out after $4 billion of trading declines.

Russia’s default risk was underestimated because value-at- risk computations used by investment banks depended on market events of the preceding two to three years, when nothing similar had occurred, according to Wilson Ervin, who’s now chief risk officer at Zurich-based Credit Suisse Group, Switzerland’s second-biggest bank after UBS.

Long-Term Capital Management, which amplified its risk by relying on borrowed money for most of its trading bets, blew up in part because it didn’t anticipate that investor panic after the Russian default would cut the value of any risky debt, whether it was issued by a country, sold by a company, or backed by mortgages.

The riskiest Russian and Brazilian bonds owned by the fund plunged far more than the safer Russian and Brazilian bonds that it had bet against as a hedge, according to “When Genius Failed,” the book written by Roger Lowenstein.

“In a market stress event, some individual sectors that previously appeared unrelated do move together, and as a result, the organization could take losses on both of them or even on positions that were previously deemed to be a hedge,” said Ed Hida, the partner who runs the risk strategy and analytics services group at Deloitte & Touche LLP in New York.

The other risk tool commonly used by securities firms, known as stress testing or scenario analysis, also failed to prepare the industry for the plummeting value of AAA-rated securities that had previously been deemed the most creditworthy, he said.

“Stress tests are only as good or as predictive as the scenarios used and in many cases the scenarios that played out were much more severe than people anticipated,” Hida said. “One lesson learned is that these stress tests should be broader, should consider more scenarios.”

Kelleher, who became Morgan Stanley’s CFO in October, explained the flaw in the firm’s stress testing in a Dec. 19 interview, the day the company reported its first unprofitable quarter.

“Our assumptions included what at the time was deemed to be a worst-case scenario,” he said. “History has proven that the worst-case scenario was not the worst case.”

At Credit Suisse, one of the firms that have so far skirted the worst subprime declines, Ervin said value at risk played no role in helping him navigate the market turmoil.

“Once you go into a crisis like this, I think risk is much more about sitting down with traders, and talking about very specific issues and scenarios,” he said. “VaR we know will kind of lag going into a crisis so we don’t really watch that as a crisis indicator.”

Still, Ervin said VaR provides a service if used every day because it can pick up fluctuations in the risk that the firm is taking in some distant region or an arcane product that might not otherwise be noticed.

Investment banks will continue to take unsafe risks as long as traders are rewarded for making profits, leaving shareholders, bondholders and sometimes taxpayers to shoulder the consequences, Taleb said.

Wall Street traders “make an annual bonus and get an annual review based on risks that don’t show up on an annual basis,” Taleb said. “You have all the incentive in the world to take these risks.”