Guest Post: Review of Pablo Triana’s "Lecturing Birds on Flying"

Submitted by Richard Smith:

This is the Black Swan gospel according to Triana. Taleb endorses it in a characteristically incendiary and intemperate foreword. He does come out all guns blazing, and you just have to go with that. Or chuck a glass of water over him, if he’s in range, I suppose.

A quick recap for anyone who has spent the last two years in a coma: Taleb put together the beginnings of a rap sheet for modern mathematical finance theory in his book “The Black Swan”, and rapidly attained worldwide celebrity when his criticisms appeared to be borne out by the recent financial crisis. The main tenet of Black Swan theory, rather dry sounding, but with dramatic consequences, is that price changes are not normally distributed (in the way that, say, human weight or height are), but follow a power law (‘fat tails’). This implies much greater extremes of price movement than those predicted under the assumption of a normal distribution. The events that cause such price moves may be perfectly intelligible in hindsight, but are not necessarily predictable: like the existence of black swans.

The point about price distributions is actually quite an old one. Paul Levy made the same observation in the 1900s; Mandelbrot’s studies of cotton prices, in the 60s, reached similar conclusions. What gives it contemporary relevance is that the finance theory underlying current regulatory practice, risk management, fund management and derivatives pricing all overwhelmingly assume that price changes are normally distributed. And they all failed at once in the recent financial crisis, when price changes were indeed far more extreme than a normal distribution implies. It doesn’t look so good for orthodox financial theory just now.

So it is a good time for Triana to review modern finance theory’s rap sheet, add items, and add more detail to the existing charges. It goes like this.

Chapter 2: modern finance theory is a crock, peddled by charlatans at business schools who have managed to seal themselves off from the usual empirical tests of a theory.

I’ll admit I don’t see what logical point there is in attacking the character of business school teachers in this manner, whether it is a correct assessment or not. However the empirical criticism really does stack up. Consider GS CFO David Viniar’s notorious comments from August 2007 when the ABS meltdown got into full swing (Ch1, p12): “We were seeing things that were 25-standard-deviation moves, several days in a row”. To which the rejoinder from an empirically-minded observer simply has to be “No you weren’t, imbecile: those observations actually mean that your models are hopelessly wrong”. There are several reasons why one can so insouciantly cheek such an august figure. If we assume Viniar means daily observations and a normal distribution, then (if the numbers I am cribbing are correct: I haven’t gone back to the equations) one should expect to wait quite a lot longer than the age of the universe to see even a 16-standard deviation event, with a 25-standard deviation event taking many, many times longer than that. I suppose I should work out the exact number of years, just to see how big of a number it is: exercise for any readers with access to an arbitrary-precision mathematical engine.

You can find an old post by Yves on the subject that helped kick off some blogosphere chat.

Even if you assume (very charitably, I grimly suspect) that Viniar is not just parroting his VaR model outputs (more on that later), and is a bit more sophisticated about his distributions, he is still goofing, big time. And if Mandelbrot, and Taleb, his follower, and Triana, his follower, are right about the kind of distribution that underlies financial market price movements, there just ain’t sech a thing as a standard deviation of price movements, nor no correlation neither. Both standard deviation and correlation are defined in terms of variance. Since variance is infinite for stable distributions (other than the normal distribution), neither standard deviation nor correlation is defined for the distribution of market prices (a Levy skew alpha stable distribution, if you want the full geeky glory). On this theory, Viniar is talking about things that just don’t exist. Not encouraging behaviour in a CFO.

So here is the bleedin’ obvious: given its track record of ultra-wild underestimates of the frequency of sharp price moves, the assumption of normal distributions in stock price changes must be among the most lavishly disconfirmed scientific hypotheses of all time. No wonder, then, that Taleb and Triana are somewhat ratty with its various obstinately blithe proponents.

Chapter 3: Is a bit of a digression. According to Triana, the quants who work at banks work mostly on bits of IT dealing infrastructure, which is useful, and less often than you might think on mathematical models used in trading. The quants tend to be physicists and engineers rather than business school graduates. Models are used in a much more sceptical, provisional way on the trading floor than they are in academia.

I’ll take his word for it. Evidently, scepticism of models doesn’t extend to the risk management department. And, uh, actually it doesn’t look as if that trading floor scepticism managed to avert 2007’s monster trading screwups, either. Except, perhaps in the case of GS, who famously hedged a lot of MBS exposure starting late in 2006, to the great indignation of folk who don’t understand where fiduciary duties stop and start for broker-dealers.

Now we get into the meaty detail chapters. The non-normalness of price distributions means that a whole bunch of financial orthodoxies are dubious on theoretical grounds, and, post meltdown, there are some nasty data points to back up the theory.

First up is the Gaussian copula (Chapter 4). This is a modelling device which was used to calculate default correlations, for MBS and other bonds, and thus to structure, price, rate, and hedge CDOs. I think we already know how well that went overall– but the detail of how the behaviour of various tranches of CDOs diverged from predicted paths during the ’07 meltdown is instructive. Triana leaves open the question of whether the Gaussian copula was adopted out of blind faith in its efficacy, or precisely because it underrated extreme events, and thus gave an excuse for assigning a high rating, and getting a good price. Were the ratings agencies knaves or fools in this respect? I doubt we’ll find out any time soon. Anyhow, from the data and testimony Triana assembles, it looks as if the Gaussian copula is dead in the water as a structured finance tool. One wonders how Remics and re-Remics are to be priced and rated. Any NC readers want to buy one?

Chapter 5: Now we are into VaR, the risk management methodology that JP Morgan gave to the world back in the early 90’s, in the sadly mistaken belief that being able to generate a firm wide “risk number” daily would be a useful contribution to financial risk management. Back then you were reasonably smug about your bank if central management actually knew what the firm’s positions were at all (vide Barings, Sumitomo, then fast forward again to SocGen in – oh dear – 2007), so VaR was pretty cool. It was later endorsed by the Basel regulatory framework. Then the paint started to flake off.

The shortcomings of VaR have been a regular topic at NC. That pesky normal distribution assumption again. Note the reminder from practitioner Irene in the discussion thread though – the officially sanctioned VaR model may use a rolling 2 year price history rather than a normal distribution. This desperate kludge has its own perverse side effects: in times of increased volatility, the models all tell banks to stay on the sidelines at the same time. Once the volatile part of the price history rolls out, the models are all happy again. This is not a commonsense way to run banking businesses.

The other perversity of that approach to VaR is that it encourages herd behaviour in volatile markets, before the banks have even made it to the sidelines. In other words, since all the models in all the banks are essentially the same model of the same data, they all start screaming ‘fire’ at the same time, with predictable consequences at the exits. All this and more is well covered by Triana: particularly the way that a long period of low volatility before 2007 meant that VaR endorsed massive positions in assets that were suddenly big loss makers, when things went sour.

Banks were Gadarene enough without VaR. VaR makes it worse.

Oh, one thing that bugs me about VaR as used is this: if price histories tell you nothing about future prices (EMH), why is it that price volatility histories tell you something about future price volatility (VaR)? I’m just asking.

Anyhow, Triana makes the challenging points, with persuasive evidence: first, VaR is perfectly useless (it works until you need it, and at that point, it packs up: it is the chocolate teapot of risk management); second, like MTM, it is actively procyclical.

Chapter 6 is a brisk injunction to business schools (specifically, Sloane) to snap out of it and start teaching useful stuff.

In Chapter 7, we get to another polemic, against the Black-Scholes option pricing model. One can’t fault the reasoning or evidence, but somehow this is the weakest part of the meat. It is built around a recent paper by Taleb and Haug in which they review the historical record on options market making and option pricing theory and announce that a) the parts of Black-Scholes theory that are correct are not original, having been long anticipated by Thorp-Bachelier option pricing b) the parts that are original are not correct (normal distributions are again assumed, and the model simply can’t accommodate non-normal ones, unlike Thorp-Bachelier) c) no practitioners actually use Black-Scholes. The key item of evidence for (c) is the ‘volatility smile’ by which options traders systematically adjust option prices, so that the implied volatility (calculated according to Black Scholes methods) of options actually increases progressively for deeper and deeper out-of-the-money options. Under Black-Scholes pricing theory the implied volatility should be constant across all option strike prices. Traders don’t do it that way: they are compensating for the way the BS model fails to accommodate fat tails. QED. And by the way, Triana adds, there’s no such thing as implied volatility anyway, just supply and demand pushing prices around.

Well, OK to all that, so call implied volatility “demand premium” or something, and concede that Black-Scholes is a roundabout way to prices that can be reached more directly under other theories. So now what? Black-Scholes has become part of banking’s infrastructure. Do we strip out all the Black Scholes models and replace them with Thorp-Bachelier models? Will it make enough of a difference to options pricing or risk management to be worth it? Triana doesn’t try to determine the ROI. Instead (in the Finale) he asks whether Merton and Scholes should be stripped of their Economic prizes, and eventually concludes that instead the RiksBank prize should be given a silly name, so that people know it is a bit of a crock. It is an amazingly lightweight way to round off an otherwise enlightening discussion. It doesn’t come off like a joke fallen flat either: just cheesy.

While I’m carping, I’ll add a comment on the style. When I started reading the book, I kept stumbling over awesome quasi-English barbarisms, such as “qualification-inundated resumes”, “dangerously faulty mathematically charged steering”; also horrific neologisms like “analyticization”, “nonenthusiastically”, “impacting” (adj., I kid you not, and repeatedly), and “scientification aroma” (my favourite – I want some – either in a spray dispenser or roll-on form, not fussy). To my relief Triana (or his copy editor) gets more of a grip in later chapters and it’s not such a terrible read in the end. Doubtless the same relief is reflected in the generous verdicts of Taleb (“lucid”), Tett (“readable”) and Skypala (“a treat”). So, do not despair if you find yourself entangled in some pretty strange thickets of verbiage early in the piece: it does get better if you plough on.

Back to Chapters 8, 9 and 10 so that we can end on a note more favourable to the book (just skip the Finale).

Chapter 8 is a good one on the way models can be used as alibis or excuses by the lazy, reckless, or incompetent. Good reading for head traders, risk managers and regulators, I’d say; and buy-siders and pension fund trustees, come to that. Chapter 9 is a quick round up of how seductive the spurious certainty of mathematical models can be, largely illustrated by LTCM and by the confusion surrounding the meaning of the VIX.

In the end, the message of the book is that quantitative finance is a delusion, and that common sense is a better starting point for risk management. Accordingly Chapter 10 is a paean to Fat Tony, the street smart invention of Taleb in “The Black Swan”, and a call to reverse the quantification of finance. The negative leg of the case is argued persuasively. It is discomfiting to recognise just how little there was to quantitative finance.

On the positive side of Traiana’s recommendation: well, you are welcome to make your own mind up about the reserves of common sense to be found in the banking industry just now.