London, March 2011

The usefulness of the black swan concept developed by Nassim Nicholas Taleb is being devalued by over-use and indiscriminate application to events that are merely quite rare and hard to forecast -- rather than so rare that they are inconceivable or unimaginable.

That would be a shame. It would leave the language poorer and do a disservice to analysts, economists, insurers and investors trying to grapple with, hedge and manage a world full of variable and surprising outcomes.

Former U.S. Defense Secretary Donald Rumsfeld captured the crucial differences between risk, uncertainty and black swans in his observation about known knowns, known unknowns and unknown unknowns -- the things we don't know we don't know because we haven't conceived or thought of them yet.

In the tradition of Frank Knight, economists and insurers have long distinguished between risk and uncertainty ("Risk, Uncertainty and Profit", 1921). Economists reserve risk to describe situations where outcomes are variable but the full set of possible outcomes is known, and the probability attached to each is also known in advance. Wagers using a fair die, pack of cards or a roulette wheel are all examples of risk.

Risk is simple to measure and estimate and fairly simple to hedge or insure against, and is the staple of financial and insurance markets. Risk corresponds to Rumsfeld's known knowns. Rumsfeld was thinking about things that were certain to happen. But the concept of known knowns can be extended to things that might happen with a clearly specified probability.

Known knowns can be 100 percent certain (the sun will rise in the east tomorrow) or probabilistic (a fair coin will land heads). Conceptually there is no difference. The expectation for the sun rising in the east tomorrow morning is simply a 100 percent chance of it rising in the east and a 0 percent chance of it rising in the west, compared with a 50 percent chance of the coin landing hands and a 50 percent chance of tails.

KNIGHTIAN UNCERTAINTY

Following Knight, economists and insurers reserve the concept of uncertainty for situations where the range of possible outcomes can be described but probabilities cannot be known or calculated in advance, at least not accurately and with confidence.

Uncertainty covers situations such as massive earthquakes off the coast of Japan or the overthrow of President Hosni Mubarak in Egypt. It is possible to specify all the outcomes in advance (earthquake, no earthquake; overthrow, no overthrow) but hard to assign precise probabilities to them.

Much of economic and insurance modelling uses observed probabilities in the past to forecast the likelihood of something happening again in future, but this approach does not work well for outcomes that happen only extremely rarely.

Extreme events such as an 8.9 earthquake, the overthrow of governments or a systemic banking crisis happen so infrequently the probabilities are hard to observe or to calculate and model in advance.

Prior to March, even 100 years of seismic data could not have predicted the occurrence of an 8.9 earthquake off Japan, and even a century of daily stock market returns did not predict the 1987 stock market crash. Extreme events are properly known as "tail risks" and are hard to measure or model using the most popular statistical techniques.

The limitations of using the Gaussian or normal distribution to measure tail risks are now well known after the banking crisis. But even if the Gaussian distribution is tweaked to take into account skew and fat tails ("kurtosis"), it still provides no useful information about the level of risks out in those tails.

Adjusting the Gaussian distribution for fat tails accepts that tail risks happen more often than the unadjusted distribution would predict. But it doesn't measure the probability of an exceptionally bad outcome. Unlike risk, uncertainty is hard to measure and therefore hard to price accurately or insure against. But neither risk nor uncertainty is an example of a true black swan.

TRUE BLACK SWANS

If risk refers to situations with known outcomes and known probabilities, and uncertainty is known outcomes and unknown probabilities, black swans are outcomes that were not or could not be known in advance (even uncertainly), because they were not thought possible or had never even been considered.

Often black swans arise because an outcome is so very, very rare it has never been observed before and so was not thought possible and had not even been considered. It is an example of the dangers of inductive logic. Just because the first 1000 swans observed happen to be white does not mean that all swans are white.

True black swan events are rare -- and having occurred once cease to be black swans. Following the discovery of black swans in Australia, finding a black swan would no longer be surprising, though finding a bright green one would be.

True black swans are genuine unknown unknowns, in Rumsfeld's terminology. They happen so infrequently they could not even been imagined or conceived. An attack on the World Trade Centre using commercial airplanes as weapons probably counts as a true black swan. Except for a handful of terrorism experts and the hijack planners themselves, the possibility had never been considered. The fact that one man setting himself alight could trigger the downfall of Tunisia's president potentially qualifies as another.

In this sense, however, Japan's earthquake was not a black swan. It was well understood that earthquakes could and did happen along the edge of the rim of fire, including some very big ones, but the probability of a quake as large as 8.9 was so very small that it was hard to measure and forecast accurately. It was an example of uncertainty (unknown probabilities) rather than a black swan.

The risk of Mubarak's overthrow in Egypt was not a black swan either. Political risk analysts have been writing for years about discontent in the country. But on any given day, or in any given month, the probability of Mubarak being overthrown was very small and hard to measure or forecast accurately.

Putting it in statistical terms, was the risk of Mubarak's overthrow in any given month 0.1 percent or 0.25 percent or 1 percent? Is it even possible to forecast events with such low probabilities accurately?

Looking forward, the threat of regime change in Bahrain, Libya, Saudi Arabia or Iran is uncertain but not a true black swan. Nor is an airstrike on Iran's nuclear facilities. Analysts and investors have pored over these threats for years and in most cases dismissed it as very far-fetched.

In contrast, some risks are genuine black swans, at least for most people the first time they occur. Systemic interdependencies in the financial system built by statistical arbitrage and quant trading strategies were black swans the first time they were revealed by the LTCM crisis in 1998 and arguably again in August 2007.

They cannot be considered black swans any more since the risk is known, though its probability is not. The whole concept of black swans is being massively overused at the moment to refer to outcomes that are merely risky, uncertain or unlikely. That's a shame because separate categories of risk, uncertainty and black swans are useful and could be devalued and obscured by poor usage.

Characterising low probability events such as the Arab revolutions or Japan's earthquake as unpredictable black swans is lazy. It also undercuts efforts to think hard about low probability extreme outcomes that nonetheless dominate investment and insurance strategies. Moreover, if too many outcomes are labelled black swans, then real black swans get lost in the flock.

By their very nature, black swans are very difficult or even impossible to conceive in advance (if they weren't they would not be black swans). But the concept is useful because it points to the limitations of knowledge and forecasting the future based on the (recent) past.

It should cause every analyst and forecaster to question assumptions and test their robustness not simply argues after the event that some things were simply unforecastable.

Ends --

By John Kemp, Reuters market analyst – for Commodities Now.

The views expressed here are his own.