London, 6 May 2011: Reuters

The brutal sell off that swept through the oil markets Thursday should finally dispel some myths about how commodity prices are set and behave. Front-month Brent crude futures sank almost $12 per barrel (well over 9 percent) in a series of vertiginous declines that took the market down from over $120 at the start of the day to under $110.

The price change was more than 4 standard deviations -- which is something that should be seen on average only once in every 63 years -- if the market was well-behaved and changes followed a normal or Gaussian distribution. At times the move has approached 5 standard deviations -- which should only occur once every 7,000 years.

In fact, spot Brent prices have moved by 9.5 percent or more no fewer than 33 times since January 1990. The most recent was the 9.7 percent rise on March 17, 2009, which marked the end of a series of no fewer than 11 massive moves in the autumn 2008 and spring 2009.

Before that we have to go back to the aftermath of the terrorist attacks on the World Trade Center and Defense Department in 2001 (2 moves), 2000 (2 moves), 1998 (1 move), 1996 (2 moves) and the first Gulf War in 1990-91 (16 moves) for similar or larger movements.

The first myth that should be demolished is the idea that commodity price movements follow even an approximately normal distribution. Large price movements occur too frequently for the normal distribution to be a useful proxy. So it may be time to throw overboard all those elegant value-at-risk (VaR) models or at least admit they mislead more than they reveal. It also spells near-fatal trouble for many option-pricing models.

Some analysts and risk professionals have tried to rescue VaR and option models by tweaking them to include fat tails ("kurtosis") that give more weight to extreme outcomes. But no amount of tweaking can cope with the frequency of 4 and 5 standard deviation moves exhibited in the oil market over the last two decades. It is more realistic to accept that price movements are not really normally distributed at all.

Such large price movements confirm the brilliant mathematician Benoit Mandelbrot was right in arguing modern financial theory was founded on shaky foundations and massively underestimated the real amount of risk lurking in markets.

Crucially, Mandelbrot discovered the average amount of volatility in prices was not constant but varied over time. He noted that markets appeared to move through distinct phases of low and high volatility. Days with unusually big price moves tended to cluster together, and so did days with unusually small variability. He described markets shifting from a mild state to a turbulent one and back again.

Mandelbrot's description of price movements in commodity markets is much more realistic than the conventional theories that populate textbooks and lurk behind much academic research -- which should come as no surprise because he originally discovered the non-normality of price movements in a long time series of cotton prices.

The second myth that deserves a decent burial is that commodity prices are driven almost exclusively by fundamental forces of supply and demand and are unaffected by the build up and liquidation of speculative positions.

This argument has never seemed very plausible. But its adherents have clung on with surprising tenacity. It remains the official line for much of the research establishment as well as regulators, who continue to argue price movements can be largely or entirely explained by fundamental factors and that there is "no evidence" that speculation or investment flows have an impact.

It would be hard to find ANY fundamental factor that could explain how Brent crude prices could be worth $120 at the start of Thursday but less than $110 under twelve hours later.

Rising risk aversion or other fundamental economic factors certainly cannot explain the move. The Dow Jones Industrial Average has fallen less than 1.5 percent. Only developments internal to the commodity markets can explain the sell off.

Once it is accepted the accumulation and liquidation of positions can affect prices to such an enormous extent over short periods owing to uncertainty about fundamentals, limitations on liquidity, and shifts in sentiment, it is logical to accept that speculation can and probably does have some impact over longer time scales.

So what next? Mandelbrot's theory suggests market participants should brace for more big moves in the days and even weeks ahead if the market has shifted from a mild to a turbulent state. There is absolute no guarantee that the market will not crater further tomorrow -- or show a sharp rebound.

Either way, Thursday's crash in oil and other commodity prices has burned more than the commodity longs.

Ends --

By John Kemp, Reuters market analyst – for Commodities Now.

The views expressed here are his own.