London, 19 September 2011: Reuters

The poor track record of most price forecasts for oil and other commodities is well known. I have written before on the problems with current forecasting techniques, and urged forecasters to embrace uncertainty by publishing forecasts in the form of probability distributions rather than a single number.

So what would a good price forecast look like? First and foremost it would recognise uncertainty on both sides of the market. Both supply and demand outlooks would be expressed in terms of a probability distribution. For each, there would be a central projection and a range of outlying outcomes based on (a) the level of ordinary variability derived from past estimating errors; (b) an assessment of whether uncertainty was higher or lower than normal; and (c) a skew reflecting specific factors expected to affect the balance of risks.

Combining the distributions for supply and demand would give a joint distribution or probability surface for the expected market balance and a basis for making probabilistic price predictions.

If this sounds complicated, the statistics are fairly simple, and in common use in other areas such as weather forecasting. In fact, probabilistic approaches are already hard-wired into the commodity markets. They underlie all futures and options trading. Hedgers and speculators implicitly compare their own (subjective) estimate for the range of outcomes with (objective) probability forecasts encapsulated in the forward curve and volatility surface in deciding whether to buy and sell futures and options contracts.

As one senior market participant observed, "the forward curve is not our forecast, it is what we trade against". Shifting to probabilistic forecasts would enable analysts to catch up with the approach already being used by most of their clients, and significantly improve the usefulness of the predictions.

RESOLVING THE PRICE PARADOX

The need to embrace a more probabilistic approach to forecasting is highlighted by the editorial in the International Energy Agency's September "Oil Market Report" which notes "Market observers are puzzling over the paradox of weakening economic growth and oil demand indicators on the one hand and $110/barrel crude on the other".

It is a theme taken up by the Financial Times last Thursday in an article describing "Traders split on knife-edge oil market outlook". It describes a "titanic battle" between some macro hedge funds betting oil prices will fall while most physical traders expect them to rise. Bearish macro funds are betting a slowdown in the United States and other advanced economies, perhaps spreading to emerging markets, will cut oil demand, pushing prices lower. In contrast, physical traders focus on the backwardation in Brent and strengthening price differentials as a sign of tight supply. Both sides are right, but failing to look at the whole picture, thus talking past one another.

Physical traders are looking mostly at the supply picture, taking demand as given, when in fact there is more uncertainty about the macro outlook than many care to acknowledge. The macro funds are using oil prices as a way to express their view of the economic outlook, while taking supply considerations as given, ignoring a number of non-macro risks.

Each side is looking at only one probability distribution (demand for the macro funds, supply for many physical traders) while taking the other side of the market as given. Many physical traders see prices as low given the current tightness in the physical market and the possibility of further supply shortfalls in 2011 and 2012.

But they are implicitly taking demand as given and assuming that the United States and other economies will successfully avoid a double dip recession. Macro funds see prices as too high given the darkening economic outlook, but give too little weight to supply problems. Once the probability distributions for both supply and demand are taken into account, though, the paradox disappears. Prices remain high at $110 on expectations about continued supply restrictions, while they are prevented from going higher by concern about the risk of recession.

DISTRIBUTIONS NOT POINT ESTIMATES

Probabilistic forecasting is essential to a proper understanding of the oil market. Point estimates for supply and demand fail to capture the full range of relevant information that drives the views of market participants and price formation.

To take an example on the demand side, a recent poll of more than 100 macro forecasters by Reuters showed only two predicted a contraction (a single quarter of negative growth) anywhere across the United States, Britain and the euro zone during Q3 2011, and none foresaw a contraction in Q4. Of 789 quarterly forecasts for growth between Q3 2011 and Q4 2012, just three showed output falling at any point (less than 0.4 percent of all forecasts submitted).

http://graphics.thomsonreuters.com/ce/GDP-FCASTS.pdf

Most forecasters expected growth to be slow, but for the major North Atlantic economies to skirt outright recession. If there is "stall speed" below which recessionary forces become self-fulfilling, something which is hotly disputed, the macro forecasters all believed the North Atlantic economies will escape this fate.

In general, oil analysts are required to use growth forecasts produced by their macro colleagues, which explains why most oil price forecasts assume the western economies will escape a double dip. But the macro forecasts reported in the Reuters survey, and used as inputs to most oil outlooks, are based on a simple point estimate or "most likely" outcome. Only if upside and downside risks were symmetric would such point forecasts, on their own, be useful. In fact at the moment they conceal substantial downside risk, most of which is therefore missing from published oil price projections.

According to a survey of forecasters published in the Wall Street Journal last Thursday, "economists see a one in three chance the U.S. will slip into recession in the next twelve months and doubt any steps the Federal Reserve might take at its meetings next week can change that" ("Economists say that U.S. recession looks more likely").

The Conference Board puts the risk at 45 percent, topping the 40 percent mark that has heralded a downturn on every occasion since 1988, according to the Journal. There is significant overlap between the contributors to the Journal and Reuters surveys, but point forecasts for the most likely outcome cannot capture the shifting amount of downside risk.

By incorporating the economic outlook into their oil demand projections via a simple point forecast rather than a probability distribution, oil analysts are missing an important piece of information.

In the past, oil analysts could afford to ignore much of the uncertainty on both the supply and demand sides of the market, or reduce them to simple characterisations of the "balance of risks". But as the market becomes increasingly forward-looking, and hedgers and speculators take positions further out along the forward curve, the need to predict medium-term variables and estimate the degree of uncertainty surrounding those forecasts is becoming increasingly important.

In the next 2-3 years, more forecasts are likely to appear in the form of probability distributions, even highly simplified ones involving "scenarios", as clients press for more detail about the whole range of possible outcomes, not just the central tendency.

Ends --

By John Kemp, Reuters market analyst – for Commodities Now.

The views expressed here are his own.