Streetwise Professor

FT Alphaville ran a long piece on oil prices, and the forward curve specifically. I guess it’s good that a mainstream outlet delves into such an arcane topic, but it would have been better if they’d done better than a Fractured Fairy Tale treatment.

The piece does a service by pointing out that yes Virginia, there is a forward curve, and as a result there are different prices of oil for different delivery dates (not to mention for different qualities). Furthermore, unlike in say the equity or FX markets, there is no real “spot market” (i.e., market for delivery on the spot/immediately) for oil–or for most commodities for that matter.

The article also rightly recognizes that “the relationship between spot and futures prices in oil, as for most commodities, is actually quite complex.” Where it goes wrong is explaining that relationship.

It starts out by explaining the cash-and-carry (no) arbitrage relationship, and claims that it doesn’t hold in oil markets. Which is correct. But it gets the reason wrong.

It says: “In practice, this is difficult because the institutional structure of oil markets impedes the process where the future price is forced to this equilibrium level through arbitrage.” Further, “[o]ne problem is that physical oil is difficult to short.” And participants in the futures market are diverse: “Another is that the participants in the physical and futures market differ. While they use futures to hedge prices, producers, refiners and consumers need the actual oil. Financial investors use the futures market to simply trade price movements.”

In fact, none of these things explain the departure of the oil forward curve from the cash-and-carry (no) arbitrage model, and in particular the existence of backwardation, which is a facial violation of the model.

My teaching mantra is that futures markets price bottlenecks/constraints in transformation processes. And since the forward curve (and thus calendar spreads) provides intertemporal relative prices, the bottleneck in the forward curve relates to transformations in time. And it is a fundamental one: time travel is impossible.

Backwardation signals that the commodity is expected to be more abundant in the future than it is today. If it were possible, we would like to use a time machine to move the commodity from the future to the present. It’s not possible, however. The best we can do is to NOT move the commodity from the present to the future, which would be like carrying coals to Newcastle. That is, backwardation signals that we should consume the commodity today, not add to storage thereby moving the commodity to the future, and in fact draw down on inventories.

Indeed, it is never optimal for inventories of the deliverable underlying a futures contract to always be positive. Instead, sometimes a “stockout” should occur. That’s essentially what we are observing now in WTI, where Cushing stocks are at “tank bottoms.” (If a stockout never occurred, then there would be stuff that is produced yet never consumed. That is suboptimal).

Formal models describe the mechanism. My book is all about such models.

The book (and books and articles by others, including Wright and Williams, and Scheinkman and Schectman) derives forward curves from dynamic programming models that optimize storage and consumption decisions. And in my models. in particular, forward prices are derived from no-arbitrage principles. Namely, to prevent arbitrage, forward prices are expected future spot prices under an equivalent probability measure. So it’s incorrect to say “actual futures prices for commodities don’t follow arbitrage models.”

They don’t follow cash-and-carry arbitrage models, but such models are applicable only to pure assets–things that are always in positive supply like stocks or bonds–and with zero transactions costs. (Positive transactions costs in stocks and bonds can result in deviations from full carry too). And that’s the key point. Commodities are not pure assets. They are consumables, and as noted above, sometimes it is optimal for the commodity underlying a futures contract (e.g., WTI stored in Cushing, Oklahoma) to be in zero supply. The possibility for a stockout breaks the transactions underlying the cash-and-carry model, which implicitly assumes that inventories are always positive.

So all of the stuff Alphaville says differentiate physical from paper (futures) markets are utterly irrelevant in explaining the forward curve, and deviations from full carry. Instead, it’s all about the fact that commodities are storable consumables, not assets analogous to a stock or a bond.

Alphaville really gets into fairy tale territory when it commits the common error of confusing backwardation as the term is used by traders and market participants, and Keynesian backwardation. Backwardation as the term is used in markets means that a nearby price at time T is above a futures price at time T for delivery after T. For example, when the May 2026 price was above the December 2026 price during April.

Keynesian backwardation is when a futures price is above the expected future spot price at the expiration of the futures contract. This is totally different. An expectation is not a traded price. A futures price is a traded price. So Keynesian backwardation compares a tradable price to a non-tradable one. A backwardation backwardation compares a tradeable price to another tradable price. Moreover, whereas backwardation as used in the markets compares, say, a spot price (or nearby futures price) to a futures price with later expiration, Keynesian backwardation compares prices related to the same point in time–the futures expiration date.

Long story short, a market can be in an actual backwardation and not in a Keynesian backwardation, or in a Keynesian backwardation and not an actual backwardation, or different amounts of backwardations. The concepts are not at all related.

Perhaps the best illustration is that stock index futures can be at full carry (adjusted for dividends), and simultaneously in a Keynesian backwardation. In fact, that’s what you’d expect. Futures prices drift up in a Keynesian backwardation, rewarding the long for taking on risk. If the expected rate of return on the index exceeds the risk free rate (as is definitely true), the index will be in a Keynesian backwardation.

Alphaville does get the gist of Keynes’ argument regarding “normal backwardation” correct:

Keynes’ premise was that the participants in the supply chain have asymmetric risk aversion. Refiners, distributors and some consumers might not hedge, as they can pass on price fluctuations. In contrast, producers incur high upfront capital expenditures and bear greater financial risk from lower prices, encouraging hedging. This gives futures prices a downward bias, with producers willing to sell at a discount to the theoretical futures prices.

That is, there is a short hedging imbalance that puts downward pressure on futures prices (relative to expected future spot) to encourage speculators to go long and absorb the imbalance.

The problem with the “downward bias” conclusion that says that hedging imbalance determines the bias in the futures price is that it is based on an archaic view of asset pricing, and arguably an archaic futures market structure. Keynes implicitly assumed futures markets were not integrated with broader financial markets, and also wrote before modern asset pricing theory showed that the risk premium on a given claim depends on its contribution to the risk of some portfolio (e.g., the market portfolio in CAPM), or its correlation with aggregate consumption, or something of the like. In Keynes’ world, speculators in market X were undiversified, and hence their risk exposure and the compensation they demanded for bearing risk was related to the volatility of the futures price. With much demonized “financialization,” however, the marginal market participant is almost certainly well diversified (e.g., a hedge fund) and hence the risk premium he requires to take a position in market X depends not on price volatility but its covariance with his portfolio, or aggregate consumption, or whatever your asset pricing model of choice tells you.

But again, Keynesian backwardation is about risk premiums, and that’s a different issue from what causes an observable forward curve to deviate from the predications of the cash-and-carry arbitrage model.

Alphaville compounds the confusion by mixing up the concept of convenience yields with Keynesian backwardation: “reflecting the fact that producers are prepared to pay a positive insurance premium (the convenience yield) to protect against unforeseen adverse price movements.” Relatedly, it says: “The level of backwardation can be exacerbated by shortages of stock available for immediate delivery from an adverse supply shock — as in the oil market as during the Iran war — or a positive demand shock. This reflects the ‘possession’ value, as refiners and consumers hoard available stocks as a security measure.” This is basically a convenience yield story: the security provides a flow of implicit benefits.

“Convenience yield” is another of my pet peeves. It was a kludge devised by Kaldor to explain deviations from full carry. He reasoned the future price of stocks is less than the future value of the spot price by the amount of dividends. That is, a cash flow on the underlying asset depresses the futures price. Commodities don’t pay a cash flow, but Kaldor argued perhaps they pay some implicit benefit analogous to a dividend that similarly reduces the futures price relative to the spot price and the future value of the spot price (i.e., the spot price plus financing costs).

Very hand-wavy. And there has never been a coherent model that predicts the existence of such an implicit benefit from the optimization decisions of economic agents in equilibrium. In my book, I show that convenience yield only arises when it is essentially assumed (as a kludge) or in models with magical properties (e.g., inventory lowers production costs even if it is never consumed).

So convenience yield is bad enough, but to mix it together with something unrelated to actual backwardation is really ridiculous.

The article does identify some factors that are associated with backwardation. These are factors that feed into structural, dynamic programming models like those in my book. So Alphaville has identified the correlations, but is clueless on the causal mechanism.

What it comes down to is that the extreme backwardation observed during the hot phase of the Iran War is simply explained using the dynamic programming logic. There was a huge adverse short term supply shock. The supply shock was not expected to last forever. That made oil scarce in the short term relative to what was expected in the longer term, when the supply shock dissipated with the end of the war. Under these circumstances, it was optimal to draw down on stocks. The forward curve moved into an extreme backwardation to incentivize this.

This also relates to another issue that has been discussed lately: why is the 2026 oil shock different than the 1979 oil shock in terms of its economic impact? Because the shocks are different. 1979 represented a structural change in the oil market that was expected to last indefinitely. The 2026 shock was expected to be transitory, and indeed that is apparently the case. Old hotness: shortage of oil. New hotness: impending glut. Persistent shocks have very different effects than temporary ones, again as demonstrated in the book.

One last thing. Alphaville starts the piece with a story about Keynes allegedly having to take delivery on wheat futures, and measuring the King’s College Chapel only to learn that it was not large enough to hold the wheat. Keynes told this story, and it is clearly a joke. Delivery on the futures contracts was in store. That is, Keynes received warehouse receipts giving him ownership of wheat in a warehouse. He could have kept it there, as long as he paid storage. No sacrilege required.

But what is revealing is that Saint John Maynard was a welsher: “Keynes cleverly avoided the problem by objecting to taking delivery on quality grounds.” The problem wasn’t finding a place to store the wheat delivered to him. Keynes just didn’t want to take on the obligations (namely paying storage) that delivery entailed. Not clever: commercially disreputable.

And that wheat was ever delivered to Keynes was either the result of his choice or negligence. He could have avoided this problem by selling his futures prior to the delivery period. Did he choose not to do so, then regret the consequences? Or did he have a brain cramp and forget? Either way, he should have manned up, rather than weaseled.