Luddism in the Oil Futures Markets
The old, old game of Pin the Tail on the Speculator has been updated. According to Bloomberg, the speculators who now “disrupt” the oil markets are not human: they are bots. Specifically, bots operated by Commodity Trading Advisors (CTAs).
This argument consists of two parts. The first being that the crude oil futures markets have been disrupted. The second being that the CTAs are the cats behind the disruption. Both plinths are defective.
Insofar as disruption is concerned, Bloomberg claims “Trading oil has perhaps never been more of a roller coaster ride than it is today.” Further:
Just in the past two months, prices threatened to reach $100 per barrel, only to whipsaw into the $70s. On one day in October, they swung as much as 6%. And so far in 2023, futures have lurched by more than $2 a day 161 times, a massive jump from previous years.
Bloomberg
Never more of a roller coaster ride? Well, let’s do something crazy. Like look at historical data.
The conventional measure of the wildness of the ride is volatility. The annualized daily volatility of crude oil during the alleged Rule of the Bots (the last two years) is 41.62 percent. The historical volatility (2010-2020) is 41.2 percent. (This omits 4/20/20 and 4/21/20, the day of the negative oil price and the following day.) Excluding the COVID months of 2020 produces a somewhat lower vol of 36 percent, not that much smaller than in the last two years. Further, extended excursions of realized volatility to above 40 percent are not unusual in the historical record. So to say that oil prices have been more volatile recently than has historically the case is categorically false.
With respect to the big daily moves, the Bloomberg analysis is fatally flawed because it looks at dollar price moves: big dollar price moves are more likely when prices are high than they are low, and by historical standards oil prices have been high in the last several years. It is appropriate instead to focus on percentage price changes (which is how vols are calculated, btw).
Rather than count the number of times an arbitrary threshold (like $2/bbl) is breached, it is more rigorous to look at a statistical measure of the frequency of extreme events: the “kurtosis.” Kurtosis bigger than zero means a distribution has fat tails relative to a Gaussian (“normal”) distribution, i.e., extreme moves up or down are more likely than under a normal distribution. The bigger the kurtosis, the more likely extreme moves are, i.e., the fatter the tails of the distribution.
Looking at the kurtosis of daily percentage changes rubbishes the Bloomberg analysis. The kurtosis in the last two years is 4.15, whereas from 2010-2020 it was 27.9! That is, the frequency of extreme daily price moves in years of alleged CTA disruption is far, far smaller than was the case prior to their alleged emergence as the dominant force in the. markets.
Interestingly, the kurtosis of dollar price changes is not that different between eras: 6.9 post-2020 vs. 7.2 2010-2020. So even extreme dollar price moves are less frequent in the alleged CAT era than previously. The difference is smaller, which demonstrates the need to take into account the level of prices in an analysis of “extremes.”
So the predicate for the article–that oil prices have been unusually volatile and unusually susceptible to extreme moves in the past couple of years–is not supported by the data.
As for the alleged causal factors, the dominance of CTAs is not evident in the data. CTAs are included in the “Managed Money” category of the CFTC’s Commitment of Traders Report. Here is a graph of the net position of Managed Money going back to 2006:

There was a peak in 2017-2018–a drilling boom in the US, to which I will return shortly–followed by a decline–a drilling drought–followed by a rebound to levels comparable to the 2017-2018 levels. Indeed, managed money net positions have actually been relatively low in the past year (with the exception of a recent spike) as compared to the post-2015 period as a whole. Certainly no Alice to the moon spike in CTA presence apparent here.
Bloomberg claims that the CTAs have become dominant in large part due to a sharp decline in producer hedging:
That coincided with the collapse of another source of futures and options trading: oil-production hedging. During the heyday of shale expansion about a decade ago, drillers would lock in futures prices to help fund their growth. But in the aftermath of the pandemic-induced price crash, a chastened US oil industry increasingly focused on returning cash to investors and eschewed hedging, which can often limit a company’s exposure to the upside in a rising market. By the first quarter of this year, the volume of oil that US producers were hedging by using derivatives contracts had fallen by more than two-thirds compared with before the pandemic, according to BloombergNEF data.
It should be noted that this claim that CTAs have achieved greater dominance due to an ebbing of hedging is implausible on its face. Futures are in zero net supply. If producers have reduced their net positions, necessarily non-hedgers–including CTAs–must have reduced their net positions.
Hedging has indeed declined. In the oil market, much (if not most) producer hedging is via the swaps market rather than direct producer participation in the futures market. Banks buy swaps from producers, and then hedge their exposure by selling futures. Here is a chart of net Producer and Merchant Plus Swap Dealer exposure from the CFTC COT data:

Note that there was a big increase in hedging activity (by this measure) in 2017-2018 that was reversed, followed by a partial resurgence, but in the last couple of years hedging activity has indeed ebbed, and reverted to its 2016 levels.
But note that this pattern of hedging mirrors closely Managed Money net positions. As is necessarily the case. If there is less hedging, speculators necessarily hold smaller positions. Meaning that this statement is nonsensical:
The recent wave of dealmaking by US oil producers threatens to further accelerate the decline in hedging. And it’s highly likely that CTAs will continue to fill the vacuum left by those traditional market players.
It’s not as if CTAs–or speculators generally–are “fill[ing] a vacuum.” If hedgers reduce positions, speculators do too.
The Bloomberg writers may dimly glimpse the truth, though they don’t realize it.
How did CTAs come to become so dominant? Like many current phenomena, the answer starts in the depths of the pandemic.
As shutdowns engulfed the world in 2020, fuel consumption collapsed by more than a quarter. All hell broke loose in the crude market. The benchmark US oil price briefly dropped to minus $40 a barrel and investors were in wholly new territory. Some funds that took longer-term views based on supply-and-demand fundamentals quickly pulled out.
Such bear markets proved to be “extinction events” for traditional funds, which made way “for algo supremacy,” the bulk of which are CTAs, said Daniel Ghali, senior commodity strategist at TD Securities. Russia’s invasion of Ukraine gave the CTAs another foothold. Spiking volatility in the futures market drove many remaining traditional investors to the exits, and open interest in the main oil contracts tumbled to a six-year low.
So if CTAs have indeed become more prevalent, it is because they have supplanted other speculators who exited the market. Futures are risk transfer markets. If some of those who previously took on the risk from hedgers have exited the market, either hedgers must hedge less or other speculators must step in. It seems that both things have been happening.
That’s not some ominous development–it’s markets at work. And CTAs shouldn’t be damned–they should be praised for stepping into the breach.
And another paragraph in the Bloomberg article suggests at what is actually happening here:
The unpredictability of this year’s market swings haven’t been kind to human traders, many of whom are making less money on oil than they did last year when they raked in record gains, according to market participants.
What is likely driving this story is whinging by the traditional specs, who have been outcompeted by the bots. “No fair! They are making money and I’m not! They must be cheating.”
Reminds me of my epigram from my manipulation book, where riffing on Ambrose Bierce’s Devil’s Dictionary I wrote something to the effect that a market is manipulated when it moves against me.
Again, this is markets at work. The fact that bots are doing well relatively to trad specs means that they are better at predicting market movements, or have lower costs of bearing risk, or both.
It does not mean that they are making the markets move.
So this Terminator Tackles the Oil Market narrative is really nothing more than Luddism. A new technology outcompetes the old. The incumbents complain. End of story.