I predicted that the Singleton study would be seized upon as evidence of the malign effects of commodity speculation, and my prediction was borne out. That’s ironic, in a way, because the Singleton paper is really about predictability and forecastability. It finds evidence that excess “returns” on crude oil futures are predictable conditional on measures of speculative activity.
The question is: what explains this predictability? Many of the possible answers to that question–answers that are consistent with Singleton’s results–do not support the view that speculation has distorted prices. Indeed, some of the explanations mean that there is too little speculation.
Predictability can arise when risk premia are forecastable. Speculation involves a transfer of commodity risk, rather than the commodity itself. Based on this fact, you would expect that speculation affects the price of risk, rather than the spot price of the commodity. The price of risk–the risk premium–determines the “drift” of a commodity futures price over time.
Why would the market price of risk–the risk premium–be predictable? More specifically, why would information on speculative positions help forecast the risk premium? In a frictionless capital market, you wouldn’t expect speculative positions to help predict returns. For instance, in a capital asset pricing model (CAPM) world, risk premia depend only on beta, and hence on the covariance between returns on individual investments and the market portfolio (whatever that is, or if you can measure it). (Katherine Dusak’s 1973 JPE paper is the first to apply CAPM to commodity futures.) Other asset pricing models have similar implications.
But in a market with frictions, this no longer holds. This is most readily seen in the Keynesian model of commodity speculation. In that model, speculators hold undiversified positions, and restrict their speculative activity to a single commodity: this implicitly imposes some sort of friction that limits the positions that speculators can hold. Hedgers want to sell futures to hedge inventory. To accommodate this hedging, speculators take on the associated price risk, which they cannot diversify away. So they demand a premium to bear the idiosyncratic risk.
If hedgers want to sell more futures, speculators have to buy more. They demand a larger premium to bear the risk. This may involve attracting more risk averse speculators to the market to bear the additional risk, or requiring the same population of specs to hold more risk. Either way, with this increase in hedging demand (a) the risk premium goes up, and (b) speculative futures positions rise.
This results in a relationship between the risk premium–and hence the trend or drift in futures prices–and the size of speculative positions. Thus, a model in which there are frictions that limit speculative participation in the futures market, and variations in speculative positions are driven by shocks in the demand to hedge (i.e., speculative positions vary to accommodate variations in hedging demand), speculative positions are going to have predictive power, and a rise in speculative long positions predicts a higher rate of “return” on futures.
David Hirshleifer’s work in the 1980s makes Keynes argument more rigorous, and integrates the Keynesian perspective (which predates portfolio theory and modern asset pricing) and asset pricing theory. Hirshleifer assumes that there is a fixed cost of speculating in a commodity futures market. This friction limits but does not eliminate diversification, and in equilibrium, both beta and idiosyncratic risks affect returns. Crucially, however, the association between speculative positions and returns discussed above holds.
Other kinds of frictions can lead to similar results. A recent paper by Duffie and Strulovici posits a different kind of friction, and generates price and return behavior that is inconsistent with asset pricing models that assume frictionless markets for risk.
We should not be surprised that there are frictions in capital markets. Evidence for such frictions abounds. Premium cycles in insurance markets (an example that motivates the Duffie-Strulovici paper) is an example. Insurance premiums rise after insurers suffer big losses even though those past losses have no power to predict future losses. The cycles arise because capital market frictions make it costly for insurers to raise new capital to replace capital paid out after a spate of unexpectedly large losses. Since they have less capital to absorb losses, their capacity to offer insurance goes down, and premiums rise to clear the market.
Indeed, hedging–the supposedly saintly use of futures markets, in contrast to that devilish speculation–only makes sense due to the existence of financial frictions. If capital markets were frictionless, a la Modigliani-Miller, financial policy would be irrelevant, and firms would have no need to manage risks. They could just pass those risks on to investors (purchasers of the firms’ securities) who could manage the risks themselves by forming diversified portfolios. Financial engineering and hedging are valuable only because of financial frictions.
One way of conceptualizing this is that when choosing financial policy/capital structure, firms face a choice between frictions. They incur costs to allocate risk via the securities markets because of frictions in those markets; these frictions could include moral hazard and adverse selection. They incur costs to allocate risks to speculators in the futures markets: these costs arise in part because of frictions that impose costs on speculators, and come in the form of risk premia paid to speculators. Speculators have to raise capital, and they face adverse selection and moral hazard too. Depending on the relative costs, firms will choose how to divvy up their risks.
But the basic point is that when there are frictions that create hedging demand but at the same time make it costly for speculators to take on risk from hedgers, commodity markets and the broader financial markets can be imperfectly integrated. In these circumstances, speculative positions can predict/forecast changes in futures prices.
My research on the pricing of electricity derivatives provides an excellent example of that. When I started this research in the late-1990s, I (and my co-author Martin Jermakyan) documented that the market price of risk in the price of electricity forward contracts was huge. It was on the order of 50 percent of the forward price for on peak delivery during summer months.
This made sense. At the time, participation in the market was limited almost exclusively to producers and consumers of power. Financial/speculative participation was almost completely absent. Moreover, contrary to the situation posited by Keynes, in electricity conditions tend to make long hedging predominant. The distribution of electricity prices is highly right skewed: prices can spike upwards, and impose substantial financial costs on firms that are caught short. This means that firms may be willing to pay a large premium to avoid the financial distress of having to buy power during a price spike.
Case in point. In 1998, Illinois Power’s Clinton plant was down for maintenance. Rather than buying power forward to cover its load obligations, the company bought power on the daily markets. It did so because it believed the forward prices were far higher than it could expect to pay buying spot. (That is, it perceived there was a big risk premium that it didn’t want to pay.) On June 25, 1998, a cascade of events resulted in a huge price spike in power prices in the Midwest. Prices, usually in the $50/MWh range spiked to as high as $7500/MWh. Illinois Power saw an entire year’s earnings wiped out in a single day as it paid these high prices to meet its obligations.
Without speculators, the only way to clear the market when long hedging would greatly predominate at a forward price equal to the expected spot price is for the risk premium–the price bias–to grow, thereby encouraging some companies to sell forward and choking off long hedging demand.
But that premium is also a reward to speculation. And during the 2000s, speculators, including many financial firms, started trading electricity. Electricity became increasingly “financialized.” System operators actually introduced features into their markets that made it easier for financial players to participate. Most notable among these is “virtual” or “convergence” bidding. These allow participants to buy in a forward market (e.g., the day ahead market) and automatically cover in the real time market without making or taking delivery. This mechanism creates cash settled forward contracts that speculators can trade.
The last time I looked closely at this, around 2006, risk premia were still large, but had declined appreciably. In PJM, they had fallen to about 15 percent of the total forward price, as compared to 50 percent less than a decade earlier. This is a clear demonstration of the benefits of speculation in a commodity market. It made hedging substantially cheaper–very substantially. I am not familiar of any other case in which the benefits of speculation can be measured, and turn out to be so large.
The data isn’t available (to my knowledge) to carry out this test, but the empirical implication of the foregoing analysis is straightforward: the positions of speculators in electricity markets should predict the movement in forward prices.
This predictability doesn’t mean that there is too much speculation in the market. If anything, it means there is too little. The predictability reflects the frictions that still impede the transfer of risk.
This explanation for the predictive power of speculative activity for commodity futures returns does not–repeat, not–provide support for measures to restrict speculation. Quite the reverse.
Other explanations, including behavioral or learning-based explanations, may–repeat, may–have different implications. But (a) the existing empirical evidence of predictability does not show that these factors, rather than financial market frictions, are the sources of predictability, and (b) even if they were, it is highly doubtful that commonly proposed measures, such as position limits, would mitigate the price biases they create, or that their effect on behavioral/learning-driven biases would more than compensate for their adverse consequences for risk bearing.
In sum, that measures of speculative activity have predictive power over commodity futures returns does not imply that there is too much speculation, or that speculation has distorted prices. Frictions that prevent complete integration of financial and commodity markets can generate–no pun intended–this result. If such frictions are indeed the source of predictability, policy should seek to reduce constraints on the flow of speculative capital to commodity markets, rather than attempt to increase them.
This means that any finding of predictability should be the beginning of any inquiry, rather than the end. What is crucial is to identify the cause of that finding. I confess that I find financial friction explanations to be the most plausible, in part because of my familiarity with the evolution of electricity markets which provides a very powerful example of how reductions in frictions can erode risk premia, and in part because there is a huge literature that identifies plausible sources of friction and documents their impact on a wide variety of economic behavior. I find other explanations less appealing, but do not reject them out of hand. Policy should be predicated on understanding, and therefore research on speculation in commodity markets should be focused on understanding the sources of predictability, and in particular, seeing if it is possible to distinguish empirically between friction-driven and behavioral-driven predictability.