Streetwise Professor

David at Deus ex Machiatto responded to my response to his post on HFT.  A brief surrebuttal, if I may.

First, regarding CLOBs.  I wrote in the Regulation piece “Thirty Years War” that I linked to in my reply that the SEC had a choice between mandating a CLOB, and an “information and linkages” approach, and chose the latter.  Its implementation of the information-and-links approach was flawed, however, because they didn’t go all the way: the limitation to top-of-book protection is the most obvious flaw.

Moreover, a linkage-based approach is problematic during periods of market stress.  Any tightly coupled system is prone to failure.  David mentions that a CLOB has a single off switch, but that coordinating a shut-down of  multiple linked markets is more difficult.  That’s definitely true.  Failures of linkages–which are prone to happen during periods of high traffic and high volatility–can propagate through a linked system in unpredictable, chaotic, and destructive ways.  All this argues for a CLOB.

But, linkages are inevitable even if all trading in a particular instrument (e.g., a particular stock) takes place on a single CLOB.  In equities, for instance, cash and futures and options and futures markets should be interrelated and interactive, but cannot be run through a single CLOB.  Thus, linkages between such markets are inevitable, these linkages can break, and this tightly coupled system is prone to destructive feedbacks during periods of market stress.  Now with the proliferation of cross-market/asset strategies, linkages are pervasive and there cannot be a true single off switch.

But this is not an artifact of computerization of the markets and HFT.  It was a problem in the Crash of ’87, for instance.  The years immediately following saw a lot of chin pulling about how to coordinate among interrelated markets.

With respect to dark markets, darkness is a mechanism for segmenting privately informed traders and those who are trading for reasons other than private information.  In the old days, with block markets, non-anonymity (counterparty transparency), reputation (“no bagging the street”), and repeat dealing were the mechanisms for doing this.  Similarly, payment for retail order flow and ex post evaluation of trading performance of various sources of order flow was a segmenting mechanism.  As long as you have some people that are trading for reasons other than private information, you will have mechanisms to help them signal their lack of information.  If you force dark markets to light up, you will see the evolution of a mechanism that performs the same functions.  As surely as the light follows the darkness.  Or something.

David and I will agree to disagree about forcing limits on order exposure.  My argument against a say 1 second minimum exposure time is based on the fact that quotes are like free options.  Forcibly extending time-in-force increases the cost of that option to the quoter.  This is especially the case during periods when order flow is toxic–as was clearly the case during the hours leading up to the Flash Crash–that cost can become appreciable.  Many quoters will decide not to quote or radically reduce size.

In other words, the supposed cure is likely to actually exacerbate the problem of liquidity supplier flight which was at the root of the Flash Crash.  Making something more expensive is not the way to get more of it.  Minimum exposure rules would raise dramatically liquidity supply costs during periods when order flow is toxic, so you will get even less liquidity then.

Remember on exchange floors the rule is that a quote is good only “as long as the breath is warm.”  That’s because locals wanted to limit the value of the options they were extending.

There’s no free lunch here.  If you impose a constraint, and the constraint binds–there is a cost associated with that.  Somebody is going to pay it.

The “warm breath” metaphor reminds me that many of the things you see in HFT have close analogies to floor trading.  Sharply limited quote exposure times is one.  Colocation gives HFT traders a time-and-space advantage like that that floor traders possessed.  Even flash trading has its parallels: it is just another time-space advantage for a subset of traders who specialize in liquidity supply.  (Though to the extent that flash trading is used primarily to exploit the lack of full book protection the analogy breaks down.)

I’m sure we haven’t reached the end of this discussion, but that’s all for now.  As the title of my Regulation piece suggested, these battles over market structure and trading rules have gone on a long, long time, and will continue for as long as asset and derivatives markets exist.

Free Exchange, one of the blogs at the Economist writes up one of my takedowns of the SPR release.

Deus ex Machiatto has an interesting post on high frequency trading.   His basic take:

I would argue that HFT has reduced trade sizes and decreased liquidity/increased costs for block trades, especially combined with the move to trading the VWAP rather than brokers taking on blocks as a risk trade, but (3) really gives me pause for thought. Is it true?

. . . .

My recipe for HFT is not to withdraw it, but to reduce the impact of its speed by (1) requiring all market participants to trade on a central order book – no dark pools and (2) requiring all quotes to be good for a minimum of half a second. This would affect real trading activity very little while completely wrecking the high frequency strategies that generate flash crash risk.

I am skeptical that requiring trading on a central order book (CLOB) will eliminate flash crashes.  Note that flash crashes have occurred on futures markets with CLOBs.  The eMini on 6 May 2010 is the most notable example, but even in markets without linkages to other markets–such as cocoa and natural gas–(a) have CLOBs, and (b) have experienced flash crashes.  I’m also skeptical that dark pools have anything to do with flash crashes, and am also not convinced that the dark pool structure–which as DEM notes, has effectively supplanted the block market for facilitating large trades–is detrimental to the interests of those who want to trade in size.  After all, a block positioner could still attempt to compete for the business of those who want to trade in blocks: the fact that dark pools have become the venue of choice suggests they outcompete the old way of trading blocks on cost.

That said, I have long been critical of the SEC rules that protect only the top of the book rather than the depth of book.  Top-of-book protection provides a way for HFTs to make money by exploiting fragmentation of liquidity.  Creating a real CLOB entails a whole set of issues (see the linked article above for a brief discussion), but these can be avoided through the creation of a virtual CLOB that protects the entire books of the various liquidity pools.

A related issue is flash quoting, in which the operator of a matching system gives those supplying liquidity on it a first chance to execute orders before they are routed to other venues.  This undermines time priority, which also precludes the creation of a virtual CLOB by integrating the books of multiple venues.  This can also affects the incentives to supply liquidity.

DEM also links to a presentation that notes that HFTs make money by collecting liquidity rebates.  He further quotes a Bank of Canada study:

Some HFT participants to overload exchanges with trade messaging activity; use their technological advantage to position themselves in front of incoming order flow, making it more difficult for participants to transact at posted prices; or withdraw activity during periods of pricing turbulence.

Both these issues relate to the subject of the very first substantive SWP post: exchange pricing.  That is, the prices that electronic exchanges charge their customers.  I wondered then, and I wonder now, why exchanges don’t charge users for causing congestion on their systems.  If quote stuffing and overloading is problematic, exchanges could penalize it through how they charge for message traffic.  You’d think they have the appropriate incentive to do so.  If it is parasitic on the customers who generate revenues for exchanges (because it is undertaken by those whom exchanges typically pay rebates–see below–and harms the people whom the exchanges charge positive prices), the exchanges would have a strong incentive to devise pricing strategies or quoting rules that would tend to deter this conduct.

Exchange pricing is also relevant for rebates.  It makes sense for exchanges to subsidize some activity (providing quotes) that increases the revenues they can realize from other activity (taking liquidity).  Again, exchanges should have the right incentives to balance the costs and benefits and come up with pricing schemes that encourage the right amount of liquidity supply.

It would certainly be odd, I should add, to want to discourage this kind of pricing if one of your concerns is the disappearance of liquidity.

And related to the disappearance of liquidity, I agree that contributes to crash risk–in both old school floor markets and newfangled electronic ones.  But restrictions on quoting–e.g., DEM’s suggestion that quotes be good for one second–could well be  counterproductive in that regard.  The current evidence suggests that order floor became much more toxic on 6 May 2010, and that’s why HFTs stopped supplying liquidity.  Forcing them to keep their quotes good for longer than they would choose to on their own in such circumstances makes it more likely they will not quote at all.  That is, establishing minimum times that quotes must be in effect is likely to accelerate the wholesale withdrawal of quotes altogether during circumstances in which flash crashes are most likely.

I think there are many issues implicated with HFT–not the least of which being whether there is even an approximate match between the social and private benefits of this activity.  Many of the issues associated with it, however, I think are only peripherally related to flash crashes.  HFT may have a role in flash crashes–though exactly what is hard to pin down, and it may also have a role in making them flashier, i.e., shorter.  But many of the controversies involving HFT don’t appear to be connected to flash crashes.  As in many issues involving financial infrastructure these days, many issues are getting mashed together.  This makes coherent analysis hard, and coherent responses even harder.

Bloomberg reports that much of the oil being sold out of the Strategic Petroleum Reserve is being put right into private storage. Which is exactly what the figures in this post showed would happen.  Indeed, the only thing that is preventing more oil from moving into private storage is the fact that market participants now anticipate that it is more likely that oil will be released in the SPR in the future, thereby reducing the returns to private storage.

I can’t believe those that made the decision to release the oil from the reserve didn’t anticipate this would happen.  Perhaps they didn’t estimate the magnitude properly, but that wouldn’t be surprising given that this depends in large part on how market participants expect the SPR will be used in the future.  If they actually thought all of the oil would be consumed over the next month, thereby causing prices to crater, they have no business making such decisions.

The best part of the article is the bonus description of the inane consequences of the Jones Act, which prevents the use foreign-flag vessels to ship oil between US ports.

In my weekend post on the IEA/US SPR release, I noted that this was likely to fuel speculation, as now market participants have to divine what the powers that be will do with SPR stocks going forward.  As if to prove my point, the deputy executive director of the IEA made a statement so oblique and open-ended that it would do Greenspan or Bernanke proud:

The International Energy Agency could decide by mid-July whether the release of strategic oil reserves needs to be extended for a month or two, an official said.

The 28-member IEA announced last week a plan to release 60 million barrels over an initial 30 days to fill the gap in supplies left by the disruption to Libya’s output.

Richard Jones, deputy executive director of the IEA, said he believed the release would be temporary since demand would likely drop in the fourth quarter.

“We do believe it could be temporary but we have to see how the market evolves. There could be other disruptions, for example, we are compensating for the losses in Libya,” Jones said at an event in Mexico City.

A decision on whether to extend the release could be made around the third week of July, he said.

“It will be up to our member countries, they could decide to continue it for a month or two. I don’t see that we’ll need to continue it for very long because we see demand declining in the fourth quarter, so we think it’s a temporary measure.”

It could be temporary.  But maybe not.  We could decide to continue.  But maybe not.  It will depend on conditions. Could be this.  Could be that.  Am I making myself clear?

As mud, Jonesy.

Way to inject certainty and transparency into the process there, Mr. Jones! How many traders in Geneva, New York, Houston, London, Singapore, etc., are parsing that statement and other Delphic pronouncements coming out of the IEA?  What mental models are they constructing that will use to try to predict what the IEA/US will do, and how the market will react, and how the Saudis will react, and how the IEA/US will react to their reactions, and on and on ad infinitum?   What data/information will they feed into their models?  How will wiggles and jiggles in that data affect oil prices?

Just what the world needs: more would-be Mandarins trying to engineer outcomes in vast, complex markets, and in the process, unleashing the creative energies and analytical skills of people whose knowledge and expertise should be put to far better uses than figuring out what said Mandarins are going to do and what effects it will have.

There may be a case for strategic commodity stockpiles: I’ve suggested one.  (Hint: it’s a way of offsetting the effects of governments’ inability to commit not to interfere stupidly in markets.)  I can state with metaphysical certainty that whatever that case may be, the IEA’s/US’s current action doesn’t meet its criteria.

The more I learn about  the implementation costs and burdens of Frank-n-Dodd, the more monstrous the thing appears.  These burdens will be immense, and affect virtually every company that touches derivatives, regardless of the extent of their involvement.  IT.  Compliance.  Reporting.  And on and on and on.

But Frank-n-Dodd’s chief evangelist, Gary Gensler, has often said not to worry.  The greater transparency created by the law, via the mandating of swap execution facilities and clearing, will drive down execution costs for derivatives.  The savings will offset the burdens of the regulation.  Transparency will make the market more competitive, to the benefit of end users.

Gensler repeated this mantra in an interview in the WSJ:

I think that the core of Dodd-Frank is to bring the benefits of transparency and risk reduction to this marketplace, which has existed for 150 years.Right now, when you enter into bilateral swap transactions, probably most of you have a handful, at most 10, counterparties that you go to. And it might be three or four. And they’re the largest financial institutions.

And I think that openness and competition will come into this marketplace because of transparency, and because for most of the market, it will have to be centrally cleared.

But later he becomes extremely equivocal about these benefits:

MR. FINK:Offstage you mentioned profits of $30 billion that Wall Street makes on derivatives. You figure that would decline to $27 billion?

MR. GENSLER: No, I don’t know what it will decline to. It might go up. What we know is that the derivatives marketplace today is concentrated among large financial institutions.

Dodd-Frank modestly shifts the needle on information because Congress thought that markets are safer and more efficient with that transparency. After the transaction, you will be able to get real-time reporting of transactions in various ways. That’s the new law, called post-trade transparency. Some portion of the market also will have some pretrade transparency, where buyers and sellers meet in platforms.

Those two forms of transparency will modestly shift some of the information advantage. When you do that, you democratize the markets a bit more. The smaller banks might compete for business with some of the larger banks, and so forth.

“[Dealer profits] might go up.”  “Modestly shifts the needle on information.”  “Modestly shift some of the information advantage.”  “Democratize the markets a bit more.”  “The smaller banks might compete for business with some of the larger banks.”  Hardly stirring stuff there.  Pretty limp justification of the competitive benefits of Frank-n-Dodd.  Not exactly the kind of thing that suggests these transparency benefits will offset all of the other massive costs arising from the law.

The title to the WSJ article is “Keep on Hedging.”  Gensler is apparently taking that advice to heart, as he is quite strikingly hedging his advocacy of the cost-reducing effects of improved transparency.

And even these benefits are likely oversold, and perhaps chimerical.  Here’s just a list of some of the reasons to doubt that Dodd-Frank will actually increase competition in a way that reduces end user transaction costs, let alone by enough to overcome the huge regulatory overhead the law creates:

• Post-trade transparency–especially real time post-trade transparency, which the Commission is advocating–need not improve liquidity.  Indeed, it can have the opposite effect.  Liquidity suppliers will be reluctant to trade in size when their trades are exposed immediately after they are made.  This will widen spreads and reduce depth.
• It is not plausible that end users are at a substantial information disadvantage for vanilla products traded in volume, such as vanilla interest rate swaps in major currencies.  They have access to quotes from multiple sources, and can compare quotes for OTC trades to contemporaneous prices for closely related exchange traded products, like Eurodollar futures (for interest rate swaps) or NYMEX futures (for NYMEX lookalike swaps–go figure).
• Spreads on vanilla OTC products are very small.  Margins on these products are razor thin.
• It is well known that institutional traders in many markets prefer to trade on dark markets rather than transparent ones.  This is because their execution costs are lower on these markets.  And “dark market” is a misleading term: as I’ve noted before, there are different kinds of transparency.  Exchange trading, or SEF trading, has pre-trade and post-trade price transparency, but counterparty opacity: the prices are lit, but the counterparties are in the dark.  OTC dealings, or dealings in dark markets, or dealings in block markets, have less price transparency, but more counterparty transparency.  The latter works to the benefit of those who are known to be unlikely to be trading for information-driven reasons.  Forcing trading onto anonymous platforms, especially order driven platforms, raises the trading costs of the verifiably uninformed, which includes many hedgers and end users.  It is categorically false to say that price transparency and counterparty opacity–exchange trading–lowers the trading costs of all liquidity demanders.  Some are made worse off: that’s why they choose to trade off exchange.
• This helps explain why many, many end users oppose the various mandates.  It raises their costs.  They aren’t stupid.  They aren’t victims of battered spouse syndrome.  When they oppose the mandates, they are talking their books.
• Gensler talks like 10 big counterparties is a small number.  Not too many other industries with 10 major competitors.
• And if you believe those 10 firms are colluding, or would like to collude, tacitly on spreads, forcing them to trade through markets with pre-trade transparency is a great way to facilitate such collusion.  Collusion is harder to sustain when buyers and seller strike deals in private.  The colluders can chisel on the agreement through secret price cuts without being caught.  Harder to do when prices are posted publicly: you can see when cheating occurs.  When cheating can be detected, it can be punished, meaning that price transparency can facilitate collusion.  Cf. NASDAQ antitrust case, Stigler’s seminal articles on collusion, the fact that the Justice Department looks with deep suspicion at price sharing arrangements, the fact that public auctions are more susceptible to collusion, etc.
• Order driven markets are prone to tipping to a single dominant platform, which can exercise market power and charge supercompetitive prices.  Surely Genlser is aware that virtually every single futures and futures option contract in the US is dominated by a single exchange.  Mandating exchange-like, order driven markets for vanilla OTC derivatives is likely to result in the same tipping process, and with the charging of supercompetitive trading fees by the winner-takes-all platforms.
• It is almost certain that any benefits arising from compressed spreads–to the extent that they exist, which the foregoing suggests may well not happen–will be distributed very differently from the cost burdens of Dodd-Frank.  As I noted earlier, any firm that is at all involved in derivatives will have to incur substantial costs to comply with the law.  These include IT expenses associated with record keeping, reporting (including the reporting of trades and positions to data repositories), regulatory compliance, oversight, etc.  With regards to regulatory compliance, to achieve any relief from position limits it will be necessary to show that every transaction is a hedge, as opposed to showing that trades are “normally” used to offset price risks.  Similar record keeping burdens will be necessary to demonstrate whether a company is a swap dealer or major swap market participant or not.  It will be necessary to perform all these tasks, and invest in the people and IT overhead needed to perform them, if you trade modest amounts of derivatives or large amounts.  Those who trade smallish amounts will not generate sufficient benefits in terms of lower trading costs–even if those costs materialize, which I doubt–to offset the regulatory overhead.  Some will choose to stay away from derivatives altogether–which may be easier said than done.

In brief: (a) even Gensler is much more guarded in his predictions about the cost-reducing effects of transparency, and (b) for many market participants, transparency and mandated exchange-like trading will actually increase their trading costs.  What’s more, the regulatory burden–you know, the costs that the CFTC adamantly refuses to estimate despite the requirement that it perform a cost-benefit analysis of its rules–is very likely to be crushing: you talk to people in industry, and you realize that they are afraid.  Very afraid.

So remind me again why all this stuff that isn’t remotely related to systemic risk is a good idea.  And only non-masochists may answer, please.

I predicted that the Singleton study would be seized upon as evidence of the malign effects of commodity speculation, and my prediction was borne out. That’s ironic, in a way, because the Singleton paper is really about predictability and forecastability. It finds evidence that excess “returns” on crude oil futures are predictable conditional on measures of speculative activity.

The question is: what explains this predictability? Many of the possible answers to that question–answers that are consistent with Singleton’s results–do not support the view that speculation has distorted prices. Indeed, some of the explanations mean that there is too little speculation.

Predictability can arise when risk premia are forecastable.  Speculation involves a transfer of commodity risk, rather than the commodity itself.   Based on this fact, you would expect that speculation affects the price of risk, rather than the spot price of the commodity.  The price of risk–the risk premium–determines the “drift” of a commodity futures price over time.

Why would the market price of risk–the risk premium–be predictable?  More specifically, why would information on speculative positions help forecast the risk premium?  In a frictionless capital market,  you wouldn’t expect speculative positions to help predict returns.  For instance, in a capital asset pricing model (CAPM) world, risk premia depend only on beta, and hence on the covariance between returns on individual investments and the market portfolio (whatever that is, or if you can measure it).  (Katherine Dusak’s 1973 JPE paper is the first to apply CAPM to commodity futures.) Other asset pricing models have similar implications.

But in a market with frictions, this no longer holds.  This is most readily seen in the Keynesian model of commodity speculation.  In that model, speculators hold undiversified positions, and restrict their speculative activity to a single commodity: this implicitly imposes some sort of friction that limits the positions that speculators can hold.  Hedgers want to sell futures to hedge inventory.  To accommodate this hedging, speculators take on the associated price risk, which they cannot diversify away.  So they demand a premium to bear the idiosyncratic risk.

If hedgers want to sell more futures, speculators have to buy more.  They demand a larger premium to bear the risk.  This may involve attracting more risk averse speculators to the market to bear the additional risk, or requiring the same population of specs to hold more risk.  Either way, with this increase in hedging demand (a) the risk premium goes up, and (b) speculative futures positions rise.

This results in a relationship between the risk premium–and hence the trend or drift in futures prices–and the size of speculative positions.  Thus, a model in which there are frictions that limit speculative participation in the futures market, and variations in speculative positions are driven by shocks in the demand to hedge (i.e., speculative positions vary to accommodate variations in hedging demand), speculative positions are going to have predictive power, and a rise in speculative long positions predicts a higher rate of “return” on futures.

David Hirshleifer’s work in the 1980s makes Keynes argument more rigorous, and integrates the Keynesian perspective (which predates portfolio theory and modern asset pricing) and asset pricing theory.  Hirshleifer assumes that there is a fixed cost of speculating in a commodity futures market.  This friction limits but does not eliminate diversification, and in equilibrium, both beta and idiosyncratic risks affect returns.  Crucially, however, the association between speculative positions and returns discussed above holds.

Other kinds of frictions can lead to similar results.  A recent paper by Duffie and Strulovici posits a different kind of friction, and generates price and return behavior that is inconsistent with asset pricing models that assume frictionless markets for risk.

We should not be surprised that there are frictions in capital markets.  Evidence for such frictions abounds.  Premium cycles in insurance markets (an example that motivates the Duffie-Strulovici paper) is an example.  Insurance premiums rise after insurers suffer big losses even though those past losses have no power to predict future losses.  The cycles arise because capital market frictions make it costly for insurers to raise new capital to replace capital paid out after a spate of unexpectedly large losses. Since they have less capital to absorb losses, their capacity to offer insurance goes down, and premiums rise to clear the market.

Indeed, hedging–the supposedly saintly use of futures markets, in contrast to that devilish speculation–only makes sense due to the existence of financial frictions.  If capital markets were frictionless, a la Modigliani-Miller, financial policy would be irrelevant, and firms would have no need to manage risks.  They could just pass those risks on to investors (purchasers of the firms’ securities) who could manage the risks themselves by forming diversified portfolios.  Financial engineering and hedging are valuable only because of financial frictions.

One way of conceptualizing this is that when choosing financial policy/capital structure, firms face a choice between frictions.  They incur costs to allocate risk via the securities markets because of frictions in those markets; these frictions could include moral hazard and adverse selection.  They incur costs to allocate risks to speculators in the futures markets: these costs arise in part because of frictions that impose costs on speculators, and come in the form of risk premia paid to speculators.  Speculators have to raise capital, and they face adverse selection and moral hazard too.  Depending on the relative costs, firms will choose how to divvy up their risks.

But the basic point is that when there are frictions that create hedging demand but at the same time make it costly for speculators to take on risk from hedgers, commodity markets and the broader financial markets can be imperfectly integrated.  In these circumstances, speculative positions can predict/forecast changes in futures prices.

My research on the pricing of electricity derivatives provides an excellent example of that.  When I started this research in the late-1990s, I (and my co-author Martin Jermakyan) documented that the market price of risk in the price of electricity forward contracts was huge.  It was on the order of 50 percent of the forward price for on peak delivery during summer months.

This made sense.  At the time, participation in the market was limited almost exclusively to producers and consumers of power.  Financial/speculative participation was almost completely absent.  Moreover, contrary to the situation posited by Keynes, in electricity conditions tend to make long hedging predominant.  The distribution of electricity prices is highly right skewed: prices can spike upwards, and impose substantial financial costs on firms that are caught short.  This means that firms may be willing to pay a large premium to avoid the financial distress of having to buy power during a price spike.

Case in point.  In 1998, Illinois Power’s Clinton plant was down for maintenance.  Rather than buying power forward to cover its load obligations, the company bought power on the daily markets.  It did so because it believed the forward prices were far higher than it could expect to pay buying spot.  (That  is, it perceived there was a big risk premium that it didn’t want to pay.)  On June 25, 1998, a cascade of events resulted in a huge price spike in power prices in the Midwest.  Prices, usually in the $50/MWh range spiked to as high as $7500/MWh.  Illinois Power saw an entire year’s earnings wiped out in a single day as it paid these high prices to meet its obligations.

Without speculators, the only way to clear the market when long hedging would greatly predominate at a forward price equal to the expected spot price is for the risk premium–the price bias–to grow, thereby encouraging some companies to sell forward and choking off long hedging demand.

But that premium is also a reward to speculation.  And during the 2000s, speculators, including many financial firms, started trading electricity.  Electricity became increasingly “financialized.”  System operators actually introduced features into their markets that made it easier for financial players to participate.  Most notable among these is “virtual” or “convergence” bidding.   These allow participants to buy in a forward market (e.g., the day ahead market) and automatically cover in the real time market without making or taking delivery.  This mechanism creates cash settled forward contracts that speculators can trade.

The last time I looked closely at this, around 2006, risk premia were still large, but had declined appreciably.  In PJM, they had fallen to about 15 percent of the total forward price, as compared to 50 percent less than a decade earlier.  This is a clear demonstration of the benefits of speculation in a commodity market.  It made hedging substantially cheaper–very substantially.  I am not familiar of any other case in which the benefits of speculation can be measured, and turn out to be so large.

The data isn’t available (to my knowledge) to carry out this test, but the empirical implication of the foregoing analysis is straightforward: the positions of speculators in electricity markets should predict the movement in forward prices.

This predictability doesn’t mean that there is too much speculation in the market.  If anything, it means there is too little.  The predictability reflects the frictions that still impede the transfer of risk.

This explanation for the predictive power of speculative activity for commodity futures returns does not–repeat, not–provide support for measures to restrict speculation.  Quite the reverse.

Other explanations, including behavioral or learning-based explanations, may–repeat, may–have different implications.  But (a) the existing empirical evidence of predictability does not show that these factors, rather than financial market frictions, are the sources of predictability, and (b) even if they were, it is highly doubtful that commonly proposed measures, such as position limits, would mitigate the price biases they create, or that their effect on behavioral/learning-driven biases would more than compensate for their adverse consequences for risk bearing.

In sum, that measures of speculative activity have predictive power over commodity futures returns does not imply that there is too much speculation, or that speculation has distorted prices.  Frictions that prevent complete integration of financial and commodity markets can generate–no pun intended–this result.   If such frictions are indeed the source of predictability, policy should seek to reduce constraints on the flow of speculative capital to commodity markets, rather than attempt to increase them.

This means that any finding of predictability should be the beginning of any inquiry, rather than the end.  What is crucial is to identify the cause of that finding.  I confess that I find financial friction explanations to be the most plausible, in part because of my familiarity with the evolution of electricity markets which provides a very powerful example of how reductions in frictions can erode risk premia, and in part because there is a huge literature that identifies plausible sources of friction and documents their impact on a wide variety of economic behavior.  I find other explanations less appealing, but do not reject them out of hand.  Policy should be predicated on understanding, and therefore research on speculation in commodity markets should be focused on understanding the sources of predictability, and in particular, seeing if it is possible to distinguish empirically between friction-driven and behavioral-driven predictability.

The IEA and the United States have announced plans to release over the next month 60 million barrels of crude oil from strategic stockpiles, including the US’s Strategic Petroleum Reserve.   Oil prices plunged, by about $5/barrel.  Initially, I was reluctant to attribute most of the price drop to the announcement, because the stock market and virtually all other commodities were down hard too, and the dollar rallied.  These broad changes were plausibly due to bad US job numbers and continued angst about Greece, and given the recent correlations between oil and stocks and the dollar, it was likely that a good part of the oil sell-off was related to these factors.  But when the stock market rebounded later in the day, recovering about 3/4ths of its previous losses, and oil only recovered slightly, it was evident that the oil stockpile release had caused a substantial drop in prices.

To understand the implications of the release, assume initially that the release is considered a one-off, with no implications for the future. In that case, it is like a sudden increase in initial availability of oil. Here is a figure based on a dynamic model of a storage economy that relates the amount of inventory carried out (on the vertical axis) to the amount of the commodity initially available (on the horizontal axis).*

A surprise increase in initial availability is a move along the horizontal axis. Since carry-out is an increasing function of initial availability, some of that new supply released from the reserve is added to private inventories. This would tend to dampen the effect of the release on prices, but some of the increased supply would be consumed, prices would fall. Here is a figure of price as a function of initial availability, which shows that as availability rises price falls. But this curve is flatter–more elastic–than the flow demand curve, because of the fact that some increases of initial supply are put into inventory which buffers the impact on price.

But the key thing here is that you can’t assume that the market believes that this will be a one-off, with no implications for the future. The availability-carry-out function depends crucially on the beliefs of market participants about how the SPR will be used going forward. The current use is unprecedented: previously, the reserve was used in extraordinary circumstances, like supply disruptions resulting from the 2005 hurricanes. Now it is being used to micromanage world oil prices–and to micromanage Obama’s political future. This dramatic change in the employment of the SPR will inevitably affect beliefs, and hence (a) the relation between carry-out and initial availability, and (b) price and initial availability.

In my opinion, the most likely outcome is for the change to affect beliefs in a way that leads to a bigger price response than implied by the two earlier figures. The reasoning goes like this. Those making decisions on how much oil to store do so in anticipation of earning a profit by selling out of those inventories when demand spikes up or supplies spike down. These private storers will now reason that there is a higher likelihood that supplies will be released from SPR under those circumstances. This reduces the amount of money they make on selling out of inventory during those condition. This makes holding private inventories less profitable, so they will hold smaller stocks, all else equal. Put differently, market participants will now view the SPR as a competitor for private storage under a wider range of economic circumstances than was previously the case, and this increased competition from public storage will drive out some private storage.

In terms of the graph, this shift in beliefs about the way the SPR will be used shifts the relation between availability and carry-out. In particular, it shifts it down, from the green curve to the blue one.

This shift moves exacerbates the price impact of the release. It means that a smaller amount of the release will be absorbed into private storage, more of it will be consumed, and hence prices will fall by a larger amount.

Going forward, private inventories will be smaller. This will make the market more reliant on public storage to smooth supply and demand shocks. That’s not comforting, because public storage decisions are not driven by commercial and market realities, but political ones, and by decision makers with poorer information and weaker incentives than commercial market participants.

Ironically, one hypothesis advanced to explain the decision is that it is designed to punish long speculators. Well, the government and the IEA have now just provided much speculative fodder: now market participants have to speculate about how SPR management policy has changed, and how it will change going forward. That is very complicated, given that it will be buffeted by numerous factors. These include how OPEC countries react, how the US reacts to the OPEC countries’ decisions, how it plays politically in the US, and on and on.

This uncertainty, the flow of information relevant to deciphering the policy shift, and the feedback mechanism among traders (e.g., the Bayesian learning dynamic mentioned in the Singleton paper) will all contribute to price volatility. In short, by attempting to punish speculation, the administration has only stoked speculation. The “constructive ambiguity” surrounding the release, and future reserve policies, will only further fuel such speculation. SPR policy will now attract the same kind of scrutiny as Federal Reserve policy, where market participants try to interpret Delphic announcements and actions in order to discern the future course of policy. They will try to infer how external events–say, Obama’s political standing–will drive policy: just what is the function that relates SPR releases to Obama’s poll numbers? All of this interpretation and inference will generate trading that will in turn generate price movements, just as is the case with respect to Fed policy. Alleged concern about volatility has led to a policy that will create volatility.

And note that even though this action has hurt some speculators–those who are long–it has been a huge windfall to others.

The numbers are pretty staggering. Based on Commitment of Trader Reports for 21 June, and just looking at NYMEX and ICE WTI and Brent crude oil futures, given a price impact of about $4/barrel, the announcement led to a shift of wealth from longs to shorts of about $17 billion in crude futures and futures options alone. Add in refined products and you’ll increase that more. Add in swaps and other OTC instruments, you will increase that number substantially. Very substantially.

Within categories, again assuming a $4 price impact and looking only at crude futures and futures options, long swap dealers lost $1.9 billion, short swap dealers made $1.7 billion; long managed money lost $1.5 million and short managed money made $460 million, other reporting long speculators lost $556 million and other reporting shorts made $380 million. Again this overlooks the transfers between longs and shorts in the swap market. And also the capital losses on unhedged private inventories; at the end of May, OECD commercial crude and product stocks totaled about 2.7 million billion barrels. US commercial stocks of crude are about 360 million barrels, and crude and product stocks (ex SPR) about 1 billion barrels. Thus, the value of inventories in private hands in the OECD fell by about $10 billion, and in the US about $4 billion–who knows how much inventories fell in value in China. (And by the way, the value of oil in the SPR fell by about $280 million$2.8 billion.)

A lot of money changing hands. A lot.

Earlier this year, I blogged about potential economic justifications for a strategic petroleum reserve, and how the reserve should be used based on such justifications. In brief, something like the SPR can be justified to correct some other market failure that would depress private storage below its optimal level: the most likely candidate for such a market failure would in fact be a government failure, such as the threat of price controls or other interventions during a crisis.

Suffice it to say that the current action cannot be justified in this way. There is no demonstrable market failure being ameliorated here. This use of public storage is not correcting some deficiency in private storage arising from some market failure or government failure. Indeed, perversely, this use of the SPR’s public storage will, as noted above, discourage private storage.

In US law, there are three elements to proving manipulation: causation, intent, and artificial price. It is clear logically and empirically that the SPR release did cause prices to move. It is also abundantly clear that the administration and the IEA had the specific intent to cause price changes: indeed, they are both boasting about the purpose and effect of the policy. Artificial price is somewhat more ambiguous here. No market failure is being corrected, so a colorable case can be made that the price movement is not moving price closer to where it should be in the absence of such a market failure. That supports a claim of artificiality. But given that the SPR might have distorted price in the first place, it is arguable that perhaps prices are now closer to where they “should” be–but that only means that the price was artificial before because the SPR was inefficiently holding oil off the market.

Causation and particularly intent are often the hardest thing to show in a manipulation case: here, in contrast, they are easily proven. Artificiality is not so clear cut here, but I think it is beyond cavil that the impact of the policy will be to enhance volatility in oil prices and lead to fluctuations based on conjectures about the future course of government policy. That interferes with the operation of these markets and will lead to misallocations of resources, which is what artificial prices do, and what the manipulation laws are intended to combat.

This leads me to conclude that there is a strong prima facie case of manipulation: certainly a stronger case than some others the CFTC has filed in the past. I therefore expect that the agency will move swiftly to file a manipulation action.

That was all tongue in cheek, for those slow on the uptake. I of course know that no such action will be forthcoming; the government has vast discretion in use of the SPR. But if you look at the perverse effects of manipulation by commercial and speculative players–massive transfers of wealth between market participants, distortions of consumption and production decisions by private players, unnecessary price volatility–all of them are present in spades here. Indeed, the effects here are huge, far larger than any private manipulation case that I am familiar with (and I am familiar with all of them, I think).

It is especially ironic that this move came during the same week that the Federal Trade Commission announced that it was launching the most recent in a continuing series of investigations of manipulation by oil companies. This happens every time prices are high, with the same result: the FTC finds nothing. It’s one of the longest running farces in Washington.

If it wants to find manipulation, it should restrict its investigation to the 202 area code.

This is not the first time, certainly, that the US government has attempted to intervene in commodity markets to control prices in order to achieve a political objective. The Hoover administration–you know, that laissez faire bunch (not!)–did so with abandon in the grain markets with the onset of the Depression. That turned out badly. I don’t expect this to turn out much better.

* This is the same model as I analyze extensively in my forthcoming book. Just received the galleys, so the end is in sight!

The World Bank has midwifed a deal to facilitate hedging of agricultural prices in developing nations.  The bank has committed to sleeve the credit risk for “simple” hedging trades that JP Morgan does with private businesses like farm coops and ag processors in developing countries.

In theory this is a great idea–though beware, it will necessarily require more evil speculators to enter the market to bear the risk!  But I have my doubts about how things are going to work out in practice.

My doubts arise from the fact that although hedges should be evaluated ex ante, they are all too often criticized ex post.  That is, whether a hedge makes sense should be determined before you see how prices move: after all, the point of a hedge is to reduce exposure to price risk, protecting against the downside by giving up the upside.  But it is all to common, and perhaps all to human, to evaluate hedges after one has seen how prices moved in fact.

Roughly speaking, 50 percent of the time you will be happy with the results of a hedge, but 50 percent of the time you will have hedger’s remorse: you will find that you would have made more money if you hadn’t hedged.  For instance, if you are an airline and you buy fuel forward as a hedge, you are quite happy with your decision if fuel prices indeed rise, for you have locked in a lower price.  But if fuel prices fall, you will have committed to buy at a price than is currently available in the marketplace.  D’OH!

But if you had perfect foresight, you wouldn’t need to hedge: you’d be a very rich speculator.  If uncertainty about future prices is costly to bear, you should be happy to reduce your exposure to that risk, and shouldn’t look back and say shoulda, coulda, woulda.

However, in the real world, managers, investors and others second guess when hedges end up losing money.  If you want a great example–or terrible example, depending on your perspective–look at all those municipalities in Europe and the US who entered into derivatives trades that protected them against higher interest rates.  They have squealed loudly in the aftermath of dramatic falls in interest rates that put those hedging contracts underwater.  They claim they were duped, and have attempted to escape their contractual obligations: indeed, they have sometimes succeeded.

The same thing is likely to happen with the World Bank facilitated ag hedges.  Sometime in the future, the hedgers are going to lose money on their hedges.  Retrospectively, they will regret their decision to hedge.  Being human, they will attempt to escape their obligations.  Given the political sensitivities associated with a large financial institution–JP Morgan–and an international organization domiciled in DC and funded and strongly influenced by western governments and institutions, it doesn’t take much imagination to write in your head the advertising spots bewailing the victimization of unsophisticated DM tillers of the soil by nefarious Wall Street and City sharpies.

Thus, the $200 million the World Bank is committing may really be an option premium, as the protection could well be one-sided, with the hedgers collecting when their trades are in the money, and finding ways to avoid payment when they are out.

Even if the WB is effectively underwriting free options (or, at least, subsidized ones), that may be a good idea nonetheless.  Especially given the financial fragility of many companies in developing nations (and remember that financial imperfections is the reason that hedging makes sense), and the political ramifications (which can include violence) of sharp changes in the price of food products in such areas, a subsidized risk transfer is likely welfare improving.  But have no illusions as to how this program might play out in practice.

The World Bank initiative is certainly a damn sight better than Sarkozy’s plans to “rebalance the structure of capitalism” (I s*** you not–that’s what he said) by clamping down on food price volatility by limiting speculation.   Which brings to mind what Adam Smith said about the subject 235 years ago:

The laws concerning corn [i.e., grain] may every-where be compared to the laws concerning religion. The people feel themselves so much interested in what relates either of their subsistence in this life, or to their happiness in a life to come, that government must yield to their prejudices, and, in order to preserve the public tranquillity, establish that system which they approve of. It is upon this account, perhaps, that we so seldom find a reasonable system established with regard to either of those two capital objects.

. . . .

The popular fear of engrossing and forestalling [commodity speculation] may be compared to the popular terrors and suspicions of witchcraft. The unfortunate wretches accused of this latter crime were not more innocent of the misfortunes imputed to them, than those who have been accused of the former. The law which put an end to all prosecutions against witchcraft, which put it out of any man’s power to gratify his own malice by accusing his neighbour of that imaginary crime, seems effectually to have put an end to those fears and suspicions, by taking away the great cause which encouraged and supported them.

But then Smith was just another Anglo-Saxon, rather than a dirigiste Frenchman.  (Not really, and any Scot would take extreme umbrage at the suggestion!  And as many of Sarkozy’s opponents in France never tire of pointing out–he’s not French!)

Note the fundamental contrast in approaches.  The World Bank is attempting to facilitate the efficient transfer of risk–transfers that are impeded by contracting frictions and lack of knowledge in developing markets.  Sarkozy is attempting to impede the voluntary transfer of risk.  The World Bank is attempting to palliate some of the consequences of the underdevelopment of capitalism in poor countries–consequences which include the inability to enforce efficient risk sharing contracts.  Sarkozy is engaged in some grandiose scheme to remake capitalism.

Yeah, that always works out so well, Nick.

Apparently “orthodox” is the sneer du jour.  A couple of weeks back I noted that IMF candidate Agustin Carstens was being denigrated as orthodox and therefore unfit to head the IMF.  And today, in a story about the Singleton oil market study I blogged about last week, Reuter’s John Kemp dismissed me, Scott Irwin, and Jeff Harris as a cohort of orthodox economists who dismiss the role of financial speculators in affecting commodity prices.

I’m only PO’d because Kemp listed Irwin first.  That wounds me to the quick.

Apparently I’m a member of an “establishment”–when do I get promoted to mastermind of an evil cabal?–that Svengali like, induced the CFTC, IOSCO, the FSA, the OECD, and presumably the IEA (though Kemp doesn’t mention it) to conclude that speculation had not distorted energy prices, or the prices of other commodities.

Who knew the vast influence I wield over official bodies world wide? I certainly didn’t.

Regarding orthodoxy, my views are certainly not considered orthodox, on say, Capitol Hill.  Or on the Champs Elysee.  Or in Brussels.

No, “orthodox” is shorthand for People Who Don’t Agree With John Kemp, whereas those who do are Brave Challengers of the Consensus.

Now to the substance of what Kemp has to say, such as it is.

First, as I made clear in my post on Singleton’s piece last week, I do not deny the possibility that speculation has “influenced prices,” as Kemp insinuates.  In fact, such a contention would be absurd, and Kemp is absurd for insinuating it.  What I have been on about is whether speculation has distorted prices.

Big difference, John.

There have been specific allegations that speculation has led to substantial distortions in commodity prices, particularly energy prices–by 10, 20, hell 50 percent.  That is what I have been responding to, and I stand by my contention that there is no evidence of such an effect.

Nor does anything in Singleton’s piece support such claims.

Variations in measures of speculation can have predictive power over returns–that is, they can influence prices–without implying that speculation has distorted them.  As I noted in my post on Singleton’s piece, limits to arbitrage-type stories predict that shocks to speculator balance sheets or hedger balance sheets can lead to associations between speculative activity and the drift of commodity prices.  This is perfectly consistent with a refinement of Keynes’s normal backwardation theory that Kemp identifies with the dreaded orthodoxy.  (Truth be told, I made an argument along these lines in a presentation at the FMAs in October of 2002.  And I’ve made it year after year in my PhD seminar on derivatives, in my discussions of incomplete markets.)

Insofar as the behavioral stories are concerned, yes,  they can also imply that variation in speculative activity have predictive power over returns.  They also can predict boom and bust cycles.  But it is notoriously difficult to determine whether these sorts of behavioral effects are what explains the predictive power of measures of speculation.

I would also note that purely rational storage models can also generate booms and busts.  Indeed, a rational storage model can match the behavior and time series properties of prices and inventories–I show that in my forthcoming book.  This is true for both pre-2004 and post-2004.  It is just flat wrong to say that the price and quantity movements observed in recent years cannot be reconciled with rational, fundamentals-driven models.  So the existence of commodity booms and busts–which way, way antedate the financialization of commodity markets–does not refute fundamentals-based explanations, or prove the existence of behaviorally driven anomalies.

Singleton points out–and Kemp emphasizes–a series of theoretical possibilities and some evidence that is consistent with theories that generate these possibilities–but which are also consistent with other explanations.  A very weak basis indeed to justify wholesale intervention in the markets.  Especially given the very vital role that speculation plays in achieving an efficient allocation of risk.

The behavioral theories are also not sufficiently well-developed to generate testable predictions about co-movements between prices and quantities and the dynamics of forward curves.  Predictions about quantities are particularly important.  As I’ve stated on numerous occasions, commodities are consumed in the here and now (in contrast to say, internet stocks), so distortions in prices should show up in distortions in quantities (notably inventories).  Do behavioral models predict this?  I haven’t seen any that have even tried.  If they do–the evidence doesn’t support it.  If they don’t–then why should we really care, as this would mean that speculation is not distorting consumption and production decisions?

And how can Kemp seriously say that Singleton’s paper “offers a richer and more realistic view of how real markets operate than the rather stylized formulations that have been popular with the old guard”?  [And who are you call old, boy?]  Has he ever looked at any of these behavioral models?  You want to see stylized?  Case in point–the Hong-Yogo model, which reverse engineers an extremely contrived–actually, artificial–behavioral setup that mimics the empirical result already produced.  Sorry, but that’s not science: model first, then test.  Moreover, as I just noted, these behavioral models don’t make any predictions about quantities, forward curves, and other crucial aspects of commodity markets.  How can you possibly consider models that don’t make predictions about such crucial variables “richer” and “more realistic”?  Especially given that these quantity predictions are what is really relevant in determining the welfare effect of speculation.

Contrast that to my work on commodities.  I take a rigorous model, derive (computationally) its implications for prices, forward curves, quantities, quantity-price comovements, volatilities, and forward price correlations, and take those predictions to the data.  I reject many of the implications of the models, and use those rejections to help understand the true richness of how prices behave.  If I’m orthodox–that’s the sense in which I am orthodox.  I’m not wedded to the models.  As I say in the introduction of the book, I look to break the models and to learn from the pieces.

And as I noted in my post on Singleton’s paper, the policy implications of the behavioral theories are ambiguous–and certainly don’t support a role for position limits on large speculators.  As I noted, if you overlook the cheesy model and believe the Hong-Yogo behavioral interpretation of their empirical evidence, behavioral effects have long been present in these markets–long before financialization.  What’s more, the behavioral models suggest that prices depend crucially on the composition of traders in the marketplace.  Policymakers can’t micromanage that composition to achieve more “rational” pricing through blunt tools like position limits.

So as I predicted in my Singleton post, people would seize on his study to advance their regulatory agenda, even though it provides only flimsy, equivocal support for that agenda.   Not surprisingly, John Kemp is leading that parade, giving further proof to  the aphorism that a little knowledge is a dangerous thing.