Streetwise Professor

April 24, 2015

A Matter of Magnitudes: Making Matterhorn Out of a Molehill

Filed under: Derivatives,Economics,HFT,Politics,Regulation — The Professor @ 10:47 am

The CFTC released its civil complaint in the Sarao case yesterday, along with the affidavit of Cal-Berkeley’s Terrence Hendershott. Hendershott’s report makes for startling reading. Rather than supporting the lurid claims that Sarao’s actions had a large impact on E Mini prices, and indeed contributed to the Flash Crash, the very small price impacts that Hendershott quantifies undermine these claims.

In one analysis, Hendershott calculates the average return in a five second interval following the observation of an order book imbalance. (I have problems with this analysis because it aggregates all orders up to 10 price levels on each side of the book, rather than focusing on away-from-the market orders, but leave that aside for a moment.) For the biggest order imbalances-over 3000 contracts on the sell side, over 5000 on the buy side-the return impact is on the order of .06 basis points. Point zero six basis points. A basis point is one-one-hundredth of a percent, so we are talking about 6 ten-thousandths of one percent. On the day of the Flash Crash, the E Mini was trading around 1165. A .06 basis point return impact therefore translates into a price impact of .007, which is one-thirty-fifth of a tick. And that’s the biggest impact, mind you.

To put the comparison another way, during the Flash Crash, prices plunged about 9 percent, that is, 900 basis points. Hendershott’s biggest measured impact is therefore 4 orders of magnitude smaller than the size of the Crash.

This analysis does not take into account the overall cumulative impact of the entry of an away-from-the market order, nor does it account for the fact that orders can affect prices, prices can affect orders, and orders can affect orders. To address these issues, Hendershott carried out a vector autoregression (VAR) analysis. He estimates the cumulative impact of an order at levels 4-7 of the book, accounting for direct and indirect impacts, through an examination of the impulse response function (IRF) generated by the estimated VAR.* He estimates that the entry of a limit order to sell 1000 contracts at levels 4-7 “has a price impact of roughly .3 basis points.”

Point 3 basis points. Three one-thousandths of one percent. Given a price of 1165, this is a price impact of .035, or about one-seventh of a tick.

Note further that the DOJ, the CFTC, and Hendershott all state that Sarao see-sawed back and forth, turning the algorithm on and off, and that turning off the algorithm caused prices to rebound by approximately the same amount as turning it on caused prices to fall. So, as I conjectured originally, his activity-even based on the government’s theory and evidence-did not bias prices upwards or downwards systematically.

This is directly contrary to the consistent insinuation throughout the criminal and civil complaints that Sarao was driving down prices. For example, the criminal complaint states that during the period of time that Sarao was using the algorithm “the E-Mini price fell by 361 [price] basis points” (which corresponds to a negative return of about 31 basis points). This is two orders of magnitude bigger than the impact calculated based on Hendershott’s .3 return basis point estimate even assuming that the algorithm was working only one way during this interval.

Further, Sarao was buying and selling in about equal quantities. So based on the theory and evidence advanced by the government, Sarao was causing oscillations in the price of a magnitude of a fraction of a tick, even though the complaints repeatedly suggest his algorithm depressed prices. To the extent he made money, he was making it by trading large volumes and earning a small profit on each trade that he might have enhanced slightly by layering, not by having a big unidirectional impact on prices as the government alleges.

The small magnitudes are a big deal, given the way the complaints are written, in particular the insinuations that Sarao helped cause the Flash Crash. The magnitudes of market price movements dwarf the impacts that the CFTC’s own outside expert calculates. And the small magnitudes raise serious questions about the propriety of bringing such serious charges.

Hendershott repeatedly says his results are “statistically significant.” Maybe he should read Deirdre McCloskey’s evisceration of the Cult of Statistical Significance. It’s economic significance that matters, and his results are economically miniscule, compared to the impact alleged. Hendershott has a huge sample size, which can make even trivial economic impacts statistically significant. But it is the economic significance that is relevant. On this, Hendershott is completely silent.

The CFTC complaint has a section labeled “Example of the Layering Algorithm Causing an Artificial Price.” I read with interest, looking for, you know, actual evidence and stuff. There was none. Zero. Zip. There is no analysis of the market price at all. None! This is a piece of the other assertions of price artificiality, including most notably the effect of the activity on the Flash Crash: a series of conclusory statements either backed by no evidence, or evidence (in the form of the Hendershott affidavit) that demonstrates how laughable the assertions are.

CFTC enforcement routinely whines at the burdens it faces proving artificiality, causation and intent in a manipulation case. Here they have taken on a huge burden and are running a serious risk of getting hammered in court. I’ve already addressed the artificiality issue, so consider causation for a moment. If CFTC dares to try to prove that Sarao caused-or even contributed to-the Crash, it will face huge obstacles. Yes, as Chris Clearfield and James Weatherall rightly point out, financial markets are emergent, highly interconnected and tightly coupled. This creates non-linearities: small changes in initial conditions can lead to huge changes in the state of the system. A butterfly flapping its wings in the Amazon can cause a hurricane in the Gulf of Mexico: but tell me, exactly, which of the billions of butterflies in the Amazon caused a particular storm? And note, that it is the nature of these systems that changing the butterfly’s position slightly (or changing the position of other butterflies) can result in a completely different outcome (because such systems are highly sensitive to initial conditions). There were many actors in the markets on 6 May, 2010. Attributing the huge change in the system to the behavior of any one individual is clearly impossible. As a matter of theory, yes, it is possible that given the state of the system on 6 May that activity that Sarao undertook with no adverse consequences on myriad other days caused the market to crash on that particular day when it didn’t on other days: it is metaphysically impossible to prove it. The very nature of emergent orders makes it impossible to reverse engineer the cause out of the effect.

A few additional points.

I continue to be deeply disturbed by the “sample days” concept employed in the complaints and in Hendershott’s analysis. This smacks of cherry picking. Even if one uses a sample, it should be a random one. And yeah, right, it just so happened that the Flash Crash day and the two preceding days turned up in a random sample. Pure chance! This further feeds suspicions of cherry picking, and opportunistic and sensationalist cherry picking at that.

Further, Hendershott (in paragraph 22 of his affidavit) asserts that there was a statistically significant price decline after Sarao turned on the algorithm, and a statistically significant price increase when he turned it off. But he presents no numbers, whereas he does report impacts of non-Sarao-specific activity elsewhere in the affidavit. This is highly suspicious. Is he too embarrassed to report the magnitude? This is a major omission, because it is the impact of Sarao’s activity, not offering away from the market generally, that is at issue here.

Relatedly, why not run a VAR (and the associated IRF) using Sarao’s orders as one of the variables? After all, this is the variable of interest: what we want to know is how Sarao’s orders affected prices. Hendershott is implicitly imposing a restriction, namely, that Sarao’s orders have the same impact as other orders at the same level of the book. But that is testable.

Moreover, Hendershott’s concluding paragraph (paragraph 23) is incredibly weak, and smacks of post hoc, ergo propter hoc reasoning. He insinuates that Sarao contributed to the Crash, but oddly distances himself from responsibility for the claim, throwing it on regulators instead: “The layering algorithm contributed to the overall Order Book imbalances and market conditions that the regulators say led to the liquidity deterioration prior to the Flash Crash.” Uhm, Terrence, you are the expert here: it is incumbent on you to demonstrate that connection, using rigorous empirical methods.

In sum, the criminal and civil complaints make a Matterhorn out of a molehill, and a small molehill at that. And don’t take my word for it: take the “[declaration] under penalty of perjury” of the CFTC’s expert. This is a matter of magnitudes, and magnitudes matter. The CFTC’s own expert estimates very small impacts, and impacts that oscillate up and down with the activation and de-activation of the algorithm.

Yes, Sarao’s conduct was dodgy, clearly, and there is a colorable case that he did engage in spoofing and layering. But the disparity between the impact of his conduct as estimated by the government’s own expert and the legal consequences that could arise from his prosecution is so huge as to be outrageous.

Particularly so since over the years CFTC has responded to acts that have caused huge price distortions, and inflicted losses in nine and ten figures, with all of the situational awareness of Helen Keller. It is as if the enforcers see the world through a fun house mirror that grotesquely magnifies some things, and microscopically shrinks others.

In proceeding as they have, DOJ and the CFTC have set off a feeding frenzy that could have huge regulatory and political impacts that affect the exchanges, the markets, and all market participants. CFTC’s new anti-manipulation authority permits it to sanction reckless conduct. If it was held to that standard, the Sarao prosecution would earn it a long stretch of hard time.

*Hendershott’s affidavit says that Exhibit 4 reports the IRF analysis, but it does not.


Print Friendly, PDF & Email


  1. Being a financial ignoramus I only just about followed your posts on the subject of Mr. Sarao. However I do have one question and I apologise if you answered it in the course of your posts but am I, and any future jury, to assume that Mr. Sarao was the *only* operator of the systems you describe? And if there were others, how do the prosecution differentiate?

    Jest askin’!

    Comment by David Duff — April 26, 2015 @ 1:39 pm

  2. @David-A good question. We don’t know how many operators there were, but there are many algorithmic traders and many HFT, so trying to differentiate the effect of Sarao’s activities from theirs is nigh on to impossible.

    The ProfessorComment by The Professor — April 26, 2015 @ 3:45 pm

  3. The more I read of this case the more it looks like we saw the trailer, but they didn’t near finish the movie yet.

    I’ve seen a lot of great trailers for what later turned out to be really shitty movies.

    Reading Coscia, a spoofer cancels lots of orders right? He offers 1000, bids 1, trades 1, then kills the 1000. Then flips.

    OK. Where in the complaint is the comparison of order canx rates for different sizes of Sarao order, where they show his little orders traded but his big ones never did? Where is that same comparison for him versus all other participants, and versus all other participants *like* him, that shows he was unusual?

    Comment by Green as Grass — April 27, 2015 @ 2:33 am

  4. Thanks, ‘Prof’, and here’s another question – sorry!

    If indeed it can be shown that there were numerous examples of Mr. Sarao’s type of operation long before the ‘crash’ you describe, is there not a case for the losers in that ‘crash’ to sue the government for lack of due diligence?

    As an aside, this case against Mr. Sarao confirmed my deep suspicions concerning the American ‘justice’ system. I have watched what appears to me to be the naked daylight robbery of BP over the Gulf oil spill in which (or so it seems from ‘over here’) “Uncle Tom Cobleigh and all” have tossed in claims for enormous amounts which have barely been checked as judges play to their home crowd. Then, of course, there are the internal activities of your DoJ which seems to be the legal arm of the Democrat party.

    I fear for Mr. Sarao!

    Comment by David Duff — April 27, 2015 @ 3:08 am

  5. Sorry again because I do seem to be taking up a lot of space on your site but by coincidence I have just read the following in my morning paper:

    It is written by a Brit who very definitely knows where-of he writes having been hauled through the American ‘justice’ system himself – rightly or wrongly I do not know. Not that it seems to make much difference because once the Feds indict you the odds are not just stacked but built higher than the Empire State building to ensure that you plead ‘guilty’ – or else! It makes the North Korean ‘justice’ system appear not too bad after all!

    Comment by David Duff — April 27, 2015 @ 4:40 am

  6. @ David

    from your link – “He would do well to take note of the statistic that nearly 98 per cent of people indicted in the Federal system in America enter into plea bargains rather than roll the dice and go to trial.”

    That is not far off the conviction rate of the Nazi People’s Court.

    Comment by Green as Grass — April 27, 2015 @ 9:14 am

  7. @Green-Exactly. To read the complaint is to get the impression that Sarao’s profits are like the Immaculate Conception. The complaint goes on and on about his offering and canceling, but nothing about his executed orders or how they executed. Last time I checked, you couldn’t make $40 million over 5 years by just canceling and modifying orders. There is information in the CFTC supporting materials about how his cancellation/modification rate was substantially higher than average, but no other comparisons.

    We do know Sarao was trading 65-85K cars a day. That should mean something, you’d think.

    The ProfessorComment by The Professor — April 27, 2015 @ 3:44 pm

  8. @David-The CME is likely immune from civil suits on this as a self-regulatory organization: suing the CFTC is definitely out.

    Your suspicions concerning the US justice system are definitely warranted, especially in politicized cases like the oil spill and cases like Sarao and some insider trading cases.

    The ProfessorComment by The Professor — April 27, 2015 @ 3:46 pm

  9. @David-No problem. Comments always appreciated. There is a high likelihood that they are piling on charges in order to extract a guilty plea from Sarao. It’s like forcing the defendant to play Russian Roulette dozens of times. They only have to get you on one or two charges.

    The ProfessorComment by The Professor — April 27, 2015 @ 3:50 pm

  10. Thanks for this prof.

    I liked John Hempton’s take on this (and not just because like me he’s Australian and a former Treasury boffin ;-> ):

    Something else:

    Market regulators get played by insiders and make geese of themselves. In other news, the weather in Singapore today will be hot and humid, with occasional showers.

    Comment by Ex-regulator on lunch break — April 28, 2015 @ 1:39 am

  11. Hi Craig, very interesting analysis…. a couple questions:

    If the magnitude of the effects are so small, why do you think they chose this case to pursue? Is it because he’s just a small time operator? His strategy was particularly obvious? Aren’t spoofing and layering fairly common? Why go after this guy? Just to make an example of someone?

    The language of the anti-spoofing provision Dodd-Frank added to the CEA is simple and broad but the judge in Coscia rejected the void for vagueness argument of the defense attorneys (in Radley, it was anti-manipulation provisions that were deemed unconstitutionally vague, not CEA 4(c)(a)(5)(C)). Whatever the wisdom of passing a provision like this (I for one support it despite the difficulties of trying to police this conduct) it seems likely from the complaint against that Sarao was engaged in submitting quotes with the intent of cancelling them before execution. So, even if the economic effects of his conduct are small, doesn’t it appear likely that he violated the law? So at any rate it’s not crazy that they are charging him, although to imply that it caused, or helped cause the flash crash looks like a serious overstatement. But I guess it goes back to the first question–why are they going after this guy when lots of people were likely using these strategies?

    Comment by Steve — May 1, 2015 @ 8:55 am

  12. @Steve-Thanks. I think there are a variety of factors at work here. First, CFTC wants to show it is tough on manipulation. Second, the connection to the Flash Crash, farcical as it is, ensured a lot of publicity and attention. Together, these factors help the CFTC make its case for a bigger budget. Chairman Massad made that argument almost immediately after the indictment: he said something to the effect if we had a bigger budget they could catch more of this conduct and do so more quickly. The fact that the guy is a lone wolf without a big pocket firm behind him also is attractive to CFTC.

    Like I wrote, there is a colorable case that Sarao spoofed/layered, and that this is against the law. But apropos Dickens,this mainly suggests the law is an ass. Or, at least it is overkill. The minimal impact that the government itself documents strongly suggests that this kind of conduct warrants a moderate fine/civil penalty, at most, not the decades-and perhaps centuries-of jail time that Sarao could serve if convicted.

    This is the fun house mirror aspect of the prosecution. It is magnifying marginally damaging conduct into some monstrous crime.

    The ProfessorComment by The Professor — May 1, 2015 @ 7:44 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress