By Irene Aldridge In the Spring of 2013, a lively discussion on LinkedIn went like this: – “If someone says ‘Big Data’ one more time, I am going to throw up”, declared Head of Marketing at a prominent software firm – “Agree, ‘Big Data’ is such an annoying buzzword”, chimed in Head of Research at a mid-tier broker-dealer – “Uh, it’s such a fad”, stated a well-funded hedge fund manager. And so it went: “Big Data” is annoying, fleeting, and, by implication, useless. Fast forward to today, even though Big Data is a much more established term, eyes still roll when the subject comes up.
Over the last few years, a number of exchanges and dark pools emerged claiming that their businesses will exclude high-frequency traders (HFTs) detrimental to institutional investors. Almost invariably, the HFTs in question happened to be the so-called Aggressive HFTs: HFTs that execute mostly using market orders and have been shown to erode liquidity, causing short-term volatility in the process. While the idea of excluding aggressive HFTs may be appealing to investors, the realities of modern microstructure preclude this from happening, as this article discusses. As a result, most of today’s exchanges in the United States have a similar proportion of aggressive HFTs by volume of
A New York Times article covering the latest Triple Crown horse race winner, American Pharaoh, noted that the horse was identified as having an amazing potential when the animal was only 1 year old. The prediction of success was made by a team of data scientists who estimated the horse’s performance by noting the size of the winner’s heart and other characteristics and comparing them with those of past race winners. On the future potential of the horse, the data scientists advised him, “to sell the house, but keep the horse.” Their prediction paid off – American Pharaoh won. The real victory, however, can be