By Irene Aldridge This article first appeared in the Big Data Finance magazine Social media has fascinated Finance for about a decade. Extracting sentiment from online posts have proven to be both innovative for gauging investor sentiment and profitable for estimating direction of the impending price move and volatility. Companies like AbleMarkets, a Big Data platform and a supplier of Internet sentiment index for most U.S.-based stocks, and Suite LLC, an industrial-grade derivatives pricing and risk management software agree: Internet sentiment is highly predictive of impending volatility. In addition to social media sentiment, a new kind of social media is entering Finance as we know it:Read More →

By Irene Aldridge This article was first published on Medium Big Data, Machine Learning and Artificial Intelligence are three du-jour buzzwords of today’s business. If your business does not do one of the three, you risk being considered tardy, inefficient, or, gasp, uncool, particularly with the dreaded taste-making millennial set. Worst of all, you may miss the next chance of becoming a unicorn — a billion-dollar entity like Google and Facebook that deployed Big Data, Machine Learning and Artificial Intelligence techniques to turn reams of data points into solid gold. Various Big Data, Machine Learning and AI methodologies, trends and ideas, will be discussed in detail atRead More →

By Irene Aldridge Many ETF and active portfolio managers need accurate end-of-day price direction forecasts to optimally execute their daily portfolio reallocation requirements. Predicting the end-of-day direction of the price is crucial to solidify performance, as the last half-hour is known to be marked by extreme volatility. The end-of-day volatility, if met on the wrong side of the price movement, may reduce the performance gains of even the most seasoned and talented portfolio managers to rubble. The main challenge with predicting market direction at the close of the day is to estimate the objectives and motivation of market participants. To simplify, most market participants inRead More →

By Irene Aldridge The last two weeks witnessed a somewhat forgotten phenomenon — a market headed south at a rapid pace for several consecutive days. Unlike flash crashes, brief spikes of downward volatility that can be predictable (see Aldridge, I., “High-Frequency Runs and Flash Crash Predictability”, Journal of Portfolio Management, 2014, and AbleMarkets Streaming Flash Crash Index), the sell-off of the past two weeks was methodical, slow and painful. AbleMarkets, a Big Data platform for finance, tracks institutional activity by pinpointing electronic algorithms used to break up large orders throughout the day. AbleMarkets uses the most granular tick-level data from exchanges to identify market microstructure footprints of institutionsRead More →

By Irene Aldridge Selling volatility has been a popular trading strategy among hedge funds over the past couple of years. At the core of the strategy’s popularity is the observation that volatility becomes considerably more severe when the markets are moving down rather than when they are rising up (see, for example, “The Cross-Section of Volatility and Expected Returns” by Ang, Hodrick, Xing and Zhang, Journal of Finance, 2005). In other words, selling volatility is a complicated way of betting on the rise of the market. During the current administration’s tenure, the U.S. markets have consistently risen, while dampening volatility in the process and generating excitement amongRead More →

Abstract In the classic portfolio management theory by Markowitz (1952), the weights of the optimized portfolios are directly proportional to the inverse of the asset correlation matrix. However, most of contemporary portfolio optimization research focuses on optimizing the correlation matrix itself, and not its inverse. We show that this is a mistake. Specifically, from the Big Data perspective, we prove that the inverse of the correlation matrix is much more unstable and sensitive to random perturbations than the correlation matrix itself. As such, optimization of the inverse of the correlation matrix adds more value to optimal portfolio selection than that of the correlation matrix. WeRead More →

In the next five years, big data analysis is poised to become one of the most important and competitive skill sets around. Portfolio analysis in particular is where pension funds are focusing their big data investments. Big data is a set of techniques embedded in the latest, most sophisticated technologies: social media analytics, digital video recognition, 5G cellular technology and much more. The capabilities of big data are incredibly powerful and extend far beyond traditional systems. Supported as a spying technology in the World War II and later, the Cold War, core big data techniques were developed in the 1940s, 1960s and 1980s and areRead More →

By Irene Aldridge The history of Finance is full of Mathematical and technological revolutions. With the expansion of computer technology, mathematical innovations are not only abstractly interesting, but also very fast and profitable. Most recently, mathematical innovations in Finance first generated opportunities in quant techniques that were followed by the revolution of high-frequency trading, Now, the opportunity expands as Big Data techniques are implemented within Finance. Many people are still puzzled by what is so special about Big Data. After all, Econometrics and other data processing tools have been used in Finance for decades. It is true that Econometrics comprises a part of Big DataRead More →

Irene Aldridge’s latest paper on Big Data optimization in portfolio management is the first to show that spectral decomposition of an inverse of the correlation matrix delivers 400% improvement over the equally-weighted and other common portfolio optimization schemes. Read more here: More →