Revised and Improved: Big Data in Portfolio Allocation — A New Approach to Successful Portfolio Optimization


In the classic portfolio management theory by Markowitz (1952), the weights of the optimized portfolios are directly proportional to the inverse of the asset correlation matrix. However, most of contemporary portfolio optimization research focuses on optimizing the correlation matrix itself, and not its inverse. We show that this is a mistake. Specifically, from the Big Data perspective, we prove that the inverse of the correlation matrix is much more unstable and sensitive to random perturbations than the correlation matrix itself. As such, optimization of the inverse of the correlation matrix adds more value to optimal portfolio selection than that of the correlation matrix. We further show the empirical results of portfolio reallocation under different common portfolio composition scenarios, and outperform traditional portfolio allocation techniques out-of-sample, delivering nearly 400% improvement over the equally-weighted allocation over a 20-year investment period on the S&P 500 portfolio with monthly reallocation.

In general, we show that the correlation inverse optimization proposed in this paper significantly outperforms the other core portfolio allocation strategies, such as equally-weighted portfolios, vanilla mean-variance optimization, and the techniques based on the spectral decomposition of the correlation matrix. The results presented herein are novel in Data Science space, extend far beyond financial data, and are applicable to any data correlation matrices and their inverses, whether in advertising, healthcare or genomics.

Keywords: Portfolio optimization, big data, investment management, correlation, inverse, data science

JEL Classification: C02, C60


Please click here to download the article PDF from SSRN