In collaboration with my classmates, I undertook a research project aimed at predicting the realized volatility of the S&P500 index using neural network models. We sought to leverage machine learning to provide more accurate volatility forecasts, which are crucial for traders and financial analysts.

Objectives

Our primary objective was to explore and validate the effectiveness of various neural network architectures, including Deep Neural Networks (DNNs), Deep & Wide Neural Networks, and Recurrent Neural Networks (RNNs) with LSTM layers. We aimed to compare these models in terms of their predictive accuracy, focusing on their ability to forecast the S&P500's realized volatility. Additionally, we experimented with different hyperparameter tuning strategies and incorporated external data such as the VIX index and treasury yields to enhance the models' input features.

Results

The project demonstrated that neural networks hold significant promise in forecasting financial volatility. The Deep & Wide Neural Network, in particular, showed substantial improvements in prediction accuracy, reducing the initial model loss from 17.5 to as low as 0.0006008 over 100 epochs. Despite the models generally following the trend of actual values, they tended to underestimate peaks in volatility, highlighting areas for further refinement. By iterating on model architectures and tuning hyperparameters, we achieved commendable results, underscoring the potential of neural networks in financial forecasting. This project honed my skills in data preprocessing, model construction, and hyperparameter tuning while deepening my interest in applying machine learning in finance.

Please feel free to access my project files at https://github.com/gurjasbatra/Predicting-S-P-500-volatility.