Could a generalized AI model like GPT-4 make better financial predictions than expert human analysts? Researchers from the University of Chicago explored this question and came up with surprising results.
The fear over AI replacing jobs has centered mainly on lower-skilled job functions, but might experts like investment and financial analysts need to rethink their careers too?
The paper the researchers published states that GPT-4 is significantly better than human experts at analyzing a company’s financial data and predicting its future earnings.
Their experiment involved passing a balance sheet and income statement in a standardized form to GPT-4 Turbo and asking the model to analyze them.
The model was then asked if the company’s economic performance was sustainable and whether its earnings would grow or decline in the following period.
By using Chain-of-Thought (CoT) prompting they coached the model to mimic a financial analyst in the way it analyzed the data.
The researchers wanted to exclusively test GPT-4’s ability to analyze financial numbers so they didn’t provide it with any other data like the company’s name, management discussions, or other corporate information.
The researchers also didn’t let GPT-4 know from which year the financial data came. These efforts prevented GPT-4 from using its training data to make inferences about the company or economic conditions when the financials were compiled.
Results
GPT-4 achieved a prediction accuracy of 60% which significantly outperformed the 53% accuracy that the human analysts achieved on the same sample of corporate figures.
Human analysts usually analyze a company’s figures and combine that analysis with their industry and private knowledge about the firm before issuing their forecasts.
It seems like GPT-4 outperformed the humans even with the disadvantage of only having the figures with no other context.
But it might be this dispassionate, unbiased analysis by GPT-4 that gives it an advantage over humans who are often influenced by sentiment or emotion to make biased predictions.
Financial analysts often employ specialized machine learning (ML) models to crunch the numbers to make predictions for them. The researchers found that GPT-4, a generalized model, performed on par or slightly better than the state-of-the-art specialized ML models.
The researchers concluded that even though they’re not sure how GPT-4 does it, its ability to outperform human specialists has the “potential for LLMs to democratize financial information processing.”
Will AI replace Wall Street analysts? It might happen sooner than you think because humans haven’t been doing a great job.
A 2020 study compared actively managed investment funds to the performance of the S&P 500 benchmark index.
That study found that if you invested in the S&P 500 you would have achieved better results than 89% of the fund managers. In other words, only 11% of the financial analysts at those investment firms beat the index in 2020.
With GPT-4 appearing to significantly outperform human analysts you may soon be asking ChatGPT whether to buy or sell, rather than your fund manager.
