TensorFlow for Stock Price Prediction - [Tutorial]
Sebastian Heinz, CEO at Statworx, has posted a tutorial on Medium about using TensorFlow for stock price prediction. Are we all deep learning enthusiasts going to become millionaires? Hold on a sec...
In an internal competition at Statworx, team members collected S&P 500 data from Google's API:
"Having this data at hand, the idea of developing a deep learning model for predicting the S&P 500 index based on the 500 constituents prices one minute ago came immediately on my mind." [source]
So, he develop a TensorFlow model and decided to write about it afterwards on Medium. If you want to replicate his work, you'll first have to download a 40 MB file of the dataset, which is very very small for DL purposes.
In his tutorial, Sebastian goes through:
- data importing and data preparation
- splitting into train/test
- scaling (using the MinMaxScaler of scikit-learn)
- short intro to TensorFlow (for beginners) - placeholders, variables
- creating the architecture of the network
- calculating the cost function
- optimizing
- training the neural net
I'd say this is a rather complete tutorial which encompasses most of the concepts and methods you'd have to implement in a typical ML/DL project. So, what better way to learn than through this type of exemplified (step-by-step) tutorial? Who knows, with the right data and a good algorithm you might get yourself a 1 followed by multiple zeroes in your bank account?! If you do that, don't forget about me! :)
Please read and get your hands on the complete tutorial at Medium:
To stay in touch with me, follow @cristi
Cristi Vlad Self-Experimenter and Author
Hi @cristi, does Heinz indicate what data sources are available as inputs? From your article it almost sounds like it's limited to historic stock prices. But that would create a terrible model because it would ignore very valuable (and perhaps more important) market information.
For example, the non-farm labor report published in the US every month has a huge impact on the US stock exchanges. Two down months are seen as a potential signal for an upcoming recession.
The Dry Baltic Index can be a good indicator for international shipping. The price of oil on world markets is another useful indicator.
For individual sectors, there are other useful indicators that don't show up in historic stock price data. For retail and restaurants, you would be interested in same store sales. For the defense industry, you would look at the award amounts for contracts negotiated with the government. For oil and gas, you would want to know the number of active wells as a rough indicator of supply.
If you look at the machine learning algorithms being used by the big arbitrage players, they go way beyond historical stock data and incorporate thousands and thousands of additional prices as features for the model.
I guess my point is, we need to be careful with claims by some of offering cheap AIs for the public with regard to stock prediction. It's not all that hard to create a deep learning neural network. Either of us could do that. And so could legions of students graduating from the top AI graduate schools (Stanford, MIT, Carnegie Mellon, Berkeley).
The real trick is getting access to all of the data that would be necessary. And those data feeds can be closely held and expensive.
yes:
"Our team exported the scraped stock data from our scraping server as a csv file. The dataset contained n = 41266 minutes of data ranging from April to August 2017 on 500 stocks as well as the total S&P 500 index price. Index and stocks are arranged in wide format." [source
I guess it may not be the best way to go, but for their illustration purposes, it makes a case. Yes, there are many variables that need to be accounted for so that you'd have a more efficient model at stock prediction.
I'd say that it's not the data that's closely held and expensive, but the model algorithms. Thoughts?
Well, think of it this way. If this actually worked (feeding stock prices into a deep neural network), every investment bank and stock broker on Wall Street would be achieving new performance levels (investment gains) that beat all their past years efforts. But they're not. These guys have access to the best AI experts and and the best AI algorithms, and they were the first business to jump on the deep learning bandwagon. And they have millions of dollars to throw at this to make it work.
So far, the only people getting truly rich from AI-driven stock trading are the arbitragers. They use the algorithms to detect differences in asset prices from one market/exchange to another.
I'm going to give you only one name: James Simmons :)
James Simons retired from Renaissance Technologies in 2009, having made hist fortune long before then. Deep learning didn't really start going until 2010 with various successful research experiments and academic competitions.
Simons has expertise in mathematics, but did any of it include neural networks or AI?
Also, there's a lot of controversy surrounding Simons. His Medallion Fund, which has been available exclusively to current and past employees and their families, surged 80% in 2008 in spite of hefty fees. But his Renaissance Institutional Equities Fund (RIEF), owned by outsiders, lost money in both 2008 and 2009. RIEF declined 16% in 2008. And then there was the questionable activity hiding day trading as long term investments, which conceivably amounted to tax fraud.
This is a form of AI, right? It's fascinating and frightening how fast AI is growing and becoming more powerful by the day... The future will be very interesting!
well, I wouldnt call it AI, but many people do. It's just deep learning, a type of machine learning.
Good post, help mi