Trying to predict the future value of a company stock or other financial instrument traded on an exchange is called stock market prediction. If we generalize the problem, we end up talking about talking about time series forecasting. This is the construction of a model which can predict future values, based on previously observed values. A common used tool for this kind of prediction are ANNs (artificial neural networks).

In this tutorial, the real life problem which we are trying to solve using artificial neural networks is the prediction of a stock market index value. We will start with an introduction to artificial neural networks, we will continue with a short introduction to Neuroph – a Java neural networks library and afterwards we will implement the prediction algorithm.

Artificial neural networks

The definion of ANNs as found on Wikipedia: “In machine learning and cognitive science, artificial neural networks (ANNs) are a family of statistical learning algorithms inspired by biological neural networks (the central nervous systems of animals, in particular the brain) and are used to estimate or approximate functions that can depend on a large number of inputs and are generally unknown. Artificial neural networks are generally presented as systems of interconnected “neurons” which can compute values from inputs, and are capable of machine learning as well as pattern recognition thanks to their adaptive nature.”

Why neural networks?

There are of course other machine learning techniques which can approximate functions, but when it comes to a large number of inputs and non linearly separable data, ANNs are the best choice. Let’s see a simple example for linear and non-linear separable data. If we take the OR function the values table looks like this:

 

As you can see, we can draw a line in order to separate the “True” values from the “False” values. If we observe the XOR function, we can see that we cannot draw a line to separate the “True” values from the “False” values, we need a special function to do that. The values are not linear separable and we should use ANNs.

 

The artificial neuron

As the name implies, the ANNs is a network of artificial neurons. Let’s see first how a biological neuron looks like:

We distinguish the following elements:

– the dendrites are the inputs of the neuron. They receive electrical signals from other neurons.

– the soma, or the body of the neuron which contains the nucleus.

– the axon is the output of the neuron, it transmits electrical signals to other neurons.

Similarly we have an artificial neuron which looks like this:

We can see here:

– the inputs X1 to Xn,  representing the dendrites. They have weights associated W1 to Wn.

– the transfer function, representing the soma from the biological neuron. Here, the inputs are multiplied by the corresponding weights Xi x Wi and the results are summed – the transfer function in the above diagram.

– the activation function (representing the axon) is a function which gets its input from the transfer function (soma) and decides if the resulted signal is transmitted forward or not. The functions usually have a sigmoid shape, the same function used in logistic regression – see Logistic regression using Apache Mahout tutorial

A network of neurons

An artificial neural network is a network of fully connected artificial neurons which usually has an input layer, one or more hidden layers and an output layer. The information flows in one direction only: from the input layer to the output layer, passing through the hidden layer. Each unit in a layer is connected in the forward direction to every unit in the next layer. Weights between units encode the network’s knowledge.

 

The most common ANN type is the backpropagation network. This usually starts with a random set of connection weights. The network is trained (or learns) supervised by iteratively adjusting these weights. The learning process has two steps :

– forward pass: passing input data through the network until it reaches the output layer.

– backpropagation pass: the values from the output layer are compared with the expected values and an error is computed for each output unit. The weights connected to the output units are adjusted to reduce those errors. The error estimates of the output units are then used to derive error estimates for the units in the hidden layers. Here also, the weight are adjusted to reduce the errors. Finally, the errors are propagated back to the connections stemming from the input units.

The error in each layer is multiplied by a term called learning rate, which steers actualy the learning process. A bigger learning rate increases the learning speed but decreases the accuracy and a smaller learning rate takes a longer time, but increases the accuracy of the neural network.

After each round of forward-backward passes, the system “learns” incrementally from the input-output pair and reduces the difference (error) between the network’s predicted output and the actual output. After extensive training, the network will eventually establish the input-output relationships through the adjusted weights on the network.

Neuroph framework

Neuroph is a lightweight Java neural networks framework for developing common neural networks architectures. It provides a Java neural network library as well as a GUI tool that supports creating, training and saving neural networks.  Neuroph is released as open source under the Apache 2.0 license. You can find out more about the framework here: Neuroph – Java neural network framework.

Stock market prediction

To predict the future values for a stock market index, we will use the values that the index had in the past. We will train the neural network with the values arranged in form of a sliding window: we take the values from 5 consecutive days and try to predict the value for the 6th day.

For this demo used the values for the S&P 500 Index. Standard & Poor’s 500, is an American stock market index based on the market capitalizations of 500 large companies having common stock listed on the NYSE or NASDAQ. I downloaded the values at 3 years from http://us.spindices.com/indices/equity/sp-500. From the resulting Excel file I kept the first column (with the date) and the last column (the one named S&P 500). The values 2 years back (in my case  from 2013 and 2014) are intended for training – rawTrainingData, the values for the current year (2015) are intended for testing – rawTestingData.

The program starts with a data preparation step, where the raw training data is written to a new file with 6 values per row: 5 values with the input data and the 6th value with the expected value. Neuroph expects the input layer to receive values from between 0 (zero) and 1 (one). For this, in this step we will also normalize the input data to have values betwee 0 and 1. This is done by finding the minimum and maximum of the index’s values and for every value to apply the formula:

Normalized value  = (Value – Min) / (Max – Min) * 0.8 + 0.1

We use here 0.8 and 0.1 in order to avoid to have 0 and 1 as input values.

The next step is to train the neural network using the normalized data. For the training step we have the option to specify the maximum number of training iterations, the learning rate or the network error after which we can stop training.

We can test the  trained neural network by taking 5 consecutive values from the testing data file – rawTestingData and  predict the 6th value.

Java project for stock market prediction

Prerequisites:

• Linux or Mac
• Java 1.3
• Apache Maven 3

Create the Maven project:

Rename the default created App class to NeuralNetworkStockPredictor using the following command:

Add the Neuroph repository and library to this project:

Add the following repository definition lines above the dependencies section and the Neuroph dependency  to the dependencies section:

Create a folder named input and copy the file containing the training data – rawTrainingData, optionally add also the testing data file –rawTestingData.

Edit the NeuralNetworkStockPredictor class file and add the following code:

Run the class by using the following command:

The output should be similar with the one below:

As you can see, the network error is decreasing with every iteration and the predicted value is very close to the expected value.In the testing method I used the last six value from the rawTestingData file.

The output which you will have will probably be slightly different, as the neural network initializes itself each time with random weight values. Also, if you run the program multiple time, you will get slightly different values each time because from the same reason.

Disclaimer: please take this demo as it is and don’t support you real life stock market buying decisions on it 🙂

GitHub repository for this project: https://github.com/technobium/neuroph-neural-network

Conclusion

Stock market prediction is just one of the usages of artificial neural networks. This interesting machine learning technique which is inspired by the human brain was succesfully used in fields like: medical diagnosis, industrial process control, sales forecasting, credit ranking, employee selection and hiring, employee retention or game development.

References

http://natureofcode.com/book/chapter-10-neural-networks/

http://www.javaworld.com/article/2071879/enterprise-java/a-neural-network-for-java-lego-robots.html

http://neuroph.sourceforge.net/tutorials/StockMarketPredictionTutorial.html