3-vector Series LSTM Can't Break 0.5 Accuracy

I have a toy series dataset of 3-vectors in the form of [[0, 0, 2], [1, 0, 3], [2, 0, 4], [3, 0, 2], [4, 0, 3], [5, 0, 4] ... [10001, 0, 4]] x always goes up by one, y is always 0

Solution 1:

Right now your model is set up as a classifier but from your description it seems you are trying to solve a regression problem. Let me know if I am misunderstanding.

Try changing the activation on the final dense layer to 'linear'. Also change the loss function to 'mean_squared_error' or another regression loss. https://keras.io/losses/

You will not be able to get an accuracy score on a regression problem, instead you will see the mean squared error and any other regression metrics you add like 'mae' for mean average error which is useful for a more human readable error number.

You should be able to solve this with a small network so increasing the number of layers and units is not necessary.

In response to you comment:

If the timseries don't interact with each other then there isnt really any reason to predict them at the same time, so you'll have to decide that first. Here is how you could change them to classification problems of you want.

Based on you description I can't see a way to frame the X axis as a classification problem since it is just an increasing number.

For the Y axis you could have the network predict whether the next point will be a zero or not. So you would want the labels for this axis to be either 0 or 1 depending on whether the point is 0 or not. The final activation would be a dense layer with 1 unit and sigmoid activation. However if the occurrences of non zero values is completely random then it would be impossible to accurately predict.

For the Z axis you could frame it as a multiclass classification problem. Your labels would have a width 3 where the correct number is one hot encoded. So if the next Z axis value was 2 then your labels would be [1, 0, 0]. The final layer should be a dense layer with 3 units. The activation should be softmax because you want it to select 1 of the 3 options, as apposed to a sigmoid activation which could select any combination of the three.

You could predict these all in one network if you used Kerases functional model API to do multi-output.