COVID19 Time Series Forecasting using LSTM-RNN (2024)

Ashutosh Krishna

Published in

DataDrivenInvestor

8 min read

Mar 28, 2020

CSSEGISandData/COVID-19

DEPRICATED WARNING--- The files below will no longer be updated. With the release of the new data structure, we are…

github.com

Let us talk about the modelling part now.

Since, for most of the countries time series plot was pretty simple as the number of cases were growing exponentially. So, I applied the Holt Winter’s method which is used for exponential smoothing. I set trend and smoothing level to be “mul” and 1.0 respectively. The numbers were increasing exponentially and predicted data was very much dependent on data points closer rather than being far. So as per Holt Winter’s method, smoothing level (0<alpha<1) needs to be tending to 1 if we want forecast to be strictly dependent on latest dates. In the figure below, number of confirmed cases in Iran has been plotted.

COVID19 Time Series Forecasting using LSTM-RNN (3)

The curve looks flat in the beginning until last week of February but there is a sudden surge in numbers from first week of March.

The Holt winter’s method yielded great results with Mean Absolute Percentage error(MAPE) being in the range of 4% — 10%. This approach was working well for countries with significant growth in number of confirmed cases such as Italy, Spain, Iran and USA. But for countries like China,Singapore and south Korea where the epidemic has been contained to a certain extent the curve was tending to be saturated.

COVID19 Time Series Forecasting using LSTM-RNN (4)

So these cases required overall data to be paid attention rather than putting more weight to the latest data. So the hyperparameters needed to be changed for these cases for MAPE to be in safe range.

Since, these involved lots of manual intervention , we decided to build a robust model which will take care of the curve whether it is flat or exponential. Let’s say, in future, Italy contains the spread of Coronavirus the Holt Winter will not yield a good result with same hyper-parameters. However, I liked the fact that Holt Winter’s method is pretty easy to implement for a particular case but it is difficult to generalize it for every case.

Five Data Science and Machine Learning Trends That Will Define Job Prospects in 2020 | Data Driven…Data Science and ML have been one of the most talked-about trends in 2019 and without any surprise, they will continue…www.datadriveninvestor.com

And the above fact set the context of this article — “ a Robust LSTM time series Model ”. Let us talk about the technical details now.

Requirements:

Jupyter notebook with Python 3 or above

pandas , numpy, sklearn, Keras, tensorflow

Step 1 : Data Collection

John Hopkins University has been publishing time series data for confirmed, recovered and death cases every day for each country here. I will be taking confirmed cases for the time being as I want my article to be precise. For recovered and dead cases you can replicate the same process.

Reading data for Iran and munging it to a proper time series format.

COVID19 Time Series Forecasting using LSTM-RNN (5)

Plot of confirmed,dead and recovered cases in Iran

COVID19 Time Series Forecasting using LSTM-RNN (6)

Step 2: Data Pre-processing

Since there is enormous variation in data we will be taking small steps for prediction . Here we are taking 5 steps i.e validation set would be of 5 data points and rest would be training set. Since data is available from 22 jan so there are total 66 days or 66 data points.

validation set= 5 data points

train= total-validation set= 66–5=61 data points

COVID19 Time Series Forecasting using LSTM-RNN (7)

Since the data is heavily skewed (it starts from zero and goes up to thousands), we will normalize the data(divide very value by max value of the training set) based on the training set.

COVID19 Time Series Forecasting using LSTM-RNN (8)

Time Series Generator

Time series Generator is a Utility class for generating batches of temporal data in keras i.e. producing batches for training/validation from a regular time series data. These batches will be fed to train the model.

For our case, we are taking 5 steps i.e taking 5 data points in account to predict 6th data point. So, The batches would be [feed=[t1,t2,t3,t4,t5],predict=t6].

COVID19 Time Series Forecasting using LSTM-RNN (9)

the flow would be

x → Neural Network/update weights → y

Step 3 : Model Building

We are implementing LSTM(Long short Term Memory) algorithm using Keras. Since any neural network is multi-layered (input → hidden layers → output layers) we are using Sequential class of keras library.

I am taking number of neurons to be around 150. However we can always get optimum number of neurons with grid search and k-fold cross validation. But as a preliminary approach , we can use the following formula.

Where, i = 3/2 * h *input_points

h is number of hidden layers, i is input neuron, input_points is number of training data points.

we have 2 hidden layers and 1 output neuron so, input neuron = 2 * 3/2 * 60 = 180. I have rounded it to 150.

Initially, we had just one layer with 150 neurons but in order to improve Accuracy and Model robustness we added one more layer with 75 neurons. So the structure of the model would be — -> pass 150 neurons to a LSTM layer → shrink the output to 75 neurons to be fed to a dense layer → one more dense layer which will further shrink the output to be 1

COVID19 Time Series Forecasting using LSTM-RNN (10)

Activation function are non-linear transformations which breaks any linearity if present in the data.Activation functions are really important for a Artificial Neural Network to learn and make sense of something really complicated and Non-linear complex functional mappings between the inputs and response variable.A Neural Network without Activation function would simply be a Linear Model, which has limited power and does not performs good most of the times. We want our Neural Network to not just learn and compute a linear function but something more complicated than that.

COVID19 Time Series Forecasting using LSTM-RNN (11)

We have chosen the optimizer to be “adam” . Adam is an optimization algorithm that can be used instead of the classical stochastic gradient descent procedure to update network weights iterative based in training data.It is appropriate for problems with very noisy/or sparse gradients, Computationally very efficient and requires less hyper-parameter tuning.

Our loss function is “mean squared Error loss/quadratic loss”. MSE is the sum of squared distances between our target variable and predicted values.

COVID19 Time Series Forecasting using LSTM-RNN (12)

validation set

we have also introduced validation set to calculate loss and MAPE. Recall, validation set is equal to number of step size i.e. 5 while batches are of the shape [(t1,t2,t3,t4,t5),(t6)] i.e [(1,5,1),(1,1)]. So total data points required would be 6 i.e. one of the data point we have to take from the train set.

COVID19 Time Series Forecasting using LSTM-RNN (13)

Training the model

We have early stopping mechanism of keras which stops training when a monitored quantity has stopped improving. In other words, if val_loss doesn’t reduce in upcoming epochs the training stops. We have given patience to be 20 i.e we have to not stop training untill 20 iterations even there is no improvement in quality. There is an important flag called “restore_best_weights” which takes the best weight from the iterations(where val loss is minimum).

COVID19 Time Series Forecasting using LSTM-RNN (14)

Step 4 : Model Performance

The model looks good as training loss and validation loss are overlapping at a minimal value. Also, both the curves are getting flattened as the number of epochs increases.

COVID19 Time Series Forecasting using LSTM-RNN (15)

If we look at both the losses separately

COVID19 Time Series Forecasting using LSTM-RNN (16)

COVID19 Time Series Forecasting using LSTM-RNN (17)

both the graphs tend to follow each other after getting trained for a couple of epochs and saturates at similar values afterwards. This validates our model being well trained.

Step 5: Forecast

We will forecast the number of confirmed cases in Iran for validation set and next 7 days from today.

COVID19 Time Series Forecasting using LSTM-RNN (18)

The output is a normalized data so we apply inverse transformations on the following.

COVID19 Time Series Forecasting using LSTM-RNN (19)

Restructuring the array to a readable pandas dataframe.

COVID19 Time Series Forecasting using LSTM-RNN (20)

Plot the curve for original and predicted data

COVID19 Time Series Forecasting using LSTM-RNN (21)

Mean Absolute Percentage Error(MAPE)

COVID19 Time Series Forecasting using LSTM-RNN (22)

COVID19 Time Series Forecasting using LSTM-RNN (23)

And accuracy would be 100-MAPE = ~ 93%

Calculation of prediction interval(95% confidence level,CL)

for 95% CL, t-multiplier is 1.96 which is calculated from degree of freedom of the sample and CL required

t-multiplier * standard_deviation gives the magnitude of interval.

and min and max range is given by :

min = value-interval

max = value + interval

COVID19 Time Series Forecasting using LSTM-RNN (24)

COVID19 Time Series Forecasting using LSTM-RNN (25)

Conclusion

The above approach can be replicated for number of death and recovered cases for each country. Just we have to run the model in a loop and get the predictions.

However, I read an article on The Washington Post that temperature and humidity may slow down the spread of the virus. Which explains the reason that spread remains relatively low in countries like Cambodia , Laos and Vietnam as compared to Europe or US.

So, Probably, next time I would try to model a multivariate time series with variables like Temperature , Humidity and Population Density.

Feel free to ask in the comment section in case of any suggestions/queries.

I would like to thank Jose Portilla for his amazing Tutorial on Time Series on Udemy.

Github Repo

FAQs

COVID19 Time Series Forecasting using LSTM-RNN? ›

Pathan et al. [17] used RNN based long short term memory (LSTM) to design a model and forecast the mutation rate of Covid-19. The nucleotide mutation rate of 400^th patient is predicted with a root mean square error of 0.06 and 0.04 for training and testing processes respectively.

Can LSTM be used for Covid 19 forecasting? ›

A deep-learning approach using Bi-LSTM architecture and open-source data can be used as a starting point for forecasting the new daily number of COVID-19 cases 14 days in advance and fewer variables could potentially be used without impacting prediction accuracy.

Read On ›

Can LSTM be used for time series forecasting? ›

Due to the model's ability to learn long term sequences of observations, LSTM has become a trending approach to time series forecasting.

Keep Reading ›

Can RNN be used for time series forecasting? ›

RNNs, with their ability to remember past information, are naturally suited for such tasks. They can capture complex relationships, seasonal patterns, and even anomalies in the data, making them a strong candidate for time-series forecasting.

Why LSTM is better than RNN in time series? ›

LSTMs are more sophisticated and capable of handling long-term dependencies, making them the preferred choice for many sequential data tasks. Check out the comparison of LSTM vs RNN in the below table.

Read On ›

When should LSTM be used? ›

A long short-term memory network is a type of recurrent neural network (RNN). LSTMs are predominantly used to learn, process, and classify sequential data because these networks can learn long-term dependencies between time steps of data.

Learn More ›

Why use LSTM over CNN? ›

A CNN processes sequence data by applying sliding convolutional filters to the input. A CNN can learn features from both spatial and time dimensions. An LSTM network processes sequence data by looping over time steps and learning long-term dependencies between time steps.

Which algorithm is best for time series forecasting? ›

Autoregressive Integrated Moving Average (ARIMA) is a commonly used statistical algorithm for time-series forecasting.

See Details ›

What is the best optimizer for time series forecasting? ›

Long Short Term Memory (LSTM)

RNNs are a powerful type of artificial neural network that can internally maintain memory of the input. This makes them particularly suited for solving problems involving sequential data like a time series.

What is CNN LSTM for time series forecasting? ›

CNN-LSTM is a hybrid model for univariate time series forecasting. The benefit of this model is that the model can support very long input sequences that can be read as blocks or subsequences by the CNN model, then pieced together by the LSTM model.

Discover More ›

What are the disadvantages of LSTM in RNN? ›

Disadvantages: Training complexity: LSTMs are more complex than traditional RNNs, which can make them more difficult to train. This complexity can also make it harder to interpret and debug an LSTM network. Overfitting: LSTMs are prone to overfitting, especially when working with small datasets.

Show Me More ›

Is LSTM good for time series prediction? ›

Since LSTMs have a memory cell, they specifically work well with time series data. These models are great because they can easily process an entire sequence of data and use the memory cell, which makes the network able to effectively associate memories with the current input.