I’ve recently been experimenting with Keras neural networks for basic example problems and I’ve been wanting to try and apply one to real-life data. While exploring I came across the NOAA’s data about carbon dioxide concentrations.
They have been measuring the CO2 concentration in Mauna Loa, Hawaii regularly since 1958. As well as this they publish annual means in a CSV file – perfect for Python!
Normalisation
In order to carry out this project successfully I had to learn about normalisation. This involves reducing the magnitude of the data. The year data is originally measured in the thousands – 2019, for example. However, neurons deal with much smaller values – the sigmoid activation function that I’m using ranges from 0 to 1.
As a result, if we passed it a large number such as 2019 it would have to find a weight that is small enough to convert this number to a more useful magnitude. For example, 2019 * 0.0005 is 1.0095.
However, it takes a long time for the neural network to find a weight of the right size because for all it knows the weight could end up anywhere between, say, -100 and 100. With such a large range of numbers to search it takes a long time to reach such a small, precise weight.
The solution to this problem is to normalise the data before we train the network. This involves changing the data so that it’s between 0 and 1. For example, the smallest number, 1958, will be 0 whereas 2018 will be 1. The data is now in a much more useful range. Additionally, as all of the features the network is using will have the same range it is easier for it to find correlations between them.
In order to normalise the data I used the min max scaler from Scikit-Learn.
Design
I decided to use the following layers:
- 500-neuron layer
- Sigmoid activation function
- 200-neuron layer
- Sigmoid activation function
- 200-neuron layer
- 1-neuron layer
The large layers at the top give the network enough neuron units to find correlations and the final single neuron layer gives us the single numeric output that we want.
Results
I was pretty pleased with the results. The predictions look reasonable and it fits the actual historic data quite well. However, don’t panic! This is not necessarily the true way that the CO2 concentration is going to change – this is if emissions continue to grow as they have done since 1958. Hopefully we’ll be able to cut down so that the curve stops being exponential.
The code
# -*- coding: utf-8 -*-
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers.core import Dense, Activation
import numpy
import matplotlib.pyplot as plt
numpy.random.seed(20)
#read csv file and store the data in a Pandas DataFrame
data = pd.read_csv('C:/Users/Brychan/Downloads/co2_annmean_mlo.txt',
sep=' +', header=None, comment='#', engine='python',
names=['year','mean','unc'])
data = data.drop(columns=['unc']) #remove uncertainty data as it is not needed
print(data[0:3])
print(data[-3:])
input("Press enter to continue...")
X=data.drop(columns=['mean']) #get the years from the dataframe
X=X.values
yearScaler = MinMaxScaler()
X = yearScaler.fit_transform(X) #normalise the years
Y=data.drop(columns=['year']) #get the concentrations from the dataframe
Y=Y.values
concentrationScaler = MinMaxScaler()
Y = concentrationScaler.fit_transform(Y) #normalise the concentrations
model=Sequential() #create the model
model.add(Dense(500, input_dim=1, kernel_initializer='normal')) #add a layer of 500 neurons
model.add(Activation('sigmoid')) #add the sigmoid activation function between the layers
model.add(Dense(200, kernel_initializer='normal'))
model.add(Activation('sigmoid'))
model.add(Dense(200, kernel_initializer='normal'))
model.add(Dense(1))
#use the mean-squared-error loss function and the Adam optimiser
model.compile(loss='mse', optimizer='adam')
model.summary()
model.fit(X, Y, epochs=10000) #train for 10000 epochs on the data from the csv
#add the years 2019 - 2060 to the array of years to allow the model to make predictions for them
for year in range(2019,2061):
X=numpy.append(X, yearScaler.transform(numpy.array([year]).reshape(-1,1)))
#add 'not a number' so that nothing is shown on the graph under 'actual' for that year.
Y=numpy.append(Y, numpy.nan)
predicted=model.predict(X) #make predictions for all years from 1958 - 2060
#'descale' the data so that it is normal years and concentrations rather than decimals
yearsDescaled=yearScaler.inverse_transform(X.reshape(-1,1))
predictedDescaled=concentrationScaler.inverse_transform(predicted.reshape(-1,1))
actualDescaled=concentrationScaler.inverse_transform(Y.reshape(-1,1))
#plot the predictions and the historic data on a graph
plt.plot(yearsDescaled, predictedDescaled, color='red', label='predicted')
plt.plot(yearsDescaled, actualDescaled, color='blue', label='actual', dashes=[6, 2])
plt.ylabel('CO2 concentration') #add a label to the y axis
plt.legend() #add a legend
plt.show()
year=int(input("Predict for year (or -1): ")) #predict for a user-specified year
while year != -1:
year = yearScaler.transform(year)
predicted=model.predict([[year]])[0][0]
print(concentrationScaler.transform(predicted), 'ppm')
year=int(input("Predict for year: "))
Brychan Thomas' projects 