It is our solution of 2018 MCM contest, our result got Meritorious award (first prize). During the contest, I am responsible for constructing models and writing the paper.
I have put all the codes and related stuff in GitHub
Here is the final paper:
Abstract of Paper
In this paper, we construct two evaluation systems: Renewable Energy Indicator (REI) and Network Instability Indicator (NII). By using data mining methods and developing two models, we succeed in visualizing, characterizing, evaluating and predicting the energy flow network and the use of renewable energy.\par
First, we process the data. We do data screening to select two groups of variables to characterize the energy profile. As some data is missing, we do imputation based on existing data and data from other reliable sources. The data constructing key indicators are normalized with all fifty states. To illustrate energy profile, we also do data visualization with vivid diagrams like Sankey diagram and radar diagram.\par
Second, we construct two evaluation systems REI and NII to trace and compare the energy profiles of four states and their evolution. NII is derived from a Constrained-Ridge model to characterize the structural stability of energy flow. To specify on renewable energy, we define REI with five key indicators. We use combined Analytic Hierarchy Process(AHP) and Entropy Weight Method(EWM) to construct REI, which characterizes the profile for use of cleaner, renewable energy. We then compare REI and NII of four states and Arizona performs best.\par
Third, we use several time series models to predict the energy profile of each state in the future. Structural profile of energy flow NII is predicted with Constrained-Ridge model, and renewable energy profile REI is predicted with Autoregressive Integrated Moving Average Model(ARIMA) and Long Short Term Memory(LSTM). Based on the comparison and prediction, we determine the renewable energy usage target for four states and propose several feasible actions.\par
Finally, we conduct sensitivity analysis of our model. We try several machine learning models to predict 5 key indicators, and use independent validation datasets to evaluate their performances. We also do a perturb-and-profile test to evaluate Constrained-Ridge model’s power to characterize network profile.
Our model is reasonable and legible with theoretical and data support. The model can be easily applied to characterize the energy profile of renewable energy and energy flow network after data training.
Data
Used data in data file, produced data in produced data
Codes and method
For quick work in limited time, all codes are written in jupyter notebook, which is a good interactive environment easier to plot and analyze data.
We have implemented some algorithm and some visualization method to analyze data. We use AHP and EWM to construct REI, use ARIMA, Machine Learning model and Deep Learning model to predict time series data. We construct a Restricted Ridge model, using optimization method to depict the network relationship. We use sankey plot, radar plot, and many ordinary plots to visualize data vividly.
Key point: manually feature selection
We use a network to extract key features to depict the energy profile. The network is visualized through sankey plot.