�ݺ�ߣ

Time Series Data Mining -
from PhD to Startup
Peter Laurinec
October 27, 2018

Highlights
Time series data mining - from PhD to start-up:
1/27

Highlights
• Problems and solutions for using large amount of long
time series (TS),
1/27

Highlights
time series (TS),
• TS data mining methods,
1/27

Highlights
time series (TS),
• PhD. study thesis - combining and developing TS data
mining methods,
1/27

Highlights
time series (TS),
mining methods,
• TSrepr R package - TS representations,
1/27

Highlights
time series (TS),
mining methods,
• Work after Phd - energy start-up,
1/27

Highlights
time series (TS),
mining methods,
• Differences and my thoughts,
1/27

Highlights
time series (TS),
mining methods,
• Differences and my thoughts,
• What we do there...
1/27

Time Series Data in Energetics
Smart metering
• Measuring electricity consumption or production
(photovoltaic panels) from every consumer or
producer (together prosumer) every 5, 15, or 30
minutes,
• This creates a large amount of time series data,
• 3 years of data from consumer 96*365*3 =
105120...from 10 thousand consumers... > 1 billion rows
of multiple columns,
• Smart grid - set of consumers and producers,
2/27

Time Series Data in Energetics
Smart metering
• Measuring electricity consumption or production
(photovoltaic panels) from every consumer or
producer (together prosumer) every 5, 15, or 30
minutes,
• This creates a large amount of time series data,
• 3 years of data from consumer 96*365*3 =
105120...from 10 thousand consumers... > 1 billion rows
of multiple columns,
• Smart grid - set of consumers and producers,
Characteristics:
• High-dimensionality,
• Multiple seasonalities (daily, weekly, yearly),
• Large amount of stochastic factors as: weather,
holidays, black-outs, changes on market etc. 2/27

Examples of Consumers TS
0 250 500 750 1000
20
40
60
0
5
10
15
0.0
0.5
1.0
1.5
2.0
0
1
2
Time (3 weeks)
ElectricityConsumption(kW)
3/27

Typical Use Cases
• Forecasting el. consumption or production - market
planning, black-outs prevention etc.,
4/27

Typical Use Cases
• Extract typical proﬁles of consumption - changes in tariffs,
create new ones etc.,
4/27

Typical Use Cases
• Optimizing electricity consumption of some consumer,
4/27

Typical Use Cases
• Optimizing whole smart grid,
4/27

Typical Use Cases
• Monitoring smart grid,
4/27

Typical Use Cases
• Monitoring smart grid,
• Anomaly detection.
4/27

TS Data Mining Methods
• Methods for working with TS:
5/27

• TS representations,
5/27

• TS distance measures,
5/27

• Tasks:
5/27

• Tasks:
• TS classiﬁcation,
• TS clustering,
• TS forecasting,
• TS anomaly detection,
• TS indexing.
5/27

PhD. Thesis Goals
• The thesis had the goal to investigate, in the broader
context, the usage of time series data mining (analysis)
methods in order to improve the predictive performance
of machine learning methods and its combinations.
6/27

PhD. Thesis Goals
• The thesis had the goal to investigate, in the broader
context, the usage of time series data mining (analysis)
methods in order to improve the predictive performance
of machine learning methods and its combinations.
• In more detail, the goal was to investigate the usage of
various time series representations for seasonal time
series, clustering, and forecasting methods for electricity
consumption forecasting accuracy improvement.
6/27

I. Time Series Representations
8/27

What can we do for solving problems with high-dimensional
TS?
9/27

TS?
• Use time series representations!
9/27

TS?
• Use time series representations!
They are excellent to:
• Reduce memory load.
• Accelerate subsequent machine learning algorithms.
• Implicitly remove noise from the data.
• Emphasize the essential characteristics of the data.
• Help to ﬁnd patterns in data (or motifs).
9/27

4.00
4.25
4.50
4.75
0 500 1000
Time
Load
4.0
4.2
4.4
4.6
4.8
0 50 100 150
Length
Load
4.0
4.2
4.4
4.6
4.8
0 50 100 150
Length
Load
10/27

4.00
4.25
4.50
4.75
0 500 1000
Time
Load
4.2
4.3
4.4
4.5
4.6
0 10 20 30 40 50
Length
Load
4.2
4.4
4.6
0 100 200 300
Length
Load
11/27

I. Time Series Representations 1
I used TS representations for:
1
Laurinec P., Lucká M., Lecture Notes in Engineering and Computer Science:
Proceedings of The World Congress on Engineering and Computer Science 2016.
12/27

• Dimensionality reduction (curse of dimensionality),
1
12/27

• Emphasising the main characteristics of data,
1
12/27

• Emphasising the main characteristics of data,
• More accurate clustering of consumers TS to create more
predictable (forecastable) groups of aggregated TS of
electricity consumption.
1
12/27

Clustered TS Representations
17 18 19 20
13 14 15 16
9 10 11 12
5 6 7 8
1 2 3 4
0 20 40 0 20 40 0 20 40 0 20 40
−1
0
1
2
3
−2
−1
0
1
2
3
−2
0
2
−2
0
2
−2
−1
0
1
2
3
−1
0
1
2
−3
−2
−1
0
1
2
−2
0
2
−2
−1
0
1
2
0
2
4
−2
0
2
4
−1
0
1
2
−2
0
2
4
−2
0
2
−2
−1
0
1
2
3
−1
0
1
2
−1
0
1
2
3
0
2
4
−2
−1
0
1
2
−2
−1
0
1
2
Length
RegressionCoefficients
13/27

Groups of Aggregated TS
17 18 19 20
13 14 15 16
9 10 11 12
5 6 7 8
1 2 3 4
0 250 500 750 1000 0 250 500 750 1000 0 250 500 750 1000 0 250 500 750 1000
−0.5
0.0
0.5
1.0
1.5
−1.5
−1.0
−0.5
0.0
0.5
1.0
−0.50
−0.25
0.00
0.25
−1
0
1
−1.0
−0.5
0.0
0.5
−1.0
−0.5
0.0
0.5
1.0
−1.0
−0.5
0.0
0.5
−1.0
−0.5
0.0
0.5
1.0
−1.0
−0.5
0.0
0.5
1.0
−0.5
0.0
0.5
1.0
0
1
0
1
2
−0.5
0.0
0.5
−1.0
−0.5
0.0
0.5
1.0
−0.5
0.0
0.5
1.0
1.5
0
1
0
1
2
0
1
2
3
4
5
−1.0
−0.5
0.0
0.5
1.0
−1
0
1
Time
NormalizedLoad
14/27

TSrepr
TSrepr - CRAN2, GitHub3
• R package for time series representations computing
• Large amount of various methods are implemented
• Several useful support functions are also included
• Easy to extend and to use
data <- rnorm(1000)
repr_paa(data, func = median, q = 10)
2
https://CRAN.R-project.org/package=TSrepr
3
https://github.com/PetoLau/TSrepr/
15/27

All type of time series representations methods are implemented, so far these:
• PAA - Piecewise Aggregate Approximation ( repr_paa )
• DWT - Discrete Wavelet Transform ( repr_dwt )
• DFT - Discrete Fourier Transform ( repr_dft )
• DCT - Discrete Cosine Transform ( repr_dct )
• PIP - Perceptually Important Points ( repr_pip )
• SAX - Symbolic Aggregate Approximation ( repr_sax )
• PLA - Piecewise Linear Approximation ( repr_pla )
• Mean seasonal proﬁle ( repr_seas_profile )
• Model-based seasonal representations based on linear model ( repr_lm )
• FeaClip - Feature extraction from clipping representation ( repr_feaclip )
Additional useful functions are implemented as:
• Windowing ( repr_windowing )
• Matrix of representations ( repr_matrix )
• Normalisation functions - z-score ( norm_z ), min-max ( norm_min_max )
16/27

Usage of TSrepr
mat <- "some matrix with lot of time series"
mat_reprs <- repr_matrix(mat, func = repr_lm,
args = list(method = "rlm", freq = c(48, 48*7)),
normalise = TRUE, func_norm = norm_z)
mat_reprs <- repr_matrix(mat, func = repr_feaclip,
windowing = TRUE, win_size = 48)
clustering <- kmeans(mat_reprs, 20)
17/27

Simple Extensibility of TSrepr
Example #1:
library(moments)
data_ts_skew <- repr_paa(data, q = 48, func = skewness)
Example #2:
repr_fea_extract <- function(x)
c(mean(x), median(x), max(x), min(x), sd(x))
data_fea <- repr_windowing(data,
win_size = 100, func = repr_fea_extract)
18/27

II. Time Series Clustering
19/27

II. Clustering Multiple Data Streams 4
Motivation:
4
https://github.com/PetoLau/ClipStream/
20/27

Motivation:
• Deal with velocity of data coming,
4
20/27

Motivation:
• Dynamic change of number of clusters,
4
20/27

Motivation:
• Automatic anomaly detection (anomalous consumers),
4
20/27

Motivation:
• Automatic change detection.
4
20/27

Motivation:
• Automatic change detection.
Approach:
• Take advantage of incrementality of clipped representation
(windowing),
• Fast detection of anomalous consumers from extracted features from
clipping,
• Change detection by Anderson-Darling test.
4
20/27

III. Time Series Forecasting
22/27

Large number of methods suitable for forecasting:
• Time series analysis methods:
• ARIMA,
• Exponential smoothing,
• Theta,
23/27

Large number of methods suitable for forecasting:
• Time series analysis methods:
• ARIMA,
• Exponential smoothing,
• Theta,
• Regression methods:
• Linear regression, GAM,
• SVR, Gaussian process,
• Regression trees, Bagging, Random Forest, Boosting,
• Artiﬁcial Neural Networks.
23/27

III. Time Series Forecasting 5
Finding the most suitable forecasting methods with
clustering...
• STL+ARIMA, Exponential smoothing, Tree-based methods,
Advanced ANNs (S2S + LSTM nets).
5
https://github.com/PetoLau/TSMedianBasedEnsembleLearning/,
https://github.com/PetoLau/UnsupervisedEnsembles/,
https://github.com/PetoLau/DensityEnsembles/
24/27

III. Time Series Forecasting 5
Finding the most suitable forecasting methods with
clustering...
• STL+ARIMA, Exponential smoothing, Tree-based methods,
Advanced ANNs (S2S + LSTM nets).
The problem of choosing the most suitable method among the
set of methods...
Solution:
• Ensemble learning - combining forecasts.
5
https://github.com/PetoLau/TSMedianBasedEnsembleLearning/,
https://github.com/PetoLau/UnsupervisedEnsembles/,
https://github.com/PetoLau/DensityEnsembles/
24/27

Life after PhD
• I was happy to be hired by start-up PowereX.
• We solve problems strongly related with my thesis.
25/27

Life after PhD
• I was happy to be hired by start-up PowereX.
• We solve problems strongly related with my thesis.
PowereX
• P2P energy sharing - commodity and also capacity,
• Analysis of consumers smart meter data,
• Forecasting and modelling maximal load (hourly, daily,
etc.).
25/27

Differences between PhD and Business
PhD:
• Strong focus on accuracy measures - ↓ % of Mean
Absolute Percentage Error, or internal validation indexes
for clustering...
26/27

PhD:
for clustering...
• Many times working with poor academic datasets.
26/27

PhD:
for clustering...
Business:
• Finding real value for customers,
• Accuracy is not that important,
• Working on real rich data.
26/27

PhD:
for clustering...
Business:
• Finding real value for customers,
• Accuracy is not that important,
• Working on real rich data.
But...they are also related and need each other...
26/27

Conclusions
TS data mining:
27/27

Conclusions
TS data mining:
• TS representations are our ﬁends in clustering,
forecasting, classiﬁcation etc.,
27/27

Conclusions
TS data mining:
• Implemented in TSrepr package,
27/27

Conclusions
TS data mining:
• PhD study is great practice before work.
27/27

Conclusions
TS data mining:
• PhD study is great practice before work.
Questions: Peter Laurinec laurinec.peter@gmail.com
Code: https://github.com/PetoLau/
More research: https://petolau.github.io/research
Blog: https://petolau.github.io
27/27

�ݺ�ߣ

Time Series Data Mining - from PhD to Startup

Convert to study guideBETA

More Related Content

Time Series Data Mining - from PhD to Startup