1. Introduction
In this module we discuss anomaly detection in the QTDB 0606 ECG dataset. The database record can be downloaded from the PhysioNet QT Database and converted into text format with
rdsamp -r sele0606 -f 120.000 -l 60.000 -p -c | sed -n '701,3000p' >0606.csv
(assuming rdsamp from the WFDB toolkit is installed on your system). We use the second column of this file; a ready-made copy ships with GrammarViz as data/ecg0606_1.csv. This is our dataset overview:
We know that the third heartbeat of this dataset contains the true anomaly, as discussed in the HOT SAX paper by Eamonn Keogh, Jessica Lin, and Ada Fu. Note that the authors were specifically interested in finding anomalies shorter than a regular heartbeat, following a suggestion by a domain expert: “…We conferred with cardiologist, Dr. Helga Van Herle M.D., who informed us that heart irregularities can sometimes manifest themselves at scales significantly shorter than a single heartbeat…” Figure 13 of the paper further explains the nature of this anomaly:
2. Variable-length exact anomaly discovery with RRA
Load the dataset with the “Load data” button and set the SAX discretization parameters to sliding window 100, PAA 3, and alphabet 3. Click “Discretize” to infer a grammar describing the input time series. Click the “Find anomalies” button to perform the discovery, then select the “GrammarViz anomalies” tab and choose the top-ranked anomaly (#0):
This highlights the grammar rule that coincides with the true anomaly. Note the variable lengths: the top anomaly is 110 points long, while the next one spans 208 points.
3. Variable-length approximate anomaly discovery with the rule density curve
We use the same dataset. First, change the GI algorithm from Sequitur to Re-Pair: open “Settings”, choose the “GI Implementation” tab, and toggle the Re-Pair algorithm:
Click “Save” to apply. Set the SAX discretization parameters to sliding window 100, PAA 5, and alphabet 5, and click “Discretize” to infer a Re-Pair grammar. Then click the “Rules density” button:
The light (low-density) region near position 480 clearly identifies the true anomaly: few grammar rules cover it, which is exactly what makes it rare.
4. Discussion
Note that both techniques shown above yield rare subsequences of variable length. Two factors make this possible: the numerosity reduction embedded in the discretization step, and the nature of GI algorithms, which build their rules from long-range correlations in the input.