-
Notifications
You must be signed in to change notification settings - Fork 9
Expand file tree
/
Copy pathREADME.Rmd
More file actions
130 lines (92 loc) · 4.73 KB
/
README.Rmd
File metadata and controls
130 lines (92 loc) · 4.73 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
<!-- rmarkdown v1 -->
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-"
)
```
# stray {STReam AnomalY} <img src="man/figures/logo.png" align="right" height="150" />
[](https://www.repostatus.org/#wip)
[](https://www.gnu.org/licenses/old-licenses/gpl-2.0.html)
[](https://travis-ci.org/pridiltal/stray)
---
[](https://cran.r-project.org/)
[](https://cran.r-project.org/package=stray)
[](commits/master)
---
[)`-yellowgreen.svg)](/commits/master)
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-"
)
```
Anomaly Detection in High Dimensional Data Space
This package is a modification of [HDoutliers package](https://CRAN.R-project.org/package=HDoutliers). The HDoutliers algorithm is a powerful unsupervised algorithm for detecting anomalies in high-dimensional data, with a strong theoretical foundation. However, it suffers from some
limitations that significantly hinder its performance level, under certain circumstances. In this package, we propose an algorithm that addresses these limitations. We define an anomaly as an
observation that deviates markedly from the majority with a large distance gap. An approach based on extreme value theory is used for the anomalous threshold calculation.
A companion paper to this work is available [here](https://arxiv.org/pdf/1908.04000.pdf). Using various
synthetic and real datasets, we demonstrate the wide applicability and usefulness of our algorithm, which we call the stray algorithm. We also demonstrate how this algorithm can
assist in detecting anomalies present in other data structures using feature engineering. We show the situations where the stray algorithm outperforms the HDoutliers algorithm both in
accuracy and computational time.
This package is still under development and this repository contains a development version of the R package *stray*.
## Installation
You can install the released version of stray from [CRAN](https://CRAN.R-project.org) with:
```{r cran-installation, eval = FALSE}
install.packages('stray', dependencies = TRUE)
```
You can install stray from github with:
```{r gh-installation, eval = FALSE}
# install.packages("devtools")
devtools::install_github("pridiltal/stray")
```
## Example
### One dimensional data set with one outlier
```{r onedim, echo =TRUE, eval = TRUE}
library(stray)
require(ggplot2)
set.seed(1234)
data <- c(rnorm(1000, mean = -6), 0, rnorm(1000, mean = 6))
outliers <- find_HDoutliers(data, knnsearchtype = "brute")
names(outliers)
display_HDoutliers(data, outliers)
```
### Two dimensional dataset with 8 outliers
```{r twodim, echo =TRUE, eval = TRUE}
set.seed(1234)
n <- 1000 # number of observations
nout <- 10 # number of outliers
typical_data <- matrix(rnorm(2*n), ncol = 2, byrow = TRUE)
out <- matrix(5*runif(2*nout,min=-5,max=5), ncol = 2, byrow = TRUE)
data <- rbind(out, typical_data )
outliers <- find_HDoutliers(data, knnsearchtype = "brute")
display_HDoutliers(data, outliers)
```
### Three dimensional dataset with 2 outliers
For data with more than two dimensions, two dimensional scatterplot is produced using the first two pricipal components.
```{r datad3, echo= TRUE, eval= TRUE}
data <- rbind(matrix(rnorm(144), ncol = 3), c(10,12,10),c(3,7,10))
output <- find_HDoutliers(data, knnsearchtype = "brute")
display_HDoutliers(data, out = output)
```
gitn
More examples are available from our paper [Anomaly Detection in High Dimensional Data](https://www.monash.edu/business/ebs/research/publications/ebs/wp20-2019.pdf)
```{r dataa, echo =TRUE, eval = TRUE}
outliers<-find_HDoutliers(data_c[,1:2], knnsearchtype= "brute")
p <- display_HDoutliers(data_c[,1:2], outliers)+
ggplot2::ggtitle("data_c")
print(p)
```
```{r datad, echo =TRUE, eval = TRUE}
outliers<-find_HDoutliers(data_d[,1:2], knnsearchtype= "brute")
p <- display_HDoutliers(data_d[,1:2], outliers)+
ggplot2::ggtitle("data_d")
print(p)
```