Skip to content

Commit 744a83f

Browse files
TomansionTomansion
authored
Debiai gui parameters
2 parents 7942420 + 9e4e27a commit 744a83f

28 files changed

Lines changed: 530 additions & 196 deletions

‎documentation/.vuepress/config.js‎

Lines changed: 29 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,35 @@ module.exports = {
9494
],
9595
},
9696

97+
// Custom algorithms
98+
{
99+
title: "Custom algorithms",
100+
path: "/customAlgorithms/",
101+
collapsable: false,
102+
initialOpenGroupIndex: -1,
103+
children: [
104+
{
105+
title: "How to use algorithms",
106+
path: "/customAlgorithms/howToUseAlgorithms.md",
107+
},
108+
{
109+
title: "Integrated algorithms",
110+
path: "/customAlgorithms/integratedAlgorithms.md",
111+
},
112+
{
113+
title: "Algo-providers",
114+
path: "/customAlgorithms/algoProviders/",
115+
children: [
116+
"/customAlgorithms/algoProviders/easyAlgoProvider.md",
117+
"/customAlgorithms/algoProviders/algoProviderTemplates.md",
118+
"/customAlgorithms/algoProviders/customImplementation.md",
119+
"/customAlgorithms/algoProviders/implementInBackend.md",
120+
"/customAlgorithms/algoProviders/addingAlgoProviders.md",
121+
],
122+
},
123+
],
124+
},
125+
97126
// Dashboard
98127
{
99128
title: "Dashboard",
@@ -130,16 +159,6 @@ module.exports = {
130159
path: "/dashboard/layouts/",
131160
collapsable: true,
132161
},
133-
{
134-
title: "Custom algorithms",
135-
path: "/dashboard/algoProviders/",
136-
collapsable: true,
137-
children: [
138-
"/dashboard/algoProviders/algoProviders.md",
139-
"/dashboard/algoProviders/implementInBackend.md",
140-
"/dashboard/algoProviders/integratedAlgorithms.md",
141-
],
142-
},
143162
{
144163
title: "Data export",
145164
path: "/dashboard/dataExport/",

‎documentation/.vuepress/public/install/addBlock.svg‎

Lines changed: 4 additions & 0 deletions
Loading

‎documentation/.vuepress/public/install/gears.svg‎

Lines changed: 50 additions & 0 deletions
Loading

‎documentation/.vuepress/public/install/heartFile.svg‎

Lines changed: 4 additions & 0 deletions
Loading

‎documentation/.vuepress/public/install/terminal.svg‎

Lines changed: 5 additions & 0 deletions
Loading

‎documentation/customAlgorithms/README.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Adding custom algorithms to DebiAI
2+
3+
Projects have their own specific needs and sometimes the algorithms that are available in DebiAI are not enough. That's why DebiAI allows you to add your own algorithms.
4+
5+
Adding an algorithm to DebiAI allows you to use it with the data directly from the dashboard and analyze the results like any other project data.
6+
7+
Different kinds of algorithms can be added to DebiAI, ranging from data quality to metrics calculation, from data transformation using machine learning models to anomaly detection.
8+
9+
**Learn more:**
10+
11+
<LinkableChoices :choices="[
12+
{
13+
title: 'How to use algorithms?',
14+
description: 'Learn how to use an algorithm from the dashboard and analyze the results',
15+
imageLink: '/getStarted/guide.svg',
16+
linkDestination: './howToUseAlgorithms',
17+
},
18+
{
19+
title: 'Integrated algorithms',
20+
description: 'DebiAI comes with a set of integrated algorithms, learn more about them',
21+
imageLink: '/install/gears.svg',
22+
linkDestination: './integratedAlgorithms',
23+
},
24+
{
25+
title: 'Add your own algorithms',
26+
description: 'Learn about Algo-providers and how to add your own algorithms to DebiAI',
27+
imageLink: '/install/addBlock.svg',
28+
linkDestination: './algoProviders',
29+
},
30+
]"
31+
/>

‎documentation/customAlgorithms/algoProviders/README.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# DebiAI Algo-providers
2+
3+
An Algo-provider is a service that can respond to the algorithms requests of DebiAI. DebiAI will interact with your Algo-provider in two ways:
4+
5+
- For getting the list of available algorithms
6+
- For running an algorithm and getting the results
7+
8+
## Creating an Algo-provider
9+
10+
We offer different ways to create an Algo-provider:
11+
12+
<LinkableChoices :choices="[
13+
{
14+
title: 'Python module',
15+
description: 'Create an Algo-provider from a single Python function',
16+
imageLink: '/install/python.svg',
17+
linkDestination: '/customAlgorithms/algoProviders/easyAlgoProvider',
18+
tag: 'Recommended'
19+
},
20+
{
21+
title: 'Algo-provider templates',
22+
description: 'Generate an Algo-provider using a pre-built template',
23+
imageLink: '/install/template.svg',
24+
linkDestination: '/customAlgorithms/algoProviders/algoProviderTemplates',
25+
},
26+
{
27+
title: 'Custom implementation',
28+
description: 'Build an Algo-provider from scratch',
29+
imageLink: '/install/build.svg',
30+
linkDestination: '/customAlgorithms/algoProviders/customImplementation',
31+
},
32+
{
33+
title: 'Modify the DebiAI backend',
34+
description: 'Add your algorithm next to the integrated algorithms',
35+
imageLink: '/install/heartFile.svg',
36+
linkDestination: '/customAlgorithms/algoProviders/implementInBackend',
37+
tag: 'Good for contributors'
38+
},
39+
]"
40+
/>
41+
42+
## Algorithms description
43+
44+
You will need to describe your algorithm in a Json format and then create the implementation of your algorithm. After that, DebiAI will be able to understand your algorithm and run it.
45+
46+
Here is, for example, the description of a simple moving average algorithm:
47+
48+
```json
49+
{
50+
"id": "moving_average",
51+
"name": "Moving average",
52+
"description": "Calculate the moving average of a data.",
53+
"tags": ["calculations"],
54+
"author": "DebiAI",
55+
"version": "1.0.0",
56+
"inputs": [
57+
{
58+
"name": "data",
59+
"description": "The data to calculate the moving average on.",
60+
"type": "array",
61+
"arrayType": "number",
62+
"lengthMin": 1,
63+
"lengthMax": 100000
64+
},
65+
{
66+
"name": "periods",
67+
"description": "The number of periods to calculate the moving average on.",
68+
"type": "number",
69+
"default": 3,
70+
"min": 1,
71+
"max": 100
72+
}
73+
],
74+
"outputs": [
75+
{
76+
"name": "moving_average",
77+
"description": "The moving average of the data. Same length as the data",
78+
"type": "array",
79+
"arrayType": "number"
80+
}
81+
]
82+
}
83+
```
84+
85+
As you can see, the input and output of the algorithm are described. This description is used by DebiAI to understand how to run the algorithm and how to display it in the dashboard and what kind of data it needs.
86+
87+
To learn more about the algorithms descriptions, you can read the [Algorithm description documentation](https://github.com/debiai/algo-provider-python-template/blob/main/algo-api/README.md).

‎documentation/customAlgorithms/algoProviders/addingAlgoProviders.md‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# Adding Algo-providers to DebiAI
2+
3+
Once you have created and deployed your Algo-provider, you can add it to DebiAI. We offer multiple ways to do so:
4+
5+
<LinkableChoices :choices="[
6+
{
7+
title: 'With debiai-gui',
8+
description: 'If using the cli provided by our python module',
9+
imageLink: '/install/terminal.svg',
10+
elementIdDestination: 'with-debiai-gui'
11+
},
12+
{
13+
title: 'From the dashboard',
14+
description: 'Easiest method, but not persistent',
15+
imageLink: '/install/screen.svg',
16+
elementIdDestination: 'from-the-dashboard'
17+
},
18+
{
19+
title: 'Env. variables',
20+
description: 'Best for Docker deployments',
21+
imageLink: '/install/world.svg',
22+
elementIdDestination: 'connecting-via-environment-variables'
23+
},
24+
{
25+
title: 'Configuration file',
26+
description: 'For development setups',
27+
imageLink: '/install/build.svg',
28+
elementIdDestination: 'connecting-via-configuration-file'
29+
}
30+
]"
31+
/>
32+
33+
## With DebiAI-gui
34+
35+
If you are using the [DebiAI-gui Python package](../../introduction/gettingStarted/installation/README.md#debiai-gui-python-package) to run DebiAI, you can add Algo-providers directly as parameters:
36+
37+
```bash
38+
debiai-gui start -ap http://localhost:4000 http://localhost:8000
39+
```
40+
41+
Once you have created and deployed your Algo-provider, you can add it to DebiAI.
42+
43+
## From the dashboard
44+
45+
You can add your Algo-provider directly from the dashboard. To do so, go to the Algo-providers page from the menu:
46+
47+
![menu](../menu.png)
48+
49+
And click on the "Add a new Algo-provider" button:
50+
51+
![add](../algo_providers_menu_1.png)
52+
53+
You will need to provide the URL of your Algo-provider. This URL should be the root URL of your Algo-provider, for example: `https://my-algo-provider.com/`.
54+
55+
![add](../algo_providers_menu_2.png)
56+
57+
Once you have added your Algo-provider, you will be able to use the algorithms it provides in the Algorithms tab of the analysis dashboard:
58+
59+
![add](../algo_providers_menu_3.png)
60+
61+
## Connecting via Environment Variables
62+
63+
For deployments, you can define environment variables to specify provider URLs.
64+
65+
### Example:
66+
67+
```bash
68+
export DEBIAI_ALGO_PROVIDER_MyAlgoProvider1=http://localhost:3000/
69+
export DEBIAI_ALGO_PROVIDER_MyAlgoProvider2=http://localhost:3010/
70+
```
71+
72+
With Docker:
73+
74+
```bash
75+
docker run -p 3000:3000 \
76+
-e DEBIAI_ALGO_PROVIDER_MyAlgoProvider1=http://localhost:3000/ \
77+
-e DEBIAI_ALGO_PROVIDER_MyAlgoProvider2=http://localhost:3010/ \
78+
debiai/app
79+
```
80+
81+
With Docker Compose:
82+
83+
```yaml
84+
version: "3.8"
85+
services:
86+
debiai:
87+
image: "debiai/app"
88+
ports:
89+
- "3000:3000"
90+
environment:
91+
- DEBIAI_ALGO_PROVIDER_MyAlgoProvider1=http://localhost:3000/
92+
- DEBIAI_ALGO_PROVIDER_MyAlgoProvider2=http://localhost:3010/
93+
```
94+
95+
For a full list fo environment variables, check the [config.env](https://github.com/debiai/DebiAI/blob/main/debiaiServer/config/config.env) file.
96+
97+
## Connecting via Configuration File
98+
99+
You can also configure providers in config.ini:
100+
Example `(debiai/debiaiServer/config/config.ini)`:
101+
102+
```ini
103+
[ALGO_PROVIDERS]
104+
MyAlgoProvider1 = http://localhost:3000/
105+
MyAlgoProvider2 = http://localhost:3010/
106+
```
107+
108+
After editing, restart DebiAI (or rebuild the Docker image if using containers).
109+
110+
::: tip Configuration priority order:
111+
112+
1. DebiAI-gui python module parameters
113+
2. Environment variables
114+
3. Configuration file settings.
115+
:::
116+
117+
If the Algo-provider is accessible and follows the API, DebiAI will list the algorithms in the dashboard analysis page.
118+
119+
::: warning
120+
**DebiAI must be able to access your Algo-provider.**
121+
122+
- If running locally, use `localhost` as the URL.
123+
- If hosted externally, use the **public IP address**.
124+
- When using **Docker**, you may need to use the public IP or `--network host` to access a provider deployed on `localhost`.
125+
More details: [Docker documentation](https://docs.docker.com/network/host/).
126+
:::

‎documentation/customAlgorithms/algoProviders/algoProviderTemplates.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Algo-provider templates
2+
3+
To help you create custom algo-providers, we have created rest api service templates. You can use it as a starting point for your own Algo-provider.
4+
5+
- Python: [Algo-provider Python template](https://github.com/debiai/algo-provider-python-template).
6+
7+
:::tip
8+
Using the [algo-provider Python module](./easyAlgoProvider.md) is the easiest and recommended way to create an Algo-provider for DebiAI if you are using Python. Use the Python template if you want to go further with your Algo-provider.
9+
:::
10+
11+
After creating your Algo-provider, you can add it to DebiAI by providing the URL of your provider. [Learn more on how to add Algo-providers to DebiAI.](./addingAlgoProviders.md#adding-algo-providers-to-debiai)

0 commit comments

Comments
 (0)