Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit ff55fb7

Browse files
irfanpenairfanpena
committed
docs: Update the Readme files
1 parent 3e41f28 commit ff55fb7

File tree

6 files changed

+131
-130
lines changed

6 files changed

+131
-130
lines changed

‎CONTRIBUTING.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
### Repo Structure
2+
```
3+
# Entity Definitions
4+
domain/ # This is the core directory where the domains are defined.
5+
abstracts/ # Abstract base classes for common attributes and methods.
6+
models/ # Domain interface definitions, e.g. model, assistant.
7+
repositories/ # Extensions abstract and interface
8+
9+
# Business Rules
10+
usecases/ # Application logic
11+
assistants/ # CRUD logic (invokes dtos, entities).
12+
chat/ # Logic for chat functionalities.
13+
models/ # Logic for model operations.
14+
15+
# Adapters & Implementations
16+
infrastructure/ # Implementations for Cortex interactions
17+
commanders/ # CLI handlers
18+
models/
19+
questions/ # CLI installation UX
20+
shortcuts/ # CLI chained syntax
21+
types/
22+
usecases/ # Invokes UseCases
23+
24+
controllers/ # Nest controllers and HTTP routes
25+
assistants/ # Invokes UseCases
26+
chat/ # Invokes UseCases
27+
models/ # Invokes UseCases
28+
29+
database/ # Database providers (mysql, sqlite)
30+
31+
# Framework specific object definitions
32+
dtos/ # DTO definitions (data transfer & validation)
33+
entities/ # TypeORM entity definitions (db schema)
34+
35+
# Providers
36+
providers/cortex # Cortex [server] provider (a core extension)
37+
repositories/extensions # Extension provider (core & external extensions)
38+
39+
extensions/ # External extensions
40+
command.module.ts # CLI Commands List
41+
main.ts # Entrypoint
42+
```

‎README.md‎

Lines changed: 6 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -17,50 +17,8 @@ Cortex currently supports two inference engines:
1717

1818
> Read more about Cortex at https://jan.ai/cortex
1919
20-
### Repo Structure
21-
```
22-
# Entity Definitions
23-
domain/ # This is the core directory where the domains are defined.
24-
abstracts/ # Abstract base classes for common attributes and methods.
25-
models/ # Domain interface definitions, e.g. model, assistant.
26-
repositories/ # Extensions abstract and interface
27-
28-
# Business Rules
29-
usecases/ # Application logic
30-
assistants/ # CRUD logic (invokes dtos, entities).
31-
chat/ # Logic for chat functionalities.
32-
models/ # Logic for model operations.
33-
34-
# Adapters & Implementations
35-
infrastructure/ # Implementations for Cortex interactions
36-
commanders/ # CLI handlers
37-
models/
38-
questions/ # CLI installation UX
39-
shortcuts/ # CLI chained syntax
40-
types/
41-
usecases/ # Invokes UseCases
42-
43-
controllers/ # Nest controllers and HTTP routes
44-
assistants/ # Invokes UseCases
45-
chat/ # Invokes UseCases
46-
models/ # Invokes UseCases
47-
48-
database/ # Database providers (mysql, sqlite)
49-
50-
# Framework specific object definitions
51-
dtos/ # DTO definitions (data transfer & validation)
52-
entities/ # TypeORM entity definitions (db schema)
53-
54-
# Providers
55-
providers/cortex # Cortex [server] provider (a core extension)
56-
repositories/extensions # Extension provider (core & external extensions)
57-
58-
extensions/ # External extensions
59-
command.module.ts # CLI Commands List
60-
main.ts # Entrypoint
61-
```
6220
## Quicklinks
63-
Cortex
21+
**Cortex**:
6422
- [Website](https://jan.ai/)
6523
- [GitHub](https://github.com/janhq/cortex)
6624
- [User Guides](https://jan.ai/cortex)
@@ -101,7 +59,8 @@ Ensure that your system meets the following requirements to run Cortex:
10159
- **Disk**: At least 10GB for app and model download.
10260

10361
## Quickstart
104-
1. Install the NPM package:
62+
To install Cortex CLI, follow the steps below:
63+
1. Install the Cortex NPM package:
10564
``` bash
10665
npm i -g @janhq/cortex
10766
```
@@ -111,16 +70,16 @@ npm i -g @janhq/cortex
11170
cortex init
11271
```
11372

114-
3. Download a GGUF model from Hugging Face
73+
3. Download a GGUF model from Hugging Face:
11574
``` bash
11675
cortex models pull janhq/TinyLlama-1.1B-Chat-v1.0-GGUF
11776
```
118-
4. Load the model
77+
4. Load the model:
11978
``` bash
12079
cortex models start janhq/TinyLlama-1.1B-Chat-v1.0-GGUF
12180
```
12281

123-
5. Start chatting with the model
82+
5. Start chatting with the model:
12483
``` bash
12584
cortex chat tell me a joke
12685
```
@@ -177,8 +136,6 @@ The following CLI commands are currently available:
177136
models start Start a specified model.
178137
models stop Stop a specified model.
179138
models update Update the configuration of a specified model.
180-
engines Execute a specified command related to engines.
181-
engines list List all available engines.
182139
```
183140
## Uninstall Cortex
184141

‎cortex-cpp/CONTRIBUTING.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
### Repo Structure
2+
3+
```
4+
.
5+
├── common # Common libraries or shared resources
6+
├── controllers # Controller scripts or modules for managing interactions
7+
├── cortex-common # Shared components across different cortex modules
8+
├── cortex-cpp-deps # Dependencies specific to the cortex-cpp module
9+
├── engines # Different processing or computational engines
10+
├── examples # Example scripts or applications demonstrating usage
11+
├── test # Test scripts and testing frameworks
12+
└── utils # Utility scripts and helper functions
13+
14+
```

‎cortex-cpp/README.md‎

Lines changed: 27 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -18,48 +18,46 @@ Remarkably compact, the binary size of cortex-cpp is around 3 MB when compressed
1818

1919
> Utilizing GPU capabilities does require CUDA.
2020
21-
### Features
22-
- Fast Inference: Built on top of the cutting-edge inference library llama.cpp, modified to be production-ready.
23-
- Lightweight: Only 3MB, ideal for resource-sensitive environments.
24-
- Easily Embeddable: Simple integration into existing applications, offering flexibility.
25-
- Quick Setup: Approximately 10-second initialization for swift deployment.
26-
- Enhanced Web Framework: Incorporates drogon cpp to boost web service efficiency.
21+
## Prerequisites
22+
### **Hardware**
2723

28-
### Repo Structure
24+
Ensure that your system meets the following requirements to run Cortex:
2925

30-
```
31-
.
32-
├── common # Common libraries or shared resources
33-
├── controllers # Controller scripts or modules for managing interactions
34-
├── cortex-common # Shared components across different cortex modules
35-
├── cortex-cpp-deps # Dependencies specific to the cortex-cpp module
36-
├── engines # Different processing or computational engines
37-
├── examples # Example scripts or applications demonstrating usage
38-
├── test # Test scripts and testing frameworks
39-
└── utils # Utility scripts and helper functions
40-
41-
```
26+
- **OS**:
27+
- MacOSX 13.6 or higher.
28+
- Windows 10 or higher.
29+
- Ubuntu 18.04 and later.
30+
- **RAM (CPU Mode):**
31+
- 8GB for running up to 3B models.
32+
- 16GB for running up to 7B models.
33+
- 32GB for running up to 13B models.
34+
- **VRAM (GPU Mode):**
4235

43-
## Quickstart
36+
- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
37+
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
38+
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
4439

45-
**Step 1: Install cortex-cpp**
40+
- **Disk**: At least 10GB for app and model download.
4641

47-
Download cortex-cpp here: https://github.com/janhq/cortex/releases
42+
## Quickstart
43+
To install Cortex CLI, follow the steps below:
44+
1. Download cortex-cpp here: https://github.com/janhq/cortex/releases
45+
2. Install cortex-cpp by running the downloaded file.
4846

49-
**Step 2: Downloading a Model**
47+
3. Download a Model:
5048

5149
```bash
5250
mkdir model && cd model
5351
wget -O llama-2-7b-model.gguf https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_M.gguf?download=true
5452
```
5553

56-
**Step 3: Run cortex-cpp server**
54+
4. Run cortex-cpp server:
5755

5856
```bash title="Run cortex-cpp server"
5957
cortex-cpp
6058
```
6159

62-
**Step 4: Load model**
60+
5. Load a model:
6361

6462
```bash title="Load model"
6563
curl http://localhost:3928/inferences/server/loadmodel \
@@ -71,7 +69,7 @@ curl http://localhost:3928/inferences/server/loadmodel \
7169
}'
7270
```
7371

74-
**Step 5: Making an Inference**
72+
6. Make an Inference:
7573

7674
```bash title="cortex-cpp Inference"
7775
curl http://localhost:3928/v1/chat/completions \
@@ -86,7 +84,8 @@ curl http://localhost:3928/v1/chat/completions \
8684
}'
8785
```
8886

89-
Table of parameters
87+
## Table of parameters
88+
Below is the available list of the model parameters you can set when loading a model in cortex-cpp:
9089

9190
| Parameter | Type | Description |
9291
|------------------|---------|--------------------------------------------------------------|
@@ -110,17 +109,6 @@ Table of parameters
110109
|`grammar_file`| String |You can constrain the sampling using GBNF grammars by providing path to a grammar file|
111110
|`model_type` | String | Model type we want to use: llm or embedding, default value is llm|
112111

113-
***OPTIONAL***: You can run Cortex-cpp on a different port like 5000 instead of 3928 by running it manually in terminal
114-
```zsh
115-
./cortex-cpp 1 127.0.0.1 5000 ([thread_num] [host] [port] [uploads_folder_path])
116-
```
117-
- thread_num : the number of thread that cortex-cpp webserver needs to have
118-
- host : host value normally 127.0.0.1 or 0.0.0.0
119-
- port : the port that cortex-cpp got deployed onto
120-
- uploads_folder_path: custom path for file uploads in Drogon.
121-
122-
cortex-cpp server is compatible with the OpenAI format, so you can expect the same output as the OpenAI ChatGPT API.
123-
124112
## Download
125113

126114
<table>
@@ -171,7 +159,7 @@ cortex-cpp server is compatible with the OpenAI format, so you can expect the sa
171159
</tr>
172160
</table>
173161

174-
Download the latest version of Cortex-cpp at https://jan.ai/ or visit the **[GitHub Releases](https://github.com/janhq/cortex/releases)** to download any previous release.
162+
> Download the latest or older versions of Cortex-cpp at the **[GitHub Releases](https://github.com/janhq/cortex/releases)**.
175163
176164

177165
## Manual Build

‎cortex-js/CONTRIBUTING.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
### Repo Structure
2+
```
3+
# Entity Definitions
4+
domain/ # This is the core directory where the domains are defined.
5+
abstracts/ # Abstract base classes for common attributes and methods.
6+
models/ # Domain interface definitions, e.g. model, assistant.
7+
repositories/ # Extensions abstract and interface
8+
9+
# Business Rules
10+
usecases/ # Application logic
11+
assistants/ # CRUD logic (invokes dtos, entities).
12+
chat/ # Logic for chat functionalities.
13+
models/ # Logic for model operations.
14+
15+
# Adapters & Implementations
16+
infrastructure/ # Implementations for Cortex interactions
17+
commanders/ # CLI handlers
18+
models/
19+
questions/ # CLI installation UX
20+
shortcuts/ # CLI chained syntax
21+
types/
22+
usecases/ # Invokes UseCases
23+
24+
controllers/ # Nest controllers and HTTP routes
25+
assistants/ # Invokes UseCases
26+
chat/ # Invokes UseCases
27+
models/ # Invokes UseCases
28+
29+
database/ # Database providers (mysql, sqlite)
30+
31+
# Framework specific object definitions
32+
dtos/ # DTO definitions (data transfer & validation)
33+
entities/ # TypeORM entity definitions (db schema)
34+
35+
# Providers
36+
providers/cortex # Cortex [server] provider (a core extension)
37+
repositories/extensions # Extension provider (core & external extensions)
38+
39+
extensions/ # External extensions
40+
command.module.ts # CLI Commands List
41+
main.ts # Entrypoint
42+
```

‎cortex-js/README.md‎

Lines changed: 0 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -17,48 +17,6 @@ Cortex currently supports two inference engines:
1717

1818
> Read more about Cortex at https://jan.ai/cortex
1919
20-
### Repo Structure
21-
```
22-
# Entity Definitions
23-
domain/ # This is the core directory where the domains are defined.
24-
abstracts/ # Abstract base classes for common attributes and methods.
25-
models/ # Domain interface definitions, e.g. model, assistant.
26-
repositories/ # Extensions abstract and interface
27-
28-
# Business Rules
29-
usecases/ # Application logic
30-
assistants/ # CRUD logic (invokes dtos, entities).
31-
chat/ # Logic for chat functionalities.
32-
models/ # Logic for model operations.
33-
34-
# Adapters & Implementations
35-
infrastructure/ # Implementations for Cortex interactions
36-
commanders/ # CLI handlers
37-
models/
38-
questions/ # CLI installation UX
39-
shortcuts/ # CLI chained syntax
40-
types/
41-
usecases/ # Invokes UseCases
42-
43-
controllers/ # Nest controllers and HTTP routes
44-
assistants/ # Invokes UseCases
45-
chat/ # Invokes UseCases
46-
models/ # Invokes UseCases
47-
48-
database/ # Database providers (mysql, sqlite)
49-
50-
# Framework specific object definitions
51-
dtos/ # DTO definitions (data transfer & validation)
52-
entities/ # TypeORM entity definitions (db schema)
53-
54-
# Providers
55-
providers/cortex # Cortex [server] provider (a core extension)
56-
repositories/extensions # Extension provider (core & external extensions)
57-
58-
extensions/ # External extensions
59-
command.module.ts # CLI Commands List
60-
main.ts # Entrypoint
61-
```
6220
## Quicklinks
6321
Cortex
6422
- [Website](https://jan.ai/)

0 commit comments

Comments
 (0)