Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit 1f57ed7

Browse files
irfanpenairfanpena
committed
docs: update the cortex-js readme
1 parent 862ccba commit 1f57ed7

File tree

1 file changed

+114
-75
lines changed

1 file changed

+114
-75
lines changed

‎cortex-js/README.md‎

Lines changed: 114 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,70 @@
1-
# Installation
1+
# Cortex - CLI
2+
3+
<p align="center">
4+
<a href="https://jan.ai/cortex">Documentation</a> - <a href="https://jan.ai/api-reference">API Reference</a>
5+
- <a href="https://github.com/janhq/cortex/releases">Changelog</a> - <a href="https://github.com/janhq/cortex/issues">Bug reports</a> - <a href="https://discord.gg/AsJ8krTT3N">Discord</a>
6+
</p>
7+
8+
> ⚠️ **Cortex is currently in Development**: Expect breaking changes and bugs!
9+
10+
## About
11+
Cortex is an openAI-compatible local AI server that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and a Typescript client library. It can be used as a standalone server, or imported as a library.
12+
13+
Cortex currently supports two inference engines:
14+
15+
- Llama.cpp
16+
- TensorRT-LLM
17+
18+
> Read more about Cortex at https://jan.ai/cortex
19+
20+
### Repo Structure
21+
```
22+
# Entity Definitions
23+
domain/ # This is the core directory where the domains are defined.
24+
abstracts/ # Abstract base classes for common attributes and methods.
25+
models/ # Domain interface definitions, e.g. model, assistant.
26+
repositories/ # Extensions abstract and interface
27+
28+
# Business Rules
29+
usecases/ # Application logic
30+
assistants/ # CRUD logic (invokes dtos, entities).
31+
chat/ # Logic for chat functionalities.
32+
models/ # Logic for model operations.
33+
34+
# Adapters & Implementations
35+
infrastructure/ # Implementations for Cortex interactions
36+
commanders/ # CLI handlers
37+
models/
38+
questions/ # CLI installation UX
39+
shortcuts/ # CLI chained syntax
40+
types/
41+
usecases/ # Invokes UseCases
42+
43+
controllers/ # Nest controllers and HTTP routes
44+
assistants/ # Invokes UseCases
45+
chat/ # Invokes UseCases
46+
models/ # Invokes UseCases
47+
48+
database/ # Database providers (mysql, sqlite)
49+
50+
# Framework specific object definitions
51+
dtos/ # DTO definitions (data transfer & validation)
52+
entities/ # TypeORM entity definitions (db schema)
53+
54+
# Providers
55+
providers/cortex # Cortex [server] provider (a core extension)
56+
repositories/extensions # Extension provider (core & external extensions)
57+
58+
extensions/ # External extensions
59+
command.module.ts # CLI Commands List
60+
main.ts # Entrypoint
61+
```
62+
## Quicklinks
63+
Cortex
64+
- [Website](https://jan.ai/)
65+
- [GitHub](https://github.com/janhq/cortex)
66+
- [User Guides](https://jan.ai/cortex)
67+
- [API reference](https://jan.ai/api-reference)
268

369
## Prerequisites
470

@@ -10,10 +76,9 @@ Before installation, ensure that you have installed the following:
1076
- **NPM**: Needed to manage packages.
1177
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.
1278

13-
<aside>
14-
💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.
1579

16-
</aside>
80+
>💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.
81+
1782

1883
### **Hardware**
1984

@@ -35,88 +100,34 @@ Ensure that your system meets the following requirements to run Cortex:
35100

36101
- **Disk**: At least 10GB for app and model download.
37102

38-
## Cortex Installation
39-
40-
To install Cortex, follow the steps below:
41-
42-
### Step 1: Install Cortex
43-
44-
Run the following command to install Cortex globally on your machine:
45-
46-
```bash
47-
# Install using NPM globally
103+
## Quickstart
104+
1. Install the NPM package:
105+
``` bash
48106
npm i -g @janhq/cortex
49107
```
50108

51-
### Step 2: Verify the Installation
52-
53-
After installation, you can verify that Cortex is installed correctly by getting help information.
54-
55-
```bash
56-
# Get the help information
57-
cortex -h
58-
```
59-
60-
### Step 3: Initialize Cortex
61-
62-
Once verified, you need to initialize the Cortex engine.
63-
64-
1. Initialize the Cortex engine:
65-
66-
```
109+
2. Initialize a compatible engine:
110+
``` bash
67111
cortex init
68112
```
69113

70-
1. Select between `CPU` and `GPU` modes.
71-
72-
```bash
73-
? Select run mode (Use arrow keys)
74-
> CPU
75-
GPU
76-
```
77-
78-
2. Select between GPU types.
79-
80-
```bash
81-
? Select GPU types (Use arrow keys)
82-
> Nvidia
83-
Others (Vulkan)
84-
```
85-
86-
3. Select CPU instructions (will be deprecated soon).
87-
88-
```bash
89-
? Select CPU instructions (Use arrow keys)
90-
> AVX2
91-
AVX
92-
AVX-512
114+
3. Download a GGUF model from Hugging Face
115+
``` bash
116+
cortex models pull janhq/TinyLlama-1.1B-Chat-v1.0-GGUF
93117
```
94-
95-
1. Cortex will download the required CPU instruction sets if you choose `CPU` mode. If you choose `GPU` mode, Cortex will download the necessary dependencies to use your GPU.
96-
2. Once downloaded, Cortex is ready to use!
97-
98-
### Step 4: Pull a model
99-
100-
From HuggingFace
101-
102-
```bash
103-
cortex pull janhq/phi-3-medium-128k-instruct-GGUF
118+
4. Load the model
119+
``` bash
120+
cortex models start janhq/TinyLlama-1.1B-Chat-v1.0-GGUF
104121
```
105122

106-
From Jan Hub (TBD)
107-
108-
```bash
109-
cortex pull llama3
123+
5. Start chatting with the model
124+
``` bash
125+
cortex chat tell me a joke
110126
```
111127

112-
### Step 5: Chat
113-
114-
```bash
115-
cortex run janhq/phi-3-medium-128k-instruct-GGUF
116-
```
117128

118129
## Run as an API server
119-
130+
To run Cortex as an API server:
120131
```bash
121132
cortex serve
122133
```
@@ -133,18 +144,42 @@ To install Cortex from the source, follow the steps below:
133144
npx nest build
134145
```
135146

136-
1. Make the `command.js` executable:
147+
4. Make the `command.js` executable:
137148

138149
```bash
139150
chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
140151
```
141152

142-
1. Link the package globally:
153+
5. Link the package globally:
143154

144155
```bash
145156
npm link
146157
```
147158

159+
## Cortex CLI Command
160+
The following CLI commands are currently available:
161+
> ⚠️ **Cortex is currently in Development**: More commands will be added soon!
162+
163+
```bash
164+
165+
serve Providing API endpoint for Cortex backend
166+
chat Send a chat request to a model
167+
init|setup Init settings and download cortex's dependencies
168+
ps Show running models and their status
169+
kill Kill running cortex processes
170+
pull|download Download a model. Working with HuggingFace model id.
171+
run [options] EXPERIMENTAL: Shortcut to start a model and chat
172+
models Subcommands for managing models
173+
models list List all available models.
174+
models pull Download a specified model.
175+
models remove Delete a specified model.
176+
models get Retrieve the configuration of a specified model.
177+
models start Start a specified model.
178+
models stop Stop a specified model.
179+
models update Update the configuration of a specified model.
180+
engines Execute a specified command related to engines.
181+
engines list List all available engines.
182+
```
148183
## Uninstall Cortex
149184
150185
Run the following command to uninstall Cortex globally on your machine:
@@ -153,3 +188,7 @@ Run the following command to uninstall Cortex globally on your machine:
153188
# Uninstall globally using NPM
154189
npm uninstall -g @janhq/cortex
155190
```
191+
## Contact Support
192+
- For support, please file a GitHub ticket.
193+
- For questions, join our Discord [here](https://discord.gg/FTk2MvZwJH).
194+
- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).

0 commit comments

Comments
 (0)