You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: cortex-cpp/README.md
+6-16Lines changed: 6 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,14 +12,14 @@
12
12
13
13
## About cortex-cpp
14
14
15
-
Cortex.cpp is a stateless, C++ server that is 100% compatible with OpenAI API (stateless endpoints). It includes a Drogon server, with request queues, model orchestration logic, and hardware telemetry, and more, for prod environments.
15
+
Cortex-cpp is a streamlined, stateless C++ server engineered to be fully compatible with OpenAI's API, particularly its stateless functionalities. It integrates a Drogon server framework to manage request handling and includes features like model orchestration and hardware telemetry, which are essential for production environments.
16
16
17
-
The binary of cortex-cpp after zipped is only ~3mb in size with none to minimal dependencies (if you use a GPU need CUDA for example) make it desirable for any edge/server deployment 👍.
17
+
Remarkably compact, the binary size of cortex-cpp is around 3 MB when compressed, with minimal dependencies. This lightweight and efficient design makes cortex-cpp an excellent choice for deployments in both edge computing and server contexts.
18
18
19
-
> Read more about Cortex-cpp at https://jan.ai/cortex
19
+
> Utilizing GPU capabilities does require CUDA.
20
20
21
21
### Features
22
-
- Fast Inference: Built on top of the cutting-edge inference library llama.cpp, modified to be productionready.
22
+
- Fast Inference: Built on top of the cutting-edge inference library llama.cpp, modified to be production-ready.
23
23
- Lightweight: Only 3MB, ideal for resource-sensitive environments.
24
24
- Easily Embeddable: Simple integration into existing applications, offering flexibility.
25
25
- Quick Setup: Approximately 10-second initialization for swift deployment.
@@ -44,17 +44,7 @@ The binary of cortex-cpp after zipped is only ~3mb in size with none to minimal
@@ -205,7 +195,7 @@ The nightly build is triggered at 2:00 AM UTC every day.
205
195
The nightly build can be downloaded from the URL notified in the Discord channel. Please access the URL from the browser and download the build artifacts from there.
206
196
207
197
## Manual Build
208
-
Manual build is a process in which the developers build the software manually. This is usually done when a new feature is implemented or a bug is fixed. The process for this project is defined in [`.github/workflows/build.yml`](.github/workflows/build.yml)
198
+
Manual build is a process in which the developers build the software manually. This is usually done when a new feature is implemented, or a bug is fixed. The process for this project is defined in [`.github/workflows/build.yml`](.github/workflows/build.yml)
209
199
210
200
It is similar to the nightly build process, except that it is triggered manually by the developers.
Copy file name to clipboardExpand all lines: cortex-js/src/main.ts
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -33,7 +33,7 @@ async function bootstrap() {
33
33
.addTag('Models','These endpoints provide a list and descriptions of all available models within the Cortex framework.')
34
34
.addTag('Messages',"These endpoints manage the retrieval and storage of conversation content, including responses from LLMs and other metadata related to chat interactions.")
35
35
.addTag('Threads','These endpoints handle the creation, retrieval, updating, and deletion of conversation threads.')
0 commit comments