Merge pull request #333 from janhq/add/update-docs

Feat: Update documentation

Author: tikikun

docs/docs/examples/chatboxgpt.md

@@ -0,0 +1,78 @@
+---
+title: Nitro on browser
+description: Nitro intergration guide for using on Web browser.
+keywords: [Nitro, Google Chrome, browser, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
+---
+
+This guide demonstrates how to use Nitro on Web browser like an copilot on Web browser.
+
+## Installation
+We'll use [chatGPTBox](https://github.com/josStorer/chatGPTBox) for this setup.
+
+- For Chrome/Edge, download directly from the [Google Web Store](https://chromewebstore.google.com/detail/chatgptbox/eobbhoofkanlmddnplfhnmkfbnlhpbbo).
+- For other browsers, refer to their [installation guide](https://github.com/josStorer/chatGPTBox/wiki/Install).
+
+## Using chatGPTBox with Nitro
+
+**1. Start Nitro server**
+
+Open your terminal and run:
+
+```bash title="Run Nitro"
+nitro
+```
+
+**2. Download Model**
+
+Download the [Stealth 7B](https://huggingface.co/jan-hq/stealth-v1.3) model with these commands:
+
+```bash title="Get a model"
+mkdir model && cd model
+wget -O stealth-7b.gguf https://huggingface.co/janhq/stealth-v1.3-GGUF/resolve/main/stealth-v1.3.Q4_K_M.gguf
+```
+
+> Explore more GGUF models at [The Bloke](https://huggingface.co/TheBloke).
+
+**3. Run the Model**
+
+To load the model, use the following command:
+
+```bash title="Load model to the server"
+curl http://localhost:3928/inferences/llamacpp/loadmodel \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "llama_model_path": "model/stealth-7b.gguf",
+    "ctx_len": 512,
+    "ngl": 100,
+  }'
+```
+
+**4. Configure chatGPTBox**
+
+Click the `chatGPTBox` icon in your browser to open the main menu.
+
+![Main menu chatGPTBox](img/chatgptbox_menu.png)
+
+In the `API Mode` section, switch to `Custom Model`.
+
+Change the name to `Stealth 7B` and set the host to `http://localhost:3928/v1/chat/completions`
+
+![Use chatGPTBox](img/chatgptbox_cus.png)
+
+Your browser is now equipped with a copilot to help you on daily basis.
+
+![Web browser with chatGPTBox](img/chatgptbox_use.png)
+
+For detailed guidance, visit the [chatGPTBox guide](https://github.com/josStorer/chatGPTBox/wiki/Guide)
+
+### Shortcut
+
+| Shortcut | Description                                 |
+|----------|---------------------------------------------|
+| Ctrl+B       | Activate chat dialog box on any page. |
+| Alt+B      | Summarize any page via the right-click menu. |
+| Ctrl+Shift+H      | Open an independent conversation page. |
+
+## Futher Usage
+
+For convenient usage, you can utilize [Jan](https://jan.ai/), as it is integrated with Nitro.

docs/docs/examples/img/chatgptbox_cus.png

@@ -27,7 +27,7 @@ Use these commands to download and save the [Llama2 7B chat model](https://huggi
 
 ```bash title="Get a model"
 mkdir model && cd model
-wget -O llama-2-7b-model.gguf https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_M.gguf?download=true
+wget -O llama-2-7b-model.gguf https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_M.gguf
 ```
 
 > For more GGUF model, please look at [The Bloke](https://huggingface.co/TheBloke).

docs/docs/examples/img/chatgptbox_menu.png

@@ -0,0 +1,25 @@
+---
+title: Logging
+description: Enabling logging in Nitro.
+keywords: [Nitro, log, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
+---
+
+
+## Enabling Logging
+
+Nitro's logging feature can be activated by specifying a log folder. This is crucial for monitoring and troubleshooting.
+
+## Setting Up Logging
+
+To configure logging, you need to specify the path to the log folder. Use the following command to set it up:
+
+```bash title="Config logging" {5}
+curl http://localhost:3928/inferences/llamacpp/loadmodel \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "llama_model_path": "/path/to/your_model.gguf",
+    "llama_log_folder": "/path/to/log/folder/"
+  }'
+```
+
+> **Note:** Ensure the log folder exists before running this command. If the specified folder does not exist, logs will default to your current directory.

docs/docs/examples/img/chatgptbox_use.png

@@ -0,0 +1,65 @@
+---
+title: Vision
+description: Inference engine for vision, the same as OpenAI's
+keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llava, bakllava, vision]
+---
+
+## Load model
+Just like loading the Chat model, for the vision model, you need two specific types:
+- the `GGUF model`
+- the `mmproj model`.
+
+You can load the model using:
+
+```bash title="Load Model" {3,4}
+curl -X POST 'http://127.0.0.1:3928/inferences/llamacpp/loadmodel' -H 'Content-Type: application/json' -d '{
+   "llama_model_path": "/path/to/gguf/model/",
+   "mmproj": "/path/to/mmproj/model/",
+   "ctx_len": 2048,
+   "ngl": 100,
+   "cont_batching": false,
+   "embedding": false,
+   "system_prompt": "",
+   "user_prompt": "\n### Instruction:\n",
+   "ai_prompt": "\n### Response:\n"
+ }'
+```
+
+Download the models here:
+- [Llava Model](https://huggingface.co/jartine/llava-v1.5-7B-GGUF/tree/main): Large Language and Vision Assistant achieves SoTA on 11 benchmarks.
+- [Bakllava Model](https://huggingface.co/mys/ggml_bakllava-1/tree/main) is a Mistral 7B base augmented with the LLaVA architecture.
+
+## Inference
+
+Nitro currently only works with images converted to base64 format. Use this [base64 converter](https://www.base64-image.de/) to prepare your images.
+
+To get the model's understanding of an image, do the following:
+
+```bash title="Inference"
+curl http://127.0.0.1:3928/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -d '{
+    "model": "gpt-4-vision-preview",
+    "messages": [
+      {
+        "role": "user",
+        "content": [
+          {
+            "type": "text",
+            "text": "What’s in this image?"
+          },
+          {
+            "type": "image_url",
+            "image_url": {
+              "url": "<base64>"
+            }
+          }
+        ]
+      }
+    ],
+    "max_tokens": 300
+  }'
+```
+
+If the base64 string is too long and causes errors, consider using [Postman](https://www.postman.com/) as an alternative.

docs/docs/examples/palchat.md

@@ -39,6 +39,7 @@ const sidebars = {
       items: [
         "features/chat",
         "features/embed",
+        "features/vision"
       ],
     },
     {
@@ -51,6 +52,7 @@ const sidebars = {
         "features/load-unload",
         "features/warmup",
         "features/prompt",
+        "features/log"
       ],
     },
     {
@@ -64,7 +66,8 @@ const sidebars = {
         "examples/palchat",
         "examples/openai-node",
         "examples/openai-python",
-        "examples/colab"
+        "examples/colab",
+        "examples/chatboxgpt"
       ],
     },
     // {