|
| 1 | +--- |
| 2 | +title: Nitro on browser |
| 3 | +description: Nitro intergration guide for using on Web browser. |
| 4 | +keywords: [Nitro, Google Chrome, browser, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama] |
| 5 | +--- |
| 6 | + |
| 7 | +This guide demonstrates how to use Nitro on Web browser like an copilot on Web browser. |
| 8 | + |
| 9 | +## Installation |
| 10 | +We'll use [chatGPTBox](https://github.com/josStorer/chatGPTBox) for this setup. |
| 11 | + |
| 12 | +- For Chrome/Edge, download directly from the [Google Web Store](https://chromewebstore.google.com/detail/chatgptbox/eobbhoofkanlmddnplfhnmkfbnlhpbbo). |
| 13 | +- For other browsers, refer to their [installation guide](https://github.com/josStorer/chatGPTBox/wiki/Install). |
| 14 | + |
| 15 | +## Using chatGPTBox with Nitro |
| 16 | + |
| 17 | +**1. Start Nitro server** |
| 18 | + |
| 19 | +Open your terminal and run: |
| 20 | + |
| 21 | +```bash title="Run Nitro" |
| 22 | +nitro |
| 23 | +``` |
| 24 | + |
| 25 | +**2. Download Model** |
| 26 | + |
| 27 | +Download the [Stealth 7B](https://huggingface.co/jan-hq/stealth-v1.3) model with these commands: |
| 28 | + |
| 29 | +```bash title="Get a model" |
| 30 | +mkdir model && cd model |
| 31 | +wget -O stealth-7b.gguf https://huggingface.co/janhq/stealth-v1.3-GGUF/resolve/main/stealth-v1.3.Q4_K_M.gguf |
| 32 | +``` |
| 33 | + |
| 34 | +> Explore more GGUF models at [The Bloke](https://huggingface.co/TheBloke). |
| 35 | +
|
| 36 | +**3. Run the Model** |
| 37 | + |
| 38 | +To load the model, use the following command: |
| 39 | + |
| 40 | +```bash title="Load model to the server" |
| 41 | +curl http://localhost:3928/inferences/llamacpp/loadmodel \ |
| 42 | + -H 'Content-Type: application/json' \ |
| 43 | + -d '{ |
| 44 | + "llama_model_path": "model/stealth-7b.gguf", |
| 45 | + "ctx_len": 512, |
| 46 | + "ngl": 100, |
| 47 | + }' |
| 48 | +``` |
| 49 | + |
| 50 | +**4. Configure chatGPTBox** |
| 51 | + |
| 52 | +Click the `chatGPTBox` icon in your browser to open the main menu. |
| 53 | + |
| 54 | + |
| 55 | + |
| 56 | +In the `API Mode` section, switch to `Custom Model`. |
| 57 | + |
| 58 | +Change the name to `Stealth 7B` and set the host to `http://localhost:3928/v1/chat/completions` |
| 59 | + |
| 60 | + |
| 61 | + |
| 62 | +Your browser is now equipped with a copilot to help you on daily basis. |
| 63 | + |
| 64 | + |
| 65 | + |
| 66 | +For detailed guidance, visit the [chatGPTBox guide](https://github.com/josStorer/chatGPTBox/wiki/Guide) |
| 67 | + |
| 68 | +### Shortcut |
| 69 | + |
| 70 | +| Shortcut | Description | |
| 71 | +|----------|---------------------------------------------| |
| 72 | +| Ctrl+B | Activate chat dialog box on any page. | |
| 73 | +| Alt+B | Summarize any page via the right-click menu. | |
| 74 | +| Ctrl+Shift+H | Open an independent conversation page. | |
| 75 | + |
| 76 | +## Futher Usage |
| 77 | + |
| 78 | +For convenient usage, you can utilize [Jan](https://jan.ai/), as it is integrated with Nitro. |
0 commit comments