Push the whole working dir as "context"

[tbruyelle]

When you run some tool like Claude Code, they seem to load all the files of the working dir and use them as a context, so then you can ask questions like "summarize this project" etc...

The context pushed by vim-ollama is a configuratble number of lines below and after the current line. Is it possible to extend that context to the whole working dir like Claude Code ? This "should" improve a lot the code completion because the LLM is aware of the whole project.

Sorry if my suggestion is a bit naive (and I think it is), I'm not familiar with LLM.

[martinengelke]

That would be a big benefit.

[gergap]

Technically, this would be not a problem for the plugin, but for the LLM and GPU memory you have at home.
See e.g. qwen2.5-coder, has a context window of 128k tokens. What is a token? The small units the text is split into and fed into the LLM. A token can consist of a simple symbol like '}' or some letter-combinations, but it it is definitely smaller than a word. It is OK to just think of letters to be on the safe side. So one large file can already fill up the whole context window of a LLM.
The "solution" to this problem is building a RAG (Retrieval-Augmented Generation). This is essentially a vector database that indexes your codebase. I don't go into details here, but the rough idea is that another AI is analyzing your request and searches in the DB for best semantic matches. It then distills the most relevant information like related API definitions or example code specific for this task out of the DB data, builds a new prompt for the LLM, that fits into the contexts window, and then the LLM has a better chance to generate plausible results.
Pros: The DB can be extended over time and "learn" new information without training a custom LLM.
Cons: Extracting the data from your codebase (source code, APIs, documentation, examples), splitting these into chunks and indexing it in a way that the DB can later find the right info is more difficult than you might think. I tried this approach using my C code base and free vectorizing models. I also mixed it with classic keyword based search approaches. But it often found the wrong information which might use similar terminology but still is the wrong API, so the LLM cannot generate plausible data. Shit-in-shit-out.
If anybody knows a reliable way to index code and find the right stuff I would be happy to know how.

I could imagine that other languages (HTML, JS, python) are easier. At least I have the impression when using commercial models, but this may also be just the result of better training for these languages.

My C codebase is about 65MB of pure source code (without examples or docs), so this is a real world problem, but maybe not the right thing to start with :-)

[martinengelke]

I generally know a bit about tensors and data processing, so I understand roughly about what you are talking.

I was just thinking about small projects. Maybe there is a way to limit the RAG at first until new approaches are born.

For Example my website has some .html files, a couple of .css files and a few .js files, which are strong connected in a way a website is working. For now the model is just seeing one file and does not have the scripts or styles in mind. It often pastes it in my .html and I have to transfer it manually.

Maybe the model could at least point to the right file and then navigate there, read and perform?
Especially for C you have .h header files, which may be sufficient for the RAG to identify where to point you? So the LLM is step by step point you to the solution rather than being blind?

[gergap]

For Example my website has some .html files, a couple of .css files and a few .js files, which are strong connected in a way a website is working. For now the model is just seeing one file and does not have the scripts or styles in mind. It often pastes it in my .html and I have to transfer it manually.

I think for this usecase the simplest way is to add the file contents using keywords like @file <filename> or @buf <name|num>. Especially, referencing open Vim buffers could be useful IMO. This would be simply a matter of pre-processing your chat queries.
So I'm thinking about the :OllamaChat and :OllamaEdit commands.
I don't believe for code-completion (tab-completion) this would make sense much sense, because it would slow down it too much.

[celer]

For Example my website has some .html files, a couple of .css files and a few .js files, which are strong connected in a way a website is working. For now the model is just seeing one file and does not have the scripts or styles in mind. It often pastes it in my .html and I have to transfer it manually.

I think for this usecase the simplest way is to add the file contents using keywords like @file <filename> or @buf <name|num>. Especially, referencing open Vim buffers could be useful IMO. This would be simply a matter of pre-processing your chat queries. So I'm thinking about the :OllamaChat and :OllamaEdit commands. I don't believe for code-completion (tab-completion) this would make sense much sense, because it would slow down it too much.

I think this makes a lot of sense, my use case is with C++, I'd like to be able to have it reference function definitions defined in the .hpp/.h in the .cpp and use them to generate code, but it needs to know they exist, so while it doesn't need the whole project context, being able to specify certain files would be nice.

Ideally in the case of C++ or C you'd just run the c pre-processor to generate all the context needed for that particular file.

[normen]

The plugin should just give the model access to some functions to grep the text, list files etc. maybe even just command line access (with confirmation for each command) like copilot-cli.

[gergap]

I have an experimental branch, which solves this issue in a little different way, but it is quiet promising.
See branch ai_codegen.

It provides new commands: OllamaCreate and OllamaModify. The main purpose of this was to let the AI create multiple files and Vim directly parses the response and creates the files for us, avoiding any manual copy/paste work. I created an AI project view which tracks all the created files as "context". With OllamaModify you can change these files in the context. So this feature also fixes the request if this issue IMO.
When opening existing files you can use OllamaTrackCurrent and OllamaTrackOpenBuffers to add files to the AI context.

Here is a little demo:
https://github.com/user-attachments/assets/cd00205b-c02c-462c-957d-87d4b91f9763

I used qwen2.5-coder:7b for this demo.