Adding NER service and dependencies of NER/NEL

[javdrher]

This PR introduces the NER service, and adds some dependencies of the AI team

• Nominatim
• QDrant

It does not fully cover additional needs from ollama for the NEL service

[Rahien]

convention is to leave the default docker.io repo out here

[brechtvdv]

Related: #57

[wdmuer]

Summary
Added entire PDF-to-ELI pipeline, as well as codelist-labeling service

How to test

1. Run docker compose pull
2. Fill in API keys in the config files: config/ner/config.json, config/pdf-content/config.json, /config/codelist/config.json
3. Add the API key as well as an environment variable in the docker-compose.override.yml for the entity-linking service
4. Run docker compose up -d
5. Run desired job

   • PDF-to-ELI:

     • choose "Harvest direct PDF link & Publish as ELI" while creating a new job on dashboard.localhost
     • Fill in one or more URLs to a PDF, or a web URL containing PDF URLs (Example: https://lblod.bredene.be/LBLODWeb/Home/Overzicht/079c62c3c9a5fe661f62f2833bb02c8c5bfe646ec97a2c1ba6055df3ba04013d/GetPublication/?filename=Notulen_Raad%20voor%20maatschappelijk%20welzijn_22-09-2025_RVMW%20notulen.pdf)
     • Choose the appropriate municipality. Note that the municipality will apply for all provided URLs!
     • You still need to pull the desired Ollama LLM in the Ollama container. To do so, execute the following command: docker compose exec ollama ollama pull LLM-NAME" (Example: LLM-NAME = mistral-nemo)

   • Codelist labeling:

     • Choose "codelist mapping (training)" or "codelist mapping (evaluation)" while creating a new job on dashboard.localhost
     • Under "Codelist", fill in the URI to the codelist (Example: http://data.lblod.gift/id/conceptscheme/sdg-simple for the SDGs)
     • Under "Decisions graph", fill in the graph URI that contains the desired target decisions (Example: http://mu.semte.ch/graphs/public/gent)
     • Note: there is a known issue that leads to the codelist pipeline being stuck on a scheduled singleton-job. To prevent this, you should make it optional for a task to have an input container. To do so, build a local image of harvesting-singleton-job-service in which you changed the WHERE clause in the query as follows:

    WHERE {
      GRAPH ?g {
        ${rst.termToString(task)}
          adms:status ?oldStatus ;
          dct:modified ?oldModified .
        OPTIONAL {
          ${rst.termToString(task)}
            task:inputContainer ?container .
        }
      }
    }

[Rahien]

remove and put in your docker-compose.override if you need it

[Rahien]

remove and put in your docker compose override if you need it, on the docker network services will be able to reach all ports

[Rahien]

remove

[Rahien]

remove. i also think that there is already an ollama service somewhere else?

[Rahien]

ah makes sense, it was moved here, please ignore and just remove the port

[Rahien]

would be best to put this kind of thing in env vars instead of config so you never accidentally commit it

[Rahien]

should this not be the ./../data/nel folder then?

[Rahien]

please DO mark all your services as restart always to that on error, they restart on the server side, then just add restart:no to the docker-compose.dev.yml