Llm based localhost proxy server

[posix4e]

I removed the local model requirement

In puppet add a http://tinyproxy.github.io like server but powered by a llm

• Add a simple http server to allow us to query it
• Add a http proxy interface, that acts as an adblocker for all localhost http requests. You can also use a VPN if that's easier.

[posix4e]

Make sure to read https://chat.openai.com/share/85a33a1a-8b2b-4942-b82a-317946dfd425 it includes how I'd do it.

[urbit-pilled]

It looks like people are having trouble with compiling for arm64 architecture in the gpt4all repo, but it should be possible just need to fix some things in the gpt4all repo to make it work with arm.
If you're ok with the speed of an llm running on a mobile and the RAM requirements, then this should be fine.

[posix4e]

For now don’t worry about the local llm and do the rest, we can switch to local
Later

[krrishdholakia]

Hey @posix4e @urbit-pilled slightly confused - why do you need a proxy here?

[urbit-pilled]

A proxy will allow us to disable/allow network requests based on if they contain ads. (However, I am going to be using VPNService instead)

[traphique]

One way to add a VPN to the Puppet project is by using the jethrocarr/roadwarrior module available on the Puppet Forge. This module configures a road warrior style IKEv2 VPN with StrongSwan and certificate-based authentication, which is ideal for mobile devices.

Try;

class { 'roadwarrior':
  manage_firewall_v4 => true,
  manage_firewall_v6 => true,
  vpn_name => 'vpn.example.com',
  vpn_range_v4 => '10.10.10.0/24',
  vpn_route_v4 => '192.168.0.0/16',
}

This will set up the firewall rules, defines the VPN name, and specifies the IP range to use for the clients as well as the IP range to route back to the client devices. Back up the /etc/ipsec.d directory, as you will want to populate it with the same data across your fleet if you are replacing or autoscaling the server running your Road Warrior VPN.