Feature Request: Implement reverse proxying, from withing netns/container

[etosan]

This took me some time to figure out, but it finally clicked! This proxy is one way only, from host into netns!

This proxy opens listener port in "host's" netns (aka netns:0 - let's call it that) and then sends itself into the target netns (aka netns:tgt), so once it accepts "host" connection and forwards it to the connection target, the forwarded connection will appear to originate from within netns:tgt and will use netns:tgt's default route a and routing table.

This is cool and fine, but is something that docker/podman and other container runtimes already implement, and it is akin to "reverse proxying" into the netns's network, like when using nginx or haproxy balancer in front of application nodes.

What I have yet to see and what I am searching for, and that nobody besides perhaps socat implemented (and I haven't had time to try it there yet), would be exactly the reverse situation:

• keep main part of the process running in netns:0 and send listening socket into "container" ie netns:tgt
• then when processes bound to given netns:tgt connects to let's say it's lo: (127.0.0.1) or dummy: (x.x.x.x)
  • the connection from inside netns:tgt would then be forwarded to netns:0, and from outside would appear as if originated from netns:0, following hosts default route and routing table
  • but it would be in fact originating in netns:tgt

I am interested into this to safely enclose "lower-trust"/"untrusted" process inside it's own netns, yet allow it to connect outside. Modularly and "unix-composably", i.e. so it plays nicely with tools like netcat, ucspi-tcp protocol tools etc.

Circa 7+ years ago I implemented something like that of FreeBSD using jails and kqueue(), but there, it was relatively straightforward in C. On linux I played around with netns in C and such, but the code required would be much bigger, not mentioning the eventloop core on linux, of which I know only of poll(), and would be well beyond my resources to implement atm.

Would you consider this special use-case as one of valid uses of your tool?

It would allow insides of netns:tgt to chain socks proxies for example wrom within it. But even just single port+connection forwarder would be great achievement for many tasks. In my experience netns separation is 100s times more safer and more secure isolation strategy than, for example, firewall rules, or, god forbid, seccomp.

But maybe you have different aims for this project, I would understand that.

[fooker]

Thanks for your feedback. I'd admit that the README could make this more clear.

I don't see implementing a SOCKS proxy as in scope of the project, as this would involve a whole set of additional protocols.

In theory it should be possible to to user netns-proxy as is and spawn it from inside the namespace as long as the root namespace is somehow reachable.

I'd accept a PR that "reverses" the direction or, to be more generic, moves the listening socket to an netns while defaulting to the one the process has been spawned in.

[etosan]

Sorry for misunderstanding, and not being clear enough: I am not suggesting/requesting SOCKS support at all! That can be already done by other software (like with ssh-embedded/dedicated SOCKS proxy packages) much better. Just that with reverse direction, chaining for example "external" (but potentially local) socks proxy would be possible, through help of this little gem.

But to confirm: all I want is reverse direction form within netns:tgt, as probably you got already, reading your last paragraph.

Just so that we are sure, that we understand each other, I made this little schematic to better explain it:

host:netns:0:[
  ssh:listen(socks5://127.0.0.1:555)<=netns-proxy[host-part]:connect(tcp://127.0.0.1:555)
]<->netns:tgt[
  <=netns-proxy[ns-injected-part]:listen(tcp://127.0.0.1:555)
]

From what I gather, listen() listened socket is bound to namespace, where the syscall was called. I am not entirely sure whether even initial socket() call needs to be made in that namespace. I guess for certainty, I would call socket() in target netns as well.

Due to nature of the netns beast, I would guess fork() for injecting the listener into netns:tgt is needed. To avoid "double pump" between two processes, I would fork() small injector child process, enter netns:tgt, did socket()+listen() there, and once listening syscall is successful, I would send the listening socket FD back over premade socketpair(), to a parent netns:0 instance, and terminate the injector child itself.

This would prevent helper process from lingering around (confusing namespace managers) and should allow direct accept()s and data transfers from withing the netns:tgt in the netns:0 ns - I hope. But this is just my conjecture of the black magic and how I would attempt it.

The problem is, that I tried to look at the code, but I realized that I am old fart now, and don't understand rust code at all. So it would be impossible for me to try this, unless I learn at least some rust (which I heard is really hard).

I guess you won't have time to attempt it either, I presume...?

EDIT:
Regarding this statement:

In theory it should be possible to to user netns-proxy as is and spawn it from inside the namespace as long as the root namespace is somehow reachable.

my main use model would be "injecting" netns-proxy:reverse from "host" or parent nses into target nses, which is what I, as administrator, would like to do. Spawning netns-proxy in netns:tgt namespace directly and somehow leak out of it into netns:0 seems like security issue to me.

[fooker]

I see. Got confused by the example.

my main use model would be "injecting" netns-proxy:reverse from "host" or parent nses into target nses, which is what I, as administrator, would like to do. Spawning netns-proxy in netns:tgt namespace directly and somehow leak out of it into netns:0 seems like security issue to me.

I'm not sure about the security implications. As an admin, I could spawn a program inside an netns as long as I've access to a path bound to that netns.
Something like

nsenter --net /proc/$CONTAINER_PID/net/ns -- netns-proxy /proc/1/ns/net 127.0.0.1:555

As this will only enter the network namespace, but no other namespaces, it should work without leaking the namespace.
BUT this is totally untested.

I will keep this issue open as a reminder. Maybe, I can find some time to implement this.

[etosan]

Hmm, yeah I guess this could work too:

nsenter --net /proc/$CONTAINER_PID/net/ns -- netns-proxy /proc/1/ns/net 127.0.0.1:555

Great idea! So if I read it correctly, this moves listening process context of netns-proxy inside netns:tgt with nsenter where the listener port is spawned, and then it makes netns-proxy "connecting context" escape through pivoting over "host's" PID1 (as long as PID namespace is the same for both processes) ...

But aren't we missing the --bind & --port there, though? More like:

nsenter --net /proc/$CONTAINER_PID/net/ns -- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

?

Also, I am not at the machine, where I compiled this atm, but I will try this ASAP. Seems promising.

Then it is some time, since I played with nsenter as well, so...

But if I am not mistaken, that invocation would still leave parent nsenter lingering around, I presume, as a waiting parent of netns-proxy, and this could interfere with signalling (like in service supervision schemes):

+ supevisor
  |
  +--- nsenter --net /proc/$CONTAINER_PID/net/ns -- ...
       |
       +--- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

While proper supervision process tree should look like:

+ supevisor
  |
  +--- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

Thus to get to "signallable/killable" single process, maybe something more like this would be in order, perhaps:

nsenter --no-fork --net /proc/$CONTAINER_PID/net/ns -- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

I am glad, I got to talk to you about this, I will experiment further with this when I get time.

Still, it would be really great, if netns-proxy (and I understand it is lot of unshare()/setns() work) could do supervision ready netns entry on it's own. That would be awesome.

As I said, there are many tools that do what this does, but probably nothing, that does the reverse (at least that is what I believe).

[fooker]

But aren't we missing the --bind & --port there, though? More like:

nsenter --net /proc/$CONTAINER_PID/net/ns -- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

There is no --port option, as it is already part of the --bind. And the --bind defaults to "everthing" plus whatever the target port is.

But if I am not mistaken, that invocation would still leave parent nsenter lingering around, I presume, as a waiting parent of netns-proxy, and this could interfere with signalling (like in service supervision schemes):

+ supevisor
  |
  +--- nsenter --net /proc/$CONTAINER_PID/net/ns -- ...
       |
       +--- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

While proper supervision process tree should look like:

+ supevisor
  |
  +--- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

Thus to get to "signallable/killable" single process, maybe something more like this would be in order, perhaps:

nsenter --no-fork --net /proc/$CONTAINER_PID/net/ns -- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

I am glad, I got to talk to you about this, I will experiment further with this when I get time.

This depends on how you supervise your services. Systemd has tools to switch the namespace for a service, too.

Still, it would be really great, if netns-proxy (and I understand it is lot of unshare()/setns() work) could do supervision ready netns entry on it's own. That would be awesome.

As I said, there are many tools that do what this does, but probably nothing, that does the reverse (at least that is what I believe).

I'm not sure about the many available tools and how they compare. Would like to see that out of pure interest.

netns-proxy does not fork right now. I'd like to try to keep it that way. But I think this can be done without forking anyways.

[etosan]

But aren't we missing the --bind & --port there, though? More like:

nsenter --net /proc/$CONTAINER_PID/net/ns -- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

There is no --port option, as it is already part of the --bind. And the --bind defaults to "everthing" plus whatever the target port is.

: target/debug/netns-proxy --help
Forwards incoming requests to a given target while outgoing connections are created from different namespace

Usage: netns-proxy [OPTIONS] <NETNS> <TARGET>

Arguments:
  <NETNS>   The namespace to open connections from. Can be namespace name or full path
  <TARGET>  Target to forward requests to

Options:
  -p, --proto <PROTO>  The network protocol to use [default: tcp] [possible values: tcp, udp, sctp]
  -b, --bind <BIND>    Listen on incoming requests
  -v, --verbose...     Verbose mode
  -h, --help           Print help
  -V, --version        Print version

Sorry my bad, I went from memory and I confused --port with --proto.

While proper supervision process tree should look like:

+ supevisor
  |
  +--- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

This depends on how you supervise your services. Systemd has tools to switch the namespace for a service, too.

Yes of course, and those cgroups great, but that is completely orthogonal.

Most alternative supervision schemes work without cgroups completely. If explanation is in order, it can be provided. Problem with using nsenter command would be, that it would be acting as interposer process, breaking many non systemd supervision toolkits. Should the netns:0 & netns:tgt switching dance be builtin, this is non issue. As long as main serving process is the same PID as PID that was started, it will work.

systemd uses cgroups solely to track legacy doubleforking services and user process populations, it does not affect supervision.

To conclude: as long as you don't ever doublefork, and keep operating as you are now, you are automatically compatible with modern standards, both systemd and supervison family (daemontoolds, runit, s6).

I'm not sure about the many available tools and how they compare. Would like to see that out of pure interest.

I meant not standalone, of course!

And yes, as far as I know, your tool is only tool, able to do this standalone, that is true.

Perhaps socat too, maybe, but I haven't had time to play with it. I noticed netns word in their manual recently.

But I meant all docker, podman, cri-o, kubectl proxy provide somewhat same thing netns-proxy does, but as an un-separable part of the whole package.

netns-proxy does not fork right now. I'd like to try to keep it that way. But I think this can be done without forking anyways.

I thought, that if you want to keep connector part running in netns:0, and listener in netns:tgt, it is impossible to pass the netns barrier without forking (sorry BSD jails thinking) and spawning helper child to do stuff there. It certainly requires fork for procns, I guess it's not a requirement for netns, then, would make sense.

So do you plan to go about it? Do you plan to preserve netns:0 on some FD, setns() to netns:tgt, spawn listerner, and setns() back to netns:0 for accept() loop?

[fooker]

Sorry my bad, I went from memory and I confused --port with --proto.

No worries.

While proper supervision process tree should look like:

+ supevisor
  |
  +--- netns-proxy --port 555 --bind 127.0.0.1:555 /proc/1/ns/net 127.0.0.1:555

This depends on how you supervise your services. Systemd has tools to switch the namespace for a service, too.

Yes of course, and those cgroups great, but that is completely orthogonal.

I'm not talking about cgroups but about the NetworkNamespacePath option of systemd.exec.

Most alternative supervision schemes work without cgroups completely. If explanation is in order, it can be provided. Problem with using nsenter command would be, that it would be acting as interposer process, breaking many non systemd supervision toolkits. Should the netns:0 & netns:tgt switching dance be builtin, this is non issue. As long as main serving process is the same PID as PID that was started, it will work.

nsenter --no-fork should to the trick, right?

I thought, that if you want to keep connector part running in netns:0, and listener in netns:tgt, it is impossible to pass the netns barrier without forking (sorry BSD jails thinking) and spawning helper child to do stuff there. It certainly requires fork for procns, I guess it's not a requirement for netns, then, would make sense.

So do you plan to go about it? Do you plan to preserve netns:0 on some FD, setns() to netns:tgt, spawn listerner, and setns() back to netns:0 for accept() loop?

I'd add an option for the outgoing netns path, open both FDs (outgoing and accepting) as early as possible, switch to the "accepting" one, open the accepting socket, switch to the "outgoing" one and run the loop.