Have bernhard attempt to set SO_REUSEADDR.

[kainz]

On an app with heavy TCP traffic, this will minimize the use of extra
TCP IP/port pairs.

[b20n]

I'm not following the need here. Why is your app creating and tearing down clients with any frequency?

[kainz]

My app is not, any longer. An older revision of it was. That said:

If you have any process opening and closing TCP client sockets quickly to a given server (say, your DB), if you are not using SO_REUSEADDR you will fail to connect to the remote end due to port exhaustion.

The root cause is port exhaustion due to sockets in TCP TIME_WAIT state.

On a linux system with default TCP stack settings you have roughly 16384 available ephemeral ports (per destination address), and your defaut TIME_WAIT timeout is two minutes. That roughly means that you can trigger port exhaustion in this situation with as few as 136 connections/sec.

Varnish, nginx, and haproxy are examples of applications which could easily trigger this situation. Setting SO_REUSEADDR on client connections mitigates this by allowing port number/ip address combinations to be reused before that timeout in certain situations.