Reuse gRPC clients

[Mirko-von-Leipzig]

Currently the gRPC clients in the ntx builder are created once per network account actor. This might be the cause of #1746.

The code in question on main:

node/crates/ntx-builder/src/actor/mod.rs

Lines 176 to 184 in 1874055

	let block_producer = BlockProducerClient::new(actor_context.block_producer_url.clone());
	let validator = Builder::new(actor_context.validator_url.clone())
	.without_tls()
	.with_timeout(Duration::from_secs(10))
	.without_metadata_version()
	.without_metadata_genesis()
	.with_otel_context_injection()
	.connect_lazy::<ValidatorClient>();
	let prover = actor_context.tx_prover_url.clone().map(RemoteTransactionProver::new);

.

Instead of creating new clients, actors should be sharing the same underlying channel per endpoint, or at minimum some pool of channels.

I think gRPC/tonic already does this multi-plexing of a channel when a client is cloned, however it would be good to check this.

What needs doing

1. Confirm that cloning a tonic gRPC client shares the underlying gRPC channel i.e. only consumes a single socket/file descriptor.
2. Inspect other internal gRPC clients for similar issues.
   • Ensure RPC clients are cloned and not created anew from a URL.
3. PR a fix into next
4. PR a fix into main, linking to the PR in (2)Once we've deployed (3) to testnet, we can confirm whether this also fixes Node consumes excessive amounts of TCP connections #1746.

If (1) turns out to be false, then we'll have to figure out a connection pool instead.

[juan518munoz]

Confirm that cloning a tonic gRPC client shares the underlying gRPC channel i.e. only consumes a single socket/file descriptor.

All clones communicate with the same background connection task via a mpsc, so only one TCP socket / file descriptor is used regardless of how many clones exist.

https://docs.rs/tonic/latest/tonic/transport/struct.Channel.html#multiplexing-requests

[Mirko-von-Leipzig]

Re-opening until the fix is on main as well

[juan518munoz]

Re-opening until the fix is on main as well

I'll open a PR cherry-picking the changes if that's ok

[juan518munoz]

Should we consider this closed with #1808 merged?

[Mirko-von-Leipzig]

Yes sorry, forgot it only auto-closes on next :)