Initial mini-protocols for Peras

[amesgen]

Description

Add NodeToNodeV_16 for Peras, and depend on Cardano.Base.FeatureFlag

We add a new (experimental) NodeToNodeVersion for Peras: NodeToNodeV_16.

Additionally, we add a function

getLocalPerasSupport :: Set CardanoFeatureFlag -> NodeToNodeVersion -> PerasSupport

which additionally checks that the PerasFlag is set, cf IntersectMBO/cardano-base#547. Eventually, once Peras is non-experimental, the PerasFlag will be removed again.

Peras enablement status is negociated through the field perasSupport in NodeToNodeVersionData. The decoder for NodeToNodeVersionData when version < v16 has been updated to set perasSupport to value PerasUnsupported (since this field is not part of the CBOR representation for earlier versions).

Add generic ObjectDiffusion mini-protocol

This protocol is going to be used for Peras vote and certificate diffusion. It is essentially a copy of TxSubmission, except that MsgInit and MsgDone are sent by the client/inbound side, as the flow of information is the reverse of that of TxSubmission.

We also add codec roundtrip tests, CDDL tests, direct, connect, and channel-based tests.

• MsgInit: currently pointless, but will get a payload indicating that we are only interested in objects after some indicator.

• The representation of object ids currently always is a list of object ids. We are potentially going to make this customizable to optimize for cases where a more succinct representation is possible (such as requesting "ranges" of ids).

• We might make some changes to how IDs are requested, such as allowing the server to reply with MsgAwaitReply when we do a blocking request, cf Make idleness explicit in ObjectDiffusion (V2) mini-protocol tweag/cardano-peras#144.

Register Peras vote and certificate diffusion mini-protocols

In Cardano.Network.NodeToNode, make the Network layer aware of Peras certificate and vote diffusion, gated behind perasSupportStatus from the negociated NodeToNodeVersionData.

Checklist

Quality

• Commit sequence makes sense and have useful messages, see ref.
• New tests are added and existing tests are updated.
• Self-reviewed the PR.

Maintenance

• Linked an issue or added the PR to the current sprint of ouroboros-network project.
• Added labels.
• Updated changelog files.
• The documentation has been properly updated, see ref.

[tbagrel1]

Hi @coot, this PR should be ready, and we would need to depend on these changes for another PR against ouroboros-consensus that will be (hopefully) adopted soon. Would you mind having a look? Many thanks!

[coot]

It looks decent, but there are a few issues to be solved - especially around how Pares is negotiated in the handshake.

[tbagrel1]

Hey @coot, thanks for your review!

I've been trying to modify the code so that nodes negociate the Peras enablement status using NodeToNodeVersionData as you suggested, see 5b23aa0

However, I'm wondering how to deal with this new extra field in NodeToNodeVersionData:

1. Should I remove the node-to-node-version-data-v14.cddl spec and replace it with node-to-node-version-data-v16.cddl ? Or should both version coexist?
2. From my understanding, a node that is not aware that Peras exist won't be using the new NodeToNodeVersionData format (with 1 extra field), so it won't be able to communicate with another node that is aware that Peras exists? (More generally, I don't understand/see how retrocompatiblity is handled for NodeToNodeVersionData). Should the perasSupportStatus field in NodeToNodeVersionData serialized spec be optional so that an older NodeToNodeVersionData has a CBOR representation that is still valid under the new NodeToNodeVersionData format?

Thanks in advance for your help!

[coot]

Both should coexists. We must support multiple versions on mainnet simultanously (we cannot require all SPOs to update node all at the same time).

Nodes negotiate the highest protocol version and the NodeToNodeVersionData. So Pares protocols should only be run by diffusion if the protocol vesion is high enough and they both negotiate pares in the NodeTo NodeVersionData.

There's no need for new data constructor for NodeToNodeVersionData if the parser for < v16 sets the filed to False.

[tbagrel1]

I've addressed a bunch of comments, I'll do the rest tomorrow

[tbagrel1]

I think I've addressed all the points you raised @coot :)
I'm available if any adjustments needs to be made. Thanks again!

[tbagrel1]

Hi @coot,

With ad4cbb8, I introduced and fixed the connectPipelined and channel-based tests for ObjectDiffusion, in addition to the existing directPipelined ones.

I believe this was the last item requested in the latest review round, but please let me know if I missed anything.

Regarding comments
#5202 (comment) and
#5202 (comment):

We currently have a test client/server implementation in ouroboros-network and a production one in ouroboros-consensus. Both share the typed-protocol-based ObjectDiffusion spec defined in ouroboros-network.

We now have direct, connect, and channel-based tests for the test implementation in ouroboros-network, and smoke tests (equivalent to channel-based tests) for the production implementation in ouroboros-consensus.

Not sure if this fully answers your questions. In any case, we plan to join this week’s network meeting on Wednesday to discuss this live with you (if that works), so we can hopefully reach a mergeable state soon. If you agree, could you please send us an invite (cc @agustinmista @nbacquey )?

Thanks for your help and feedback.

[tbagrel1]

• Adjust/Add changelogs

[coot]

LGTM