docs: add Gossipsub v1.4 specification draft for large payload optimizations

[adarsh-7-satyam]

Description

The Gossipsub specification repository documents versions up to v1.3, which introduced the Extensions Control Message mechanism allowing peers to advertise experimental capabilities without requiring a new protocol ID. However, the features prototyped under this mechanism — Partial Messages, Batch Publishing, IDONTWANT on First Publish, and Wait-For-Receipt Gossip — have matured to the point where they deserve a formal, stable protocol version. Without a gossipsub-v1.4.md, these features remain scattered across individual extension notes, ethresear.ch posts, and GitHub PRs, leaving implementers without a single authoritative reference when building support in py-libp2p, nim-libp2p, and other implementations.

Problem

There is no formal Gossipsub v1.4 specification. The implementation-status.md file shows that Batch Publishing and IDONTWANT on First Publish are already implemented in go-libp2p-pubsub and rust-libp2p, and Partial Messages has an open PR. These features are production-ready but lack a versioned specification document that formally consolidates them, defines their interactions, specifies normative requirements, and declares a stable protocol ID /meshsub/1.4.0.

Changes Made

Added pubsub/gossipsub/gossipsub-v1.4.md — a formal Working Draft specification for Gossipsub version 1.4 with protocol ID /meshsub/1.4.0. The document covers:

Overview and Motivation — Explains that v1.4 consolidates the v1.3 experimental features into a stable version targeting high-throughput large-payload systems including Ethereum DAS, distributed AI model updates, and large-scale event logs. Maintains backwards compatibility with /meshsub/1.3.0.

Bundled Features:

1. Partial Messages (4.1) — Formally specifies the SubOpts changes (requestsPartial, supportsSendingPartial), the PartialMessagesExtension message fields (topicID, groupID, partialMessage, partsMetadata), and normative MUST/SHOULD requirements for replacing IHAVE/IWANT with bitmap-based metadata exchange on large message topics. References partial-messages.md for full details.

2. Batch Publishing (4.2) — Specifies the interleaved scheduling algorithm for publishing N related messages to D mesh peers simultaneously. Defines the head-of-line blocking problem with sequential publishing and the normative requirement to prioritize the initial injection of every unique message before sending second copies.

3. IDONTWANT on First Publish (4.3) — Specifies the pre-emptive IDONTWANT mechanism where a node MUST send IDONTWANT to all mesh peers immediately before its own initial publish to eliminate the boomerang effect during high-concurrency publishing.

4. Wait-For-Receipt Gossip (4.4) — Specifies the path-aware propagation suppression mechanism where nodes track arrival latency (Lin) vs estimated link latency (Lout) and suppress eager pushes when Lin > Lout + threshold. Requires configurable threshold to prevent accidental network fragmentation.

Interaction with Existing Mechanisms — Documents how v1.4 features integrate with Peer Scoring P4/P7 from v1.1, RED circuit breaker from red.md, IDONTWANT limits from v1.2, and the Extensions Control Message from v1.3.

Parameters Table — Defines four new parameters with suggested defaults: max_partial_group_ids (1024), max_partial_response_bytes (2 MiB), wfr_latency_threshold (50ms), and batch_publish_fanout (8).

Complete Protobuf Schema — Updated RPC definition consolidating all v1.2, v1.3, and v1.4 additions with inline version annotations showing which version introduced each field.

References — Full normative and informative reference list including ethresear.ch posts for Batch Publishing and WFR Gossip, Ethereum consensus-specs PR 4558, and all related spec documents.

How It Solves The Problem

This document provides the single authoritative reference that implementers need to build interoperable Gossipsub v1.4 support. By formally declaring /meshsub/1.4.0, defining normative requirements with MUST/SHOULD/MAY language, and consolidating the protobuf schema, it gives py-libp2p and nim-libp2p developers a clear implementation target and a foundation for interoperability testing.

Impact

• Formally moves Gossipsub from a collection of experimental extensions to a stable v1.4 protocol version
• Provides the foundational document required for progressing toward Working Draft and Candidate Recommendation status
• Enables interoperability validation between py-libp2p and nim-libp2p by providing a shared normative reference
• Directly delivers on the core goal of the DMP 2026 project — creating a specification PR for Gossipsub 1.4 Large Message Handling

Related Issue

Closes #16