Add JSON-RPC batch support to channel_rpc and body reader#741
Closed
eynhaender wants to merge 1 commit intolibbitcoin:masterfrom
Closed
Add JSON-RPC batch support to channel_rpc and body reader#741eynhaender wants to merge 1 commit intolibbitcoin:masterfrom
eynhaender wants to merge 1 commit intolibbitcoin:masterfrom
Conversation
Member
|
This fails to send the response an an array, so is only half complete. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add JSON-RPC batch request support (Phase 1 — network layer)
Background
JSON-RPC 2.0 §6 allows a client to send an array of request objects in a single
message. Until now, any such message caused the channel to drop: the body reader
called
boost::json::value_to<request_t>()on an array root, which threw, andthe resulting
boost_codepropagated as a read error.This PR adds full batch support at the network layer. No changes to
libbitcoin-serverare required for the Electrum TCP path; HTTP batch dispatchis a separate follow-up (Phase 3).
Each response is sent as an independent socket write — one at a time, without
buffering all N responses server-side. This bounds memory use regardless of
batch size and eliminates a DoS vector. The client correlates responses by
idas they arrive.
What changed
Parsing (
rpc/model.hpp,rpc/body.hpp,rpc/body.cpp)model.hpp— addsusing batch_t = std::vector<request_t>. Noserialization
tag_invokeis needed; the reader deserializes each elementindividually using the existing
value_to<request_t>.body.hpp— adds a full template specializationmessage_type<request_t>that carries bothrequest_t message(single path)and
batch_t batch(batch path), plus anis_batch()predicate. The genericmessage_type<T>template is untouched;message_type<response_t>isunaffected. Typedefs
rpc::request,rpc::request_cptr,rpc::reader, etc.are unchanged.
body.cpp—body<request_t>::reader::finish()gains a three-branchdispatch on the JSON root type:
validation)
jsonrpc_batch_empty),rejects non-object elements (
jsonrpc_batch_item_invalid), deserializeseach element into
value_.batchvia the existingvalue_to<request_t>jsonrpc_requires_method(unchanged from priorfallthrough)
Error policy: fail-fast. A malformed batch is a protocol violation; the
channel stops. No partial-batch recovery.
Dispatch (
channel_rpc.hpp,channel_rpc.ipp)channel_rpc.hpp— two new private members (batch_source_,batch_cursor_), a new virtualdispatch_batch(), and a non-virtualdispatch_next().dispatch_batch()is virtual so derived channels canreject batch (e.g.
stop(error::not_implemented)).dispatch_next()isnon-virtual because the one-at-a-time sequencing is a correctness invariant,
not a policy.
channel_rpc.ipp:handle_receive()— routes todispatch_batch()or the existing singlepath based on
is_batch(). The prior TODO block is retained asRESOLVED:comments with a reference to the implementation.
dispatch_batch()— storesbatch_source_(shared_ptr, no copy) and callsdispatch_next().dispatch_next()— while-loops past notification items (noid), thendispatches the next request item. On unknown method it calls
send_error()and returns;
handle_send()will advance the cursor. On batch exhaustion itresets state and calls
receive().handle_send()— the finalreceive()is replaced by a conditional:dispatch_next()if a batch is in flight, otherwisereceive().stopping()— resetsbatch_source_andbatch_cursor_on stop.Error codes (
error.hpp,error.cpp)Two new enumerators appended after
jsonrpc_writer_exception:jsonrpc_batch_emptyjson-rpc batch array must not be empty[]receivedjsonrpc_batch_item_invalidjson-rpc batch element is not a JSON objectTests
test/messages/rpc/body_reader.cpp— 13 new cases insideHAVE_SLOW_TESTS:message_type<request_t>specialization default state andis_batch()predicateid), HTTPpath (no terminator)
valid/invalid, scalar root, string root
test/error.cpp— 2 new cases forjsonrpc_batch_emptyandjsonrpc_batch_item_invalid(value, truthiness, message string).Note: the pre-existing test cases in
body_reader.cppuse stale API names(
rpc::body::value_type,body.request,reader.is_done()) that do not matchthe current code and will not compile. These are tracked separately and will be
fixed before
HAVE_SLOW_TESTSis enabled in CI.Channel-level dispatch tests (
dispatch_batch,dispatch_next,handle_sendrouting,
stoppingcleanup) require a strand-aware mock that does not yetexist. Scenarios are documented in
doc/json-rpc-batch-network.md.What is NOT changed
rpc::request/rpc::request_cptr/rpc::readertypedefs — thespecialization is transparent
rpc::response/message_type<response_t>— separate generic instantiationhttp_body.hpp/http::body::reader::assign_reader()—rpc::requestnowhandles both cases internally; the HTTP body variant is untouched
dispatcher<I>::notify()— called per-item insidedispatch_next()channel_rpc::dispatch()/receive()/send()/send_code()/send_result()/send_error()— all unchangedprotocol_rpc<Channel>and all derived protocol classes — no changes