~~proof of reserves,~~ proof of a transfer#799
~~proof of reserves,~~ proof of a transfer#799skaunov wants to merge 13 commits intoNeptune-Crypto:masterfrom
Conversation
aszepieniec
left a comment
aszepieniec
left a comment
There was a problem hiding this comment.
I left some comments inline, but I think those superficial comments are subordinate to high-level directional feedback which follows here.
Please explain how the CLI commands ought to work. What information do they take, and what information do they produce? How these CLI commands work will establish constraints that the tasm program must satisfy.
I'm thinking out loud here. What happens if you (re-)derive a perturbed RemovalRecord. Essentialy a different AsboluteIndexSet derived the same way except for also including the block hash in the input. Then your proof certifies the claim "this PerturnedAsboluteIndexSet tailored to block X, corresponds to a UTXO in the UTXO set at block X (so in particular it was confirmed earlier and has not been spent yet), amounts to Y Neptune coins with release date Z." So the claim is (W: PerturbedAsboluteIndexSet, X: BloclHeight, Y: NativeCurrencyAmount, Z: Timestamp).
With this construction, you can combine (claim, proof) pairs. What you care about is that all the proofs are valid, the block heights are identical, and all the PerturbedAsboluteIndexSets are unique.
If that's the goal, then the CLI commands probably look like something like this:
neptune-cli prove-reservestakes
- a UTXO index
- a block height
- an output file name
and writes the (claim, proof) pair to the given file.
neptune-cli verify-reservestakes
- an input file name or list of them
and verifies each proof, verifies that allPerturbedAbsoluteIndexSets are unique, and outputs one line per file containing the amount and, if any, release date.
| /// to parallelization. | ||
| /// | ||
| /// # running | ||
| /// Correct me if I'm wrong: for proving a new Neptune CLI command should be added. |
There was a problem hiding this comment.
/// Correct me if I'm wrong: for proving a new Neptune CLI command should be added.
Yes that makes sense.
| /// # running | ||
| /// Correct me if I'm wrong: for proving a new Neptune CLI command should be added. | ||
| /// # checks out of Triton | ||
| /// - AOCL must be from the block of interest |
There was a problem hiding this comment.
/// - AOCL must be from the block of interest
More precisely: the AdditionRecord must live in the AOCL of the block of interest.
There was a problem hiding this comment.
that's proven and the AOCL digest is output, so a verifier must find the digest in a canonical block
| /// | ||
| /// # running | ||
| /// Correct me if I'm wrong: for proving a new Neptune CLI command should be added. | ||
| /// # checks out of Triton |
There was a problem hiding this comment.
Checks outside of Triton VM, meaning that the verifier verifies the proof and some extra data. Doesn't this extra data not leak privacy? For instance, if you need to include the AdditionRecords, then that leaks exactly when the reserves were obtained.
There was a problem hiding this comment.
That's why it keeps these private. X) And the prover can prove for the latest block AOCL which is relevant to the task in hand.
In a sense a good proof of the reserves requires to leak some privacy. Like it makes sense to leak the lock post-image. So the question is where to draw the line. I start the current version with the comment on the choice I present here.
| NativeCurrencyAmount, | ||
| ), | ||
| } | ||
| impl crate::protocol::proof_abstractions::tasm::program::ConsensusProgram for PublicData { |
There was a problem hiding this comment.
Consider naming the program something more specific than PublicData. Just about every proof type used in NeptuneCash has some kind of public data.
| crate::util_types::mutator_set::ms_membership_proof::MsMembershipProof, | ||
| >::default(), | ||
| )); // TODO | ||
| let payload = triton_asm! { |
There was a problem hiding this comment.
What is the stack signature of this blob of tasm code?
There was a problem hiding this comment.
Oh, I had this question! How do you denote the empty stack? Jan Ferdinand told me recently the stack is never empty, but I feel like that doesn't impact a stack signature.
There was a problem hiding this comment.
There are always at least 16 elements on the stack. Any less -- crash!
If the stack contains nothing interesting, you can denote it with // _.
|
|
||
| /* finish of the things from the RR integrity file | ||
| ____________________________ */ | ||
|
|
There was a problem hiding this comment.
Please add a stack signature.
| /* pasted '8.' from <removal_records_integrity.rs> | ||
| ============== */ | ||
| /* 8. */ |
There was a problem hiding this comment.
But what does "8." do? Please summarize.
If significant chunks of tasm code is being copied, consider factoring it out into a separate blob or even BasicSnippet. In this case, an appropriate name for the blob or snippet will obviate the need for explanatory comment.
There was a problem hiding this comment.
It's not certain that subsections of RemovalRecordsIntegrity can be factored out into a BasicSnippet without changing the function signature of RemovalRecordsIntegrity, which would be a hard-fork (which we certainly don't want!).
Maybe something can be factored out into a block of code but even that is not certain, as the relevant chunk of code in RemovalRecordsIntegrity might depend on a very specific stack setup, or statically-allocated memory addresses.
|
I'm not sure I entirely grasped your idea. My design goal here was enabling someone with just the verified proof and an explorer to watch the proof is still valid. I mean they can even set a notification in such an explorer on when the UTXO backing the reserves is spent. And I don't yet understand how to achieve this with that approach. Another problem seem to re-arise here is that one can reuse the UTXO in the different blocks.
What would be the benefits to combine it? Why would someone want to put their resources on that? I can only think of putting a proof on-chain, but that implies an use-case where it should be proven recursively. I can think of that, but it feels so much in the future yet.
|
That is quite different from what I propose. Can you be more explicit? How to represent the claim and what does that sound like in human words? In my proposal, the proof is tailored to a block. And so if a new block is found, you have no guarantees about the reserves any more -- they could have been spent and you would not know. Your proposal requires publishing the
I don't think this is correct. The proof is tailored to a block. Different block -> proof is invalid. Different proof -> |
I'm thinking that exchanges typically sit on thousands of UTXOs. If they have to produce one proof covering all UTXOs, that might be prohibitively expensive -- unless if you use recursion to combine multiple smaller proofs ;-) And putting the proofs on chain is not the only application. They could publish the proof on their website. Or make smart contracts that are capable of verifying them -- in this case the proofs do not need to be on-chain because they can be non-deterministic input to the smart contract. |
|
Thank you, I expected that this thing will take some discussion and am glad to look at the thing from different aspects! As I tried my best to address below the concerns which were noted I finally could formulate what was bugging me! I believe without disclosing the RR it should be called a proof of historical reserve since each time a verifier receives the proof he should expect that a tx spending this reserve is already in the mempool. Could you paint a use-case, pls. At least what resources an exchange would need for this on their thousands of the UTXO. Like I'm thinking about an exchange which doesn't do mining and doesn't want to pay a lot for proving but publishes their reserves to attract the users. They say: sorry we can't afford to prove every block so we do it less often (idk every other block, or once a day). Which leave different ways to reuse the funds to make the proofs for amounts more than the coins holding.
Absolutely, just guide me with questions, pls -- I tried to do that in that explanation already. X) I might be wrong and am not insisting but here's my reasoning. A verifier don't want the proof to be invalidated by a new block and to have this window of uncertainty / leap of faith between discovery of a new block (even not a canonical!) in the network and receiving the proof for that block. A prover hardly appreciate the cost of proving for each block too. I mean what the problem (what's exposed) when the RR are published for the reserves: the public will know the date the UTXO existed already, the public will track when the UTXO is spent, the amount of the UTXO, and the address unlocking it. To me it looks like exactly what you want when checking someone reserves, so I'd call it 'selective disclosure' instead of privacy compromisation (as the consequences are controlled, confined, and wanted). Also it feels like a smaller hurdle to prepare the UTXO for this than making a proof per block.
Correct me if I'm wrong: you're still speaking here about one proof for the whole publication amount? My point that currently it makes more sense to produce a proof per the UTXO. And I thought it would be actually cheaper to do. As some UTXO get spent they can add more proofs for the fresher UTXO, and not immediately but as they see fit to show their reserves. Providing them all from a website is what I had in mind. Also per an UTXO approach allows smaller guys to make the proves for other than an exchange purposes (and cheaper proving and not needing to do it for each block makes the difference for a smaller setup). I thought about using in a smart contract too, but with the current state of the smart contracts making the UTXO for it seems very feasible work-around.
|
I think that's a fine name stressing the important downside. Let's use this name going forward.
If the hypothetical exchange is sitting on 600 UTXOs, and if the time cost to produce one proof of historical reserve is 1 second, then the exchange cannot feasibly prove all reserves with every block. Unless they parallelize this task but even then the cost grows rather quickly. "It's too expensive so we do it less often -- once a week." All good, except: that defeats the purpose because no-body in their right mind is going to accept a one-block-old proof of historical reserves, let alone a one-week-old one. What they can do instead is put up a bond, say 10% of the claimed reserves, into a smart contract that releases it to a challenger if the the exchange fails to produce a full proof of historical reserves within, say, one day. The challenger has to pay to activate the challenge. As long as no-one challenges, the exchange can save on the cost of proving. And if they do challenge, then the challenge fee can pay for the cost of producing the proofs.
I think you make a compelling case for having durable proofs of reserves -- proofs that don't expire unless identifiable removal records are spent. But I do think there is a trade-off; the question is whether the cost of the information leakage is worth the benefits. Clearly, if proving were free and instantaneous, then there would be no benefit.
The public will know:
I'm thinking that the first three are unavoidable for durable proofs of reserves. By making a single proof over all UTXOs in custody, you could publish only a lower bound on the total amount. Except of course: that single proof will expire once any one of the individual UTXOs is spent.
Yes. If you aggregate proofs, you improve privacy. That holds for both proofs of historical reserves as well as durable proofs of reserves. For durable proofs of reserves aggregation has the downside of making them more prone to expiry, thus defeating the purpose to some degree. |
|
Yes! Let's allow this to sit for a few days to think which is the better way/path to take now. (Perspectively both can just co-exist.) Meanwhile I can just start to really look into a transfer proof which probably just doesn't has such public/private tug. P.s. I thought I've added those privacy/anonymity sets reduction that publishing of a RR does, but I thought they're quite insignificant and ultimately forgot to edit this in. I mean that Monero approach seems to be working here: if an user shared his like viewing key but doesn't want to show the transfer he just includes the intermediate tx to a burner address, which restores all the properties (especially when it's not back-to-back, which seems to have even less effect in Neptune). |
aszepieniec
left a comment
aszepieniec
left a comment
There was a problem hiding this comment.
For proof-of-payment, the claim should have the following format.
- program digest: hash of proof-of-payment program
- input:
- UTXO
- sender randomness
- receiver digest
- aocl leaf index
- block digest
- output: (empty)
Including the entire UTXO means that this proof-of-payment mechanism works for other tokens beyond NativeCurrency, in addition to NativeCurrency coins. Moreover, in this case you do not need to take explicit care of a potential TimeLock. Lastly, note that the lock script hash is already part of the UTXO.
I'm afraid I cannot make sense of how the proof-of-reserves is supposed to work. Could you write a draft RPC endpoint? That ought to be enough that ought to be enough to reverse engineer my way through.
Make sure to add tests for the programs. Both positive tests and negative tests -- the latter ones trigger specific assert codes.
| } else { | ||
| Err(todo![]) | ||
| } |
There was a problem hiding this comment.
A let-else pattern instead of an if-let-else pattern avoids needless indentation and puts the failure actions close to the failure condition.
There was a problem hiding this comment.
I don't know how to avoid return with it, so I use it only for procedural/effects things, as return takes much more effort/attention to reason about the types compared to identation and code folding. If that's possible, it would be magnificient if you could show/teach me how to employ let _ else without return.
And you're absolutely right on everything else: just added ? as a better, more idiomatic, and types-friendly solution to that!
| } | ||
|
|
||
| #[derive(Debug)] | ||
| pub struct The(WitnessMemory, Claim); |
There was a problem hiding this comment.
Please avoid overly generic struct names.
There was a problem hiding this comment.
I mostly covered this in the other comments. 1) Just give it a better name. 2) In the example/RPC it doesn't even come up explicitly. I don't remember if I was already saying this: I was thinking to promote it the module up, but then the data and the logic in the different files which is sad. And here I just want to communicate that it's the only structure of the module which is not really important per se, like "focus on the other items in the module."
| @@ -0,0 +1,2 @@ | |||
| pub mod reserves; | |||
| pub mod sent; | |||
There was a problem hiding this comment.
I would suggest that "proof of payment" or even just "payment" is a better description of what this module is about.
| /// `receiver_digest`; | ||
| /// `release_date` ([add `MINING_REWARD_TIME_LOCK_PERIOD`](https://github.com/Neptune-Crypto/neptune-core/blob/5c1c6ef2ca1e282a05c7dc5300e742c92758fbfb/neptune-core/src/protocol/consensus/type_scripts/native_currency.rs#L365)) | ||
| input: (Digest, Timestamp), | ||
| /// `lock_postimage` of the address; | ||
| /// the record to check if the reserve is still unspent; | ||
| /// AOCL digest; | ||
| /// the 'reserve' | ||
| /// the timelocked 'reserve' | ||
| output: ( | ||
| Digest, | ||
| crate::util_types::mutator_set::removal_record::RemovalRecord, | ||
| Digest, | ||
| NativeCurrencyAmount, | ||
| NativeCurrencyAmount, | ||
| ), |
There was a problem hiding this comment.
These types are too complex. Please separate into many fields and implement convenience functions input() and output().
For tests it seems I'd need a wallet with an outgoing tx. I glanced over the tests which |
|
I'm constraining an UTXO by its (I was a bit surprised to see that (Also, it was a surprise the basic snippet takes a pointer to the element next to the SI.) I'm out of ideas how to further fix this, so your advice would be very helpful! |
Unfortunately, I don't know where to find this one.
I don't think this is correct. The size-indicator does take into account fields with a statically known size. Are you sure you are not looking at the size-indicator for
I'm not sure what the problem is; can you describe in more elaborate terms please? |
I discovered it with https://github.com/skaunov/neptune-core/blob/16038b1e79469d1c5180a781af328ee3d0145ea4/neptune-core/src/application/util_proof/sent/tests.rs#L135: length is obviously 18 but the first element which is SI is 13. |
|
Size-indicator value of 13 suggests to me something like this layout.
So I do think the size indicator you're looking at is the size indicator of the field |
I remembered this imprecise -- I thought just to hit each assertion. But a LLM reminded me there are often error ids (sadly I'm out of the LLM tokens), and I guess that's what you meant. If so, my first question is how do I choose a new error code value/number? |
|
The document Feel free to define your own error codes and add to this file. |
|
Thank you! An unexpected format tbh. I can't stop thinking why it's not a Rust file which would be included in the docs. |
@aszepieniec , could you give a review again. Proof of a transfer progressed a lot and I'd like to get directions on finishing it further; especially yours suggestions that I don't feel enough to press resolve but they need another look, a discussion, or an OK. Meanwhile I'll |
I'm not sure if https://github.com/skaunov/neptune-core/tree/reserves_sketch is what you meant -- pls guide if not. But it's to demonstrate the idea: fast verification and fast proving thanks to a proof per an UTXO. The down thing is it discloses the removal record in a claim, but I still argue it's not really an issue since the owner always can choose to restore his funds privacy just by sending it himself. Then the reserves will be not proved anymore -- a verifier would see the removal record on the chain. For huge numbers of the UTXO this design is actually better then a compression via succinctness as a prover only needs to make the new proofs/arguments only for the spent UTXO and they're so much easier than a proof over few UTXO. And doesn't need to redo the proof(s) for each new block. It only worse for putting such a proof on-chain. But 1) that would be a different story, 2) can be worked around via putting the reserves into a single UTXO. |
aszepieniec
left a comment
aszepieniec
left a comment
There was a problem hiding this comment.
Please use meaningful struct names, such as e.g., ProofOfPayment and ProofOfPaymentWitness. I cannot orient myself without these markers. Once you've applied that refactor I will scrutinize the tasm code. The spec looks correct at this point.
Please add documentation:
- as docstrings on the API functions, including an example
- as a separate .md file in the mdbook.
I think the thing you want to achieve is a little under-specified and you need to make some choices. There may be other questions but on my mind is this one: disclose the UTXO or not? If not, then I see no reason to disclose the sender randomness. Also, if not, then I don't think allowing any other coins but one NativeCurrency is a good idea -- how else will the recipient prove knowledge of the UTXO when they try to spend it? But if yes, then you need to decide over which channel to send the UTXO (as it does not have to be the program's input or output -- the hash of the UTXO suffices there).
In general I am concerned that the recipient of a payment that comes with a proof of payment, can still be incapable of spending the "received" UTXO. I wonder if there is a unit test or property of the source code that can convince me or other readers that this nightmare can never happen.
| /// A [`NeptuneProof`] will argument that the tx UTXO (determined by the indices, see below) transferred | ||
| /// the `amount` (it discloses) of [`NativeCurrency`] to the address with the components it discloses | ||
| /// (`receiver_digest` & `lock_postimange`). The UTXO is binded by hashing `sender_randomness` which serves for | ||
| /// identification between the similar transfers. | ||
| /// | ||
| /// The data is taken from the wallet of this node. `tx_ix` & `utxo_ix` are indicies for getting the UTXO you | ||
| /// want to prove from the wallet. | ||
| /// | ||
| /// The `block` is needed to prove the UTXO was mined. It's determined by its [`Digest`], the relevant data | ||
| /// is taken from this node DB. During verification the same data will be pulled from the same block. | ||
| /// | ||
| /// A verifier should use [`super::super::super::protocol::proof_abstractions::verifier::verify`] exposed on his API/RPC, or use the [TUI](https://github.com/TritonVM/triton-tui). | ||
| /// | ||
| /// # Panics | ||
| /// The implementation detail is when `tx_ix` is out of its bound it crashes the node until | ||
| /// <https://github.com/Neptune-Crypto/neptune-core/issues/816> is done. |
There was a problem hiding this comment.
I appreciate this docstring but it's not perfectly clear to me yet. Let me try to rephrase and then you can correct where I made the wrong guess.
Produce a proof of payment.
The result is a [`NeptuneProof`] that certifies that a given transaction output transfers the given
amount of [`NativeCurrency`] to the identified receiving address. The amount, the addition record,
the sender randomness, and the `receiver_digest` and `lock_postimage` (which identify the
receiving address) are disclosed by the claim. Besides verifying this proof against the claim, the
verifier must also test membership of the given addition record in the AOCL.
It is a zero-knowledge proof of knowledge revolving around a [`Utxo`] (not disclosed) that hashes
to the given addition record using the sender randomness and receiver digest. The UTXO in
question has the given `lock_script_hash` and k > 1 [`Coin`]s of type script hash [`NativeCurrency`]
whose states are valid positive amounts and sum to the given amount.
Also:
It is not clear to me from this description whether the UTXO is disclosed or not. It strikes me that there might be a problem if the UTXO is not disclosed: I produce a proof of payment for a UTXO that contains, besides the NativeCurrency coin of the right amount, some other coin. The recipient has no way of spending this UTXO. One way to get around this obstacle would be to prove instead that the UTXO has only one coin, which is the NativeCurrency coin with the given value.
I don't get this sentence:
The UTXO is binded by hashing
sender_randomnesswhich serves for identification between the similar transfers.
If I read correctly, this function does 3 things that I probably would have written as separate functions:
- Gather the data that the prover needs.
- Produce the proof.
- Produce the claim.
There was a problem hiding this comment.
Also, please add a doctest example how to use prove in conjunction with verify.
There was a problem hiding this comment.
a doctest
I spent some time thinking about this when working on the earlier an example suggestion. As it needs a mined spent tx, two addresses, and the wallet --- could you show me anything similar in the code, I'd build off something existing, especially the testing/mocking utils.
another Coin
I remember we had the discussion on this like two months ago after you explained me the mechanism in general on the Discourse. (I don't remember where was the discussion - on IRC, or Tg, or there, or here. 😵💫) I left with the feeling I raised this concern and we concluded it's not an issue at this stage. Like a thing which should be added when this simpler approach/iteration works (in the wild). I think adding an issue on it when merged is the best choice.
The rephrase preformat...
...seems to be more like a summary of the code, but that's just me. Coin can be single. But generally no objects for it instead the docs I wrote.
Two items mentioned are not disclosed!
- The sender randomness digest is disclosed which serves just to be able to distinguish the proofs/arguments with the same parameters. Does that helps with the quoted sentence?
- The addition record is only proved AFAIR.
The three things
are there indeed. I'd make it a post-merge issue just because I'd separate it not without fine types as all the interactions are not that general/reusable, but introducing the types now could restrict the next proof features designs. I mean it would make sense to do that basing on the needs and hopefully more similar modules to structure.
| let sent = sent::new( | ||
| sent::claim_outputs( | ||
| sent::claim_inputs( | ||
| tasm_lib::triton_vm::proof::Claim::new(sent::hash()), | ||
| tx_output.receiver_digest(), | ||
| // TODO this can be developed further | ||
| utxo.release_date().unwrap_or_default(), | ||
| ), | ||
| sender_randomness.hash(), | ||
| // aocl_digest, | ||
| utxo.lock_script_hash(), | ||
| tx_output.native_currency_amount(), | ||
| ), | ||
| block_aocl, | ||
| sender_randomness, | ||
| aocl_leaf_ix, | ||
| utxo, | ||
| aocl_membership_proof, | ||
| ); |
There was a problem hiding this comment.
This code block gives me no clue about the type of the variable sent. Furthermore, since "sent" is a very generic word, I have no idea what it does or is supposed to do.
There was a problem hiding this comment.
Reading ahead slightly, I guess that the type of sent is a struct that implements ConsensusProgram. Is that right? If so, why not call StructName::new explicitly?
There was a problem hiding this comment.
Because I'm bad at naming entities. I'm glad to put a better name on this.
I found it more natural to use the module name, but am not insisting at all.
|
|
||
| // TODO tune `cases` but the single speeds up the development | ||
| #[test_strategy::proptest(async = "tokio", cases = 1, rng_seed = RngSeed::Fixed(0))] | ||
| async fn property_test_happy_path( |
There was a problem hiding this comment.
Please describe the property that you are testing in the name of the test. The motivation is so that you can write
cargo test test_name
and run only one test without triggering a whole bunch of pattern matches.
|
|
||
| // Consolidated negative test: AOCL proof verification failure. | ||
| #[test_strategy::proptest(async = "tokio", cases = 1, rng_seed = RngSeed::Fixed(0))] | ||
| async fn aocl_proof_verification_failed( |
There was a problem hiding this comment.
Good job! This test function has the right template. Please add one for every assert statement in the specification / tasm code of the TritonProgram.
There was a problem hiding this comment.
It's the only one left. 🤷 Other potential assertions were basically pushed out to be the public outputs to be checked outside of Triton. (There was another one which left the yet commented out error code, but it has its own thread here.)
The path is cleared: the names are added. 🏎️ |
link the issue fix reading "from right to left" mistake (#799 (comment)) resolve few review suggestions resolve <#799 (comment)> optimize out `if let` fix couple of the review suggestions a bit of docs which were suggested in the review add `impl ConsensusProgramSpecification` replace `swap` with `pick` and a smarter approach first version of `ConsensusProgramSpecification` expansion with the generalized trait the better version of the trait expansion debugging debugging debugging debugging debugging the happy/positive test passes well negative tests move `prove_transfer` to `api` develop the comments in Tasm code verification fixes regarding the review make `new` a method process review suggestions - #799 (comment) - #799 (comment) - #799 (comment) Update neptune-core/src/application/util_proof/sent/mod.rs Co-authored-by: aszepieniec <alan.szepieniec+github@gmail.com> process the suggestions - #799 (comment) - #799 (comment) process naming suggestions extract the module from huge <server.rs> the missing changes from the dirty/development files fmt docs improvement add example / process <https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2644656254> ditch multiline `use` [rename `ConsensusProgram` to `TritonProgram`](https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2684962708) groom the module with Tasm fix the test setup `api::wallet` has its own locking approach, so the proving preparation moved back to the RPC can be moved again somewhere - but I have no idea for a right place yet develop the tests groom new modules deal with the warnings separate reserves into the other stage lint fix: Rebase fallout
link the issue
fix reading "from right to left" mistake (#799 (comment))
resolve few review suggestions
resolve <#799 (comment)>
optimize out `if let`
fix couple of the review suggestions
a bit of docs which were suggested in the review
add `impl ConsensusProgramSpecification`
replace `swap` with `pick` and a smarter approach
first version of `ConsensusProgramSpecification` expansion with the generalized trait
the better version of the trait expansion
debugging
debugging
debugging
debugging
debugging
the happy/positive test passes well
negative tests
move `prove_transfer` to `api`
develop the comments in Tasm code
verification
fixes regarding the review
make `new` a method
process review suggestions
- #799 (comment)
- #799 (comment)
- #799 (comment)
Update neptune-core/src/application/util_proof/sent/mod.rs
Co-authored-by: aszepieniec <alan.szepieniec+github@gmail.com>
process the suggestions
- #799 (comment)
- #799 (comment)
process naming suggestions
extract the module from huge <server.rs>
the missing changes from the dirty/development files
fmt
docs improvement
add example / process <https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2644656254>
ditch multiline `use`
groom the module with Tasm
fix the test setup
`api::wallet` has its own locking approach, so the proving preparation moved back to the RPC
can be moved again somewhere - but I have no idea for a right place yet
develop the tests
groom new modules
deal with the warnings
separate reserves into the other stage
lint
Co-authored-by: skaunov <skaunov@disroot.org>
aszepieniec
left a comment
aszepieniec
left a comment
There was a problem hiding this comment.
- Please mind the git commit history. Often times it's easier for reviewers to analyze individual updates to the code base in isolation before looking at the whole thing at once. Also, standalone and well-motivated commits are easier to rebase. We use conventional commits format: https://www.conventionalcommits.org/en/v1.0.0/.
- Please scope the PR and remove features that are out-of-scope. That includes:
- Spelling and grammar fixes. (Yes they bug me also, but the point is that these fixes make reviewing harder than necessary. Feel free to make a separate PR with a definite scope for them.)
- Moved code, including test.
- Inconsistent naming: in some cases the new functionality is called
transferbut then it lives in directorysent. Please synchronize. In my opinion, "proof-of-payment" is the most descriptive and clear. - Except for integration tests, please put tests into the same module as the thing being tested. I believe this is standard rust practice and it is certainly the way we do it in the rest of the codebase. In fact, I think all of module
sentshould be one file because it all contains closely related logic. - We are going to migrate from the tarpc RPC server to the JSON-RPC server. So whenever you add an RPC endpoint to the tarpc RPC server, be sure to add a symmetrical one to the JSON-RPC server.
- I think that
util_proofs/does not belong inapplication/. Theapplication/directory is meant for components like the event loops, their coordination, and the mechanisms that keep the system’s asynchronous operations behaving cohesively. Everything here participates directly in the main flow of the application. By contrast,util_proofs/contains standalone opt-in tools. The closest match isutil_types/which, though not opt-in, does have the same standalone quality and abstraction from application control flow. Please find a way to merge or integrate these latter two directories. - Please limit comments to 80 characters.
- Where is the input format defined? I'm inferring it from
source,program, andclaim_inputs, and I'm not getting a single consistent answer. (Is the release date first or not?) - Please name tests after the thing that's being tested, with a unique and descriptive name, so that we can run that one test by running
cargo test <unique-and-descriptive-test-name>. - Please add a helper function that produces the proof for you. Then invoke that helper function in the test and in the RPC endpoint.
- How do I verify a proof-of-payment? What CLI command do I invoke? What machinery is activated behind the scenes? Unless I'm mistaken, this functionality is missing; please add.
- If in the course of a dispute process I am duly convinced that I did receive the UTXO but it is not showing up in my wallet for some reason, how do I make the wallet aware of the UTXO's existence? This should be supported since otherwise there is a potential for limbo, in which the proof is valid but the UTXO that was transferred cannot be spent.
- The fields in the claim, what's the criterion for assigning them to input versus output? Outputting the amount rather than asserting its correctness makes sense because it allows you skip one assert. But for the other fields, I don't see an immediate benefit.
- Missing tests: negative tests -- wrong {lock_script_hash, receiver_digest, hash-of-sender_randomness, aocl-mmr-hash, release-date, amount}. Full circle test: produce a proof and verify it. Integration test: Alice pays Bob. Bob claims non-payment. Alice produces proof. Bob verifies and claims transferred UTXO.
- If I read correctly, nothing is happening with the release date. Meaning that
[utxo_amount']will hold the amount of coins in the UTXO regardless of whether the time lock is present and, if present, regardless of whether it was expired. I see two ways forwards. One: disallow time-locks entirely. If you want to pay someone with potential for proof-of-payment, you had better not use time-locks in your payment. Two: pass a timestamp as extra input (part of the claim) and verify that the computed amount of coins really is spendable now and not just at some future date.
⏳ git commit historyI'm fascinated each time I see how you develop it with small steps. My path is much more messy and touches a lot of stuff (as I learn the system you invented as I go) so it sadly ends in bulky items. =( I compensate by writing the docs, comments, and a lengthy commit message with snippets when allowed, as an average developer is exhausted after a dozen of commit messages hopping -- having one guide to it has its merit. Also rebasings demotivate too as it's hard to guess how it will get squashed till merging. =( I should go over this before merging. #postponedBeforeMerging ⏳ scope the PRWill do before merging. #postponedBeforeMerging (I'm not chasing those, they just impede me too much irritating. I'm filtering it now: couple dozen of the files were never staged as I'm good having it locally.) ⏳ Inconsistent namingWe discussed this privately in Tg. #postponedBeforeMerging ✅ put tests into the same moduleDone. X) Meanwhile let me note that 1) there's a lot of repos which keep it separate, 2) to be consistent the shared tests module should be moved to a higher level file too I guess. To me this is up to a module size; after some number of items separation brings better Ctrl+F experience and cleaner outline in such viewer(s). ⏳ add an RPC endpointThis is definitely #postponedBeforeMerging. Core changes faster then this receives progress. =( ✅
|
link the issue fix reading "from right to left" mistake (Neptune-Crypto#799 (comment)) resolve few review suggestions resolve <Neptune-Crypto#799 (comment)> optimize out `if let` fix couple of the review suggestions a bit of docs which were suggested in the review add `impl ConsensusProgramSpecification` replace `swap` with `pick` and a smarter approach first version of `ConsensusProgramSpecification` expansion with the generalized trait the better version of the trait expansion debugging debugging debugging debugging debugging the happy/positive test passes well negative tests move `prove_transfer` to `api` develop the comments in Tasm code verification fixes regarding the review make `new` a method process review suggestions - Neptune-Crypto#799 (comment) - Neptune-Crypto#799 (comment) - Neptune-Crypto#799 (comment) Update neptune-core/src/application/util_proof/sent/mod.rs Co-authored-by: aszepieniec <alan.szepieniec+github@gmail.com> process the suggestions - Neptune-Crypto#799 (comment) - Neptune-Crypto#799 (comment) process naming suggestions extract the module from huge <server.rs> the missing changes from the dirty/development files fmt docs improvement add example / process <https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2644656254> ditch multiline `use` [rename `ConsensusProgram` to `TritonProgram`](https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2684962708) groom the module with Tasm fix the test setup `api::wallet` has its own locking approach, so the proving preparation moved back to the RPC can be moved again somewhere - but I have no idea for a right place yet develop the tests groom new modules deal with the warnings separate reserves into the other stage lint
|
Just today I understood you probably want me just to slap it onto the added module itself. I guess #861 mentality had confused me. ~~@aszepieniec , thank you for the review! I'd approach this picking/focusing the most/top important items until we're there. To me it's the following one -- pls highlight another one if you see it differently. ~~
|
|
If my replies list to the points of the review is correct, then the question left is integration test.
|
… program in a VM queue
link the issue fix reading "from right to left" mistake (Neptune-Crypto#799 (comment)) resolve few review suggestions resolve <Neptune-Crypto#799 (comment)> optimize out `if let` fix couple of the review suggestions a bit of docs which were suggested in the review add `impl ConsensusProgramSpecification` replace `swap` with `pick` and a smarter approach first version of `ConsensusProgramSpecification` expansion with the generalized trait the better version of the trait expansion debugging debugging debugging debugging debugging the happy/positive test passes well negative tests move `prove_transfer` to `api` develop the comments in Tasm code verification fixes regarding the review make `new` a method process review suggestions - Neptune-Crypto#799 (comment) - Neptune-Crypto#799 (comment) - Neptune-Crypto#799 (comment) Update neptune-core/src/application/util_proof/sent/mod.rs Co-authored-by: aszepieniec <alan.szepieniec+github@gmail.com> process the suggestions - Neptune-Crypto#799 (comment) - Neptune-Crypto#799 (comment) process naming suggestions extract the module from huge <server.rs> the missing changes from the dirty/development files fmt docs improvement add example / process <https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2644656254> ditch multiline `use` [rename `ConsensusProgram` to `TritonProgram`](https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2684962708) groom the module with Tasm fix the test setup `api::wallet` has its own locking approach, so the proving preparation moved back to the RPC can be moved again somewhere - but I have no idea for a right place yet develop the tests groom new modules deal with the warnings separate reserves into the other stage lint
docs correction lints fix intended logging level fixup: manual squash step add `helper` and the tests for it as per <Neptune-Crypto#799 (review)>
commit 934dfaf319e41ad2894488d6b83deba80cddd50c
Author: skaunov <skaunov@disroot.org>
Date: Fri Feb 20 21:39:22 2026 +0300
fmt
docs correction
lints
fix intended logging level
fixup: manual squash step
add `helper` and the tests for it as per <https://github.com/Neptune-Crypto/neptune-core/pull/799#pullrequestreview-3993074459>
commit 2dde83065e720d6979f00492cb985a4943db6bfe
Author: skaunov <skaunov@disroot.org>
Date: Mon Dec 15 00:39:14 2025 +0300
outline the RPC API
link the issue
fix reading "from right to left" mistake (https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2644806284)
resolve few review suggestions
resolve <https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2644778929>
optimize out `if let`
fix couple of the review suggestions
a bit of docs which were suggested in the review
add `impl ConsensusProgramSpecification`
replace `swap` with `pick` and a smarter approach
first version of `ConsensusProgramSpecification` expansion with the generalized trait
the better version of the trait expansion
debugging
debugging
debugging
debugging
debugging
the happy/positive test passes well
negative tests
move `prove_transfer` to `api`
develop the comments in Tasm code
verification
fixes regarding the review
make `new` a method
process review suggestions
- https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2685085571
- https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2685066585
- https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2685137060
Update neptune-core/src/application/util_proof/sent/mod.rs
Co-authored-by: aszepieniec <alan.szepieniec+github@gmail.com>
process the suggestions
- https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2787593534
- https://github.com/Neptune-Crypto/neptune-core/pull/799#discussion_r2787588585
process naming suggestions
extract the module from huge <server.rs>
the missing changes from the dirty/development files
fmt
docs improvement
add example / process <https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2644656254>
ditch multiline `use`
[rename `ConsensusProgram` to `TritonProgram`](https://github.com/Neptune-Crypto/neptune-core/pull/799/changes#r2684962708)
groom the module with Tasm
fix the test setup
`api::wallet` has its own locking approach, so the proving preparation moved back to the RPC
can be moved again somewhere - but I have no idea for a right place yet
develop the tests
groom new modules
deal with the warnings
separate reserves into the other stage
lint
commit bf82b8c73c7d7cf3212e1e92c074eea7636db7b7
Author: skaunov <skaunov@disroot.org>
Date: Sat Dec 13 00:46:49 2025 +0300
actually that fix sits better in the beginning of the program
commit 0cd3873fc968353972b343a22f9564dff006d559
Author: skaunov <skaunov@disroot.org>
Date: Sat Dec 13 00:41:32 2025 +0300
fix the lack of `lock_script_digest` in the Tasm code
commit bf8d333bf173d0f05a19c24d10cf61602dbe36f3
Author: skaunov <skaunov@disroot.org>
Date: Fri Dec 12 19:49:29 2025 +0300
clean the comment from what was discussed in #505
commit 73e0fb029cbd2091f110633da62b3af98783d574
Author: skaunov <skaunov@disroot.org>
Date: Fri Dec 12 14:20:58 2025 +0300
fix the missing `hash` on `randomness_sender`
commit 1a02bfe3566c1d35b5afde71f4ba869bd6ca9a87
Author: skaunov <skaunov@disroot.org>
Date: Fri Dec 12 01:29:25 2025 +0300
minor readability
commit 0f6cffb6b4942e84f7e42f3239e8d1135db94bae
Author: skaunov <skaunov@disroot.org>
Date: Fri Dec 12 01:18:02 2025 +0300
add `SecretWitness` and `ConsensusProgram` implementations to use the program in a VM queue
commit 7bacded0e5863df63d30be1c1d8184015b1f3706
Author: skaunov <skaunov@disroot.org>
Date: Wed Dec 10 14:09:11 2025 +0300
proof of a transfer
commit 49839170ecbd2c4a62fda1272ed15c9b067fcd7c
Author: skaunov <skaunov@disroot.org>
Date: Wed Dec 3 20:03:51 2025 +0300
irrelevant leftover removed
commit 884b309954fa1267dce2e99dd90afaba23391e29
Author: skaunov <skaunov@disroot.org>
Date: Wed Dec 3 18:40:14 2025 +0300
`fmt`
commit 90a6d5354768938c954002c6f416570530685ebe
Author: skaunov <skaunov@disroot.org>
Date: Wed Dec 3 18:13:14 2025 +0300
initial approach
commit ab130187fa47ed99bbf705a18ef550f7f6b717e4
Author: skaunov <skaunov@disroot.org>
Date: Sat Mar 28 18:45:20 2026 +0300
feat: add sent tx getter for a wallet
commit 293fe3f40554b0fea61f61fb24537a9df23c1d5a
Author: Alan Szepieniec <alan@neptune.cash>
Date: Wed Mar 25 11:53:28 2026 +0100
ci: Update release workflow files
Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
commit e92f4a7d4c547757429294cb0f34fa70dfac0980
Author: Alan Szepieniec <alan@neptune.cash>
Date: Wed Mar 25 11:28:26 2026 +0100
chore: Release v0.8.0
commit 1bf82a23bec50c18b0126b246fd6e7020517f587
Author: sword-smith <thor@neptune.cash>
Date: Wed Mar 25 10:56:06 2026 +0100
docs: Bump RPC API version
Due to the breaking changes to the RPC API in
299e9841b042ac9998e36d2b47c22be020c94915, the RPC version number is
bumped.
This commit also adds documentation for this constant, about when it
should be bumped.
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
commit 66ce82c78cebdbec5aa1916822ec568571e7523a
Author: aszepieniec <alan@neptune.cash>
Date: Tue Mar 24 17:41:14 2026 +0100
style: Rename "consensus program" to "triton program"
Merge PR #889.
The rename is made necessary by the advent of programs that do not affect
blockchain state updates, such as proofs of payment and proofs of reserves.
These are not consensus programs because nodes may opt out of validating them
without sacrificing their capability to validate new blocks independently.
However, much of the machinery for consensus programs applies to this kind of
program as well.
- rename trait `ConsensusProgram` -> `TritonProgram`
- drop `#[doc(hidden]`: expose trait
- rename trait `ConsensusProgramSpecification` -> `TritonProgramSpecification`
- mark `pub` instead of `pub(crate)`
- rename `ConsensusError` -> `TritonError`
- rename `prove_consensus_program` -> `prove_triton_program`
Co-authored-by: skaunov <skaunov@disroot.org>
commit c111ee77a8dae2480af1e56ee55ba3ae5c0cce85
Author: aszepieniec <alan@neptune.cash>
Date: Tue Mar 24 17:40:10 2026 +0100
Merge PR #892: Sync Loop Fixes
- Identify edge case bug.
- Trigger edge case bug reliably through new test.
- Fix edge case bug and verify that tests pass.
- Adapt CI command for parallelism-unfriendly tests.
- Bias sync block height sampler towards lower-bound, benefiting sync experience. (Addresses #884.)
This commit is a squash of several other commits, whose commit messages are replicated here:
* test(sync-loop): Add test triggering resume edge case
An edge case behavior in the resumption logic for the rapid block download was
noticed and is reliably exposed by the new test,
`can_resume_block_download_from_saved_overcomplete_state`.
The edge cases is when successor blocks to the sync anchor are already present
in the sync directory that the sync is being resumed from. This edge case can
be triggered in the wild, although it is very unlikely. Specifically: if you
abort a sync relative to A's tip; and then restart syncing relative to B's tip,
and for whatever reason, B's tip is less than A's tip.
* fix(sync-loop): Fix edge case bug related to resuming saved sync state
Fixes edge case bug identified in previous commit. Solution: expand bitmask if
necessary. Tests pass.
* feat(sync-loop): Bias sync block height sampler towards lower bound
Previously, the sync loop would sample block heights uniformly from the set of
missing block heights. This commit uses a different distribution from uniform,
one that is biased towards the lower bound. As a result, you are more like to
sample and download immediate successors to your current tip, which you can
apply immediately.
The net effect of this change is that syncing nodes are less likely to lose the
hard work of downloading blocks if they power down before completing the sync,
because downloaded blocks are more likely to be applied to the state update
function. So expect the user experience to improve.
That said, there is a tradeoff and a sweet spot. Specifically, this change also
increases the probability of downloading the same block twice from different
peers. The design space in which this tradeoff lives, and the dimensions for
quantifying success are rather poorly understood and defined presently. The
urgency of the problem does not warrant a systematic solution built out of
rigorous research. So instead we opt for a slow and lazy approach based on
product feedback. So the sampler may be changed in response to constructive
complaints.
Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
commit 7fc853af7df1d8fe90abd9edec9ea5f514b318b0
Author: 21cypher <twentyfirstcypherpunk@proton.me>
Date: Tue Mar 24 22:54:34 2026 +0700
merge PR #857: Fix incorrectly rendered table in peer screen
Fix logic error leading to layout bug.
Fixes #883.
commit fc318527399ac645d6d87466342b295f4e3af7c9
Author: Sergey Kaunov <skaunov@disroot.org>
Date: Tue Mar 24 18:46:02 2026 +0300
Merge PR #690: Add granularity to `MainToMinerChannel::send`
Pattern match on the return type of `try_send` to log a message tailored to the severity of the failure (if necessary):
- `Closed` --> miner loop does not exist any more; big problem.
- `Full` --> miner loops is busy with a backlog of messages; probably okay but worth logging a warning.
commit 874b350ece5d69dcb833b507937c31b921d99c65
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 14:15:14 2026 +0100
devops: Cleaner tokio dependency
Instead of using the "full" feature, explicitly list the features we
rely on.
Also require minimum version of latest LTS, 1.47.x.
Doesn't change the version referenced in Cargo.lock, so for normal,
independent builds, nothing changes. If used as a library, behavior
might change.
commit 61e621f8e0db7ed3ce3a55db70e53c4d6f862282
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 14:14:20 2026 +0100
test: Avoid sampling target block interval of 0
commit 422fa4451c8de964d1d5a2aad7de06afb17f4b3b
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 13:21:34 2026 +0100
chore: Remove unused dependencies
Since the "derive" feature is activated for both the `serde` and the
`strum` dependencies, we can get rid of the `serde_derive` and
`strum_macros` dependencies.
commit b2f527aa5547773dfe8270dc95844271b6e9a2be
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 12:51:06 2026 +0100
style: Avoid collect_vec failing in IDE
For some reason my IDE didn't like this `collect_vec`. Alan's IDE also
had this problem. We both use VS Code + rust-analyzer. I'm on Rust
nightly, Alan is on stable.
commit 32af4585f2a549fda6d5adc74c02873a87094596
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 12:40:17 2026 +0100
Revert "ci: Cleanup build cache on closed PRs"
This reverts commit b42ce25b33d5018044e08c35d95025fb481c4592.
Doesn't work.
commit f61f61e251aed63bb0f327d46f593066d9cddf21
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 12:35:40 2026 +0100
chore: cargo update
commit b42ce25b33d5018044e08c35d95025fb481c4592
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 12:22:56 2026 +0100
ci: Cleanup build cache on closed PRs
commit fb2863125b753d7a4501af66b6683256e1e7c80f
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 12:22:34 2026 +0100
ci: Share build cache in `main` job
commit 57a3b77ff1bd72c96f8571b53b88bd6710b91f92
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 23 16:24:32 2026 +0100
ci: Skip installation of clang in build without Cargo.lock
`clang` is already present in `ubuntu-latest`. No reason to update or
install it again.
commit 431b9254e85aa03fd1e8949dc9e5e29d8f6f4eb5
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 23 16:16:55 2026 +0100
ci: More faster builds
Caching, skip linking (`cargo check` instead of `cargo build`), updated
checkouts, nextest over `cargo test`.
Also: Just check benches, don't compile them The build was there to
ensure that benchmarks still build. There's no reason to actually
compile very perfomant binaries since benchmarks not actually being
run.
commit 73a519ed94db13fb783301501217d23b6285ed4f
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 23 15:50:50 2026 +0100
ci: Faster CI failures & runs
Use caching for faster CI builds. And only run certain CI steps
(formating and linting) on Ubuntu.
commit 69a616037748e671aec442807f4f707ae8a6180d
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 24 11:43:39 2026 +0100
test: Verify *all* proof servers work
Previously, this test only verified that at least *one* of the proof
servers worked. Now it fails if just one of them are failing.
This test is skipped in CI since it would allow CI to fail through a DOS
of the proof servers. That seems annoying and unintentional. Local
testing of this property will have to do.
Also: Change domain name to raw IP for one of the proof servers, because
the DNS entry was changed for its previous domain.
commit 6f2b1ddabf5ec8843434fedc317c5c4cc2826511
Author: sword-smith <thor@neptune.cash>
Date: Fri Mar 20 13:40:38 2026 +0100
refactor(json_rpc): Increase max body size beyond 2MB
Some requests, like the submitting of a block and that of a
transaction, are much larger than the default max body size of 2MB. For
that reason, we increase the max request size for submission of
transactions to 120MB, which is more than enough for very big proof
collection-backed transactions. To set a method-specific upper limit for
requests, the interface of `rpc_handler` was changed. I don't think it
negatively affects performance though since only partial serialization
is done before the size limit is checked.
commit 6b999dca365018df08d8f10b3d4d43f1ea698040
Author: sword-smith <thor@neptune.cash>
Date: Fri Mar 20 13:27:50 2026 +0100
feat(json_rpc): Better error messages in HTTP client
Don't just return JsonError::InternalError if something goes wrong, show
where in the TCP/HTTP stack things go wrong.
commit eb477547e9940424220028a01a21ee4398425f87
Author: sword-smith <thor@neptune.cash>
Date: Wed Mar 18 14:02:59 2026 +0100
refactor: Make things `pub` for downstream VXB wallet
Make muliple functions and types `pub` such that the downstream VXB
wallet can rely on an unmodified `neptune-core` as a dependency.
commit 914cf0c722ba7f44ddc9a5c0cb85b72dbd165317
Author: sword-smith <thor@neptune.cash>
Date: Wed Mar 18 13:30:19 2026 +0100
perf(json_rpc): wallet::get_blocks
Respond faster to the JSON RPC endpoint wallet_getBlocks which returns a
list of blocks on the form `RpcWalletBlock`. This block format contains
the full block kernel, but instead of the proof, it contains only the
hash of the proof.
This allows for the calculation of the block's hash without having to
transfer the ~1MB block proof.
The speedup is achieved by adding the new function
`ArchivalState::get_block_kernel_with_proof_digest` which returns the
proof of the hash without having to recalculate it, since the hash of
the proof was already calculated once, when the block was stored in the
archival state.
Should save ~50ms on fast machines for each block that is returned,
since that what I've observed it takes to calculate a block proof's
hash. But take that number with a grain of salt as I haven't performed
precise benchmarks.
commit bd62b6ce69ac2804f6fa8f4c76eb44da79cefb78
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 17 11:50:44 2026 +0100
chore(json_rpc): Make some data structures `Copy`
commit 299e9841b042ac9998e36d2b47c22be020c94915
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 17 11:48:17 2026 +0100
fix(json_rpc): make Rpc block hash agree with block hash
Change data structure to represent the transcation inputs to one that
allows for repeated chunk index values.
Prior to this commit a BTreeMap was used with the chunk indices as key.
This didn't allow for a chunk dictionary with repeated chunk indices
which is allowed when the chunk dictionary is compressed.
This closes #886.
commit 4a6bcfcf4d0dc7a0a688cbd12a437dc04316300f
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 17 12:42:54 2026 +0100
perf(set_new_tip): Don't load parent block unless necessary
If the new block being applied in `set_new_tip` is the direct descendant
of the current tip, the parent mutator set accumulator can be read from
the light state. Previously the parent block was always loaded from
disk. Now that only happens when new block is now direct descendant of
previous tip.
Really a no-brainer from a performance perspective: Reading from disk is
**slow**!
But in case there are some doubters out there, I also present proof from
my log:
Prior to this commit:
2026-03-17T11:37:06.653828632Z DEBUG ThreadId(01) neptune_cash: executed set_new_tip_internal() in 0.197089092 secs. location: neptune-core/src/state/mod.rs:2795:9
With changes from this commit:
2026-03-17T11:39:35.360406891Z DEBUG ThreadId(01) neptune_cash: executed set_new_tip_internal() in 0.063810427 secs. location: neptune-core/src/state/mod.rs:2795:9
So a 3x speedup in this canonical state updater function, in the happy
case. Nice!
More comprehensive log:
Before this commit:
2026-03-17T11:37:06.226871983Z INFO ThreadId(01) neptune_cash::state: Updated state with block of height 1346.
2026-03-17T11:37:06.283486751Z DEBUG ThreadId(01) neptune_cash::protocol::consensus::block::validity::block_program: ** Calling triton_vm::verify to verify block proof ...
2026-03-17T11:37:06.283589299Z DEBUG ThreadId(01) neptune_cash::protocol::consensus::block::validity::block_program: ** Call to triton_vm::verify to verify block proof completed; verdict: true.
2026-03-17T11:37:06.456743004Z DEBUG ThreadId(01) neptune_cash::state: Applying block to archival state.
2026-03-17T11:37:06.469013998Z DEBUG ThreadId(01) neptune_cash::state::archival_state: Writing block to: /home/thor/.local/share/neptune/main/blocks/blk10.dat
2026-03-17T11:37:06.469246715Z DEBUG ThreadId(01) neptune_cash::state::archival_state: Size of file prior to block writing: 79944008
2026-03-17T11:37:06.469372598Z DEBUG ThreadId(01) neptune_cash::state::archival_state: New file size: 81023678 bytes
2026-03-17T11:37:06.538732722Z DEBUG ThreadId(01) neptune_cash::state::archival_state: Updating mutator set: adding block with height 1347. Mined: 2025-08-14T06:51:49.968+02:00
2026-03-17T11:37:06.571604856Z DEBUG ThreadId(01) neptune_cash::state::archival_state: sanity check: was AMS updated consistently with new block?
2026-03-17T11:37:06.582744423Z DEBUG ThreadId(01) neptune_cash::state: Applying block to mempool.
2026-03-17T11:37:06.639880673Z DEBUG ThreadId(01) neptune_cash::state: Applying block to wallet.
2026-03-17T11:37:06.653683779Z DEBUG ThreadId(01) neptune_cash::state::wallet::wallet_state: Scanned block for incoming UTXOs; received 0 onchain notifications, 0 onchain through scan mode, 0 offchain notifications
2026-03-17T11:37:06.653805398Z DEBUG ThreadId(01) neptune_cash::state: Applying block mempool events.
2026-03-17T11:37:06.6538171Z DEBUG ThreadId(01) neptune_cash::state: Done setting new tip.
2026-03-17T11:37:06.653828632Z DEBUG ThreadId(01) neptune_cash: executed set_new_tip_internal() in 0.197089092 secs. location: neptune-core/src/state/mod.rs:2795:9
2026-03-17T11:37:06.653886894Z INFO ThreadId(01) neptune_cash::state: Updated state with block of height 1347.
After this commit:
2026-03-17T11:39:35.190230766Z INFO ThreadId(01) neptune_cash::state: Updated state with block of height 1363.
2026-03-17T11:39:35.240836007Z DEBUG ThreadId(01) neptune_cash::protocol::consensus::block::validity::block_program: ** Calling triton_vm::verify to verify block proof ...
2026-03-17T11:39:35.240873844Z DEBUG ThreadId(01) neptune_cash::protocol::consensus::block::validity::block_program: ** Call to triton_vm::verify to verify block proof completed; verdict: true.
2026-03-17T11:39:35.296596904Z DEBUG ThreadId(01) neptune_cash::state: Applying block to archival state.
2026-03-17T11:39:35.301267083Z DEBUG ThreadId(01) neptune_cash::state::archival_state: Writing block to: /home/thor/.local/share/neptune/main/blocks/blk10.dat
2026-03-17T11:39:35.301330816Z DEBUG ThreadId(01) neptune_cash::state::archival_state: Size of file prior to block writing: 97903370
2026-03-17T11:39:35.3014002Z DEBUG ThreadId(01) neptune_cash::state::archival_state: New file size: 98955700 bytes
2026-03-17T11:39:35.353387387Z DEBUG ThreadId(01) neptune_cash::state::archival_state: Updating mutator set: adding block with height 1364. Mined: 2025-08-14T09:10:34.126+02:00
2026-03-17T11:39:35.353810059Z DEBUG ThreadId(01) neptune_cash::state::archival_state: sanity check: was AMS updated consistently with new block?
2026-03-17T11:39:35.354986938Z DEBUG ThreadId(01) neptune_cash::state: Applying block to mempool.
2026-03-17T11:39:35.355312463Z DEBUG ThreadId(01) neptune_cash::state: Applying block to wallet.
2026-03-17T11:39:35.360358682Z DEBUG ThreadId(01) neptune_cash::state::wallet::wallet_state: Scanned block for incoming UTXOs; received 0 onchain notifications, 0 onchain through scan mode, 0 offchain notifications
2026-03-17T11:39:35.360394412Z DEBUG ThreadId(01) neptune_cash::state: Applying block mempool events.
2026-03-17T11:39:35.360400642Z DEBUG ThreadId(01) neptune_cash::state: Done setting new tip.
2026-03-17T11:39:35.360406891Z DEBUG ThreadId(01) neptune_cash: executed set_new_tip_internal() in 0.063810427 secs. location: neptune-core/src/state/mod.rs:2795:9
2026-03-17T11:39:35.360414089Z INFO ThreadId(01) neptune_cash::state: Updated state with block of height 1364.
commit cf6d34041a315916dfcec65c977a0cd3cc3add81
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 23 13:20:40 2026 +0100
refactor(LightState): Store mutator set accumulator after
Instead of recalculating the mutator set accumulator every time we need
it, a field for it is added to `LightState`. This is a tiny
performance optimization and reduces boilerplate code since the
`expect`/`unwrap` of the derivation of the mutator set after now happens
when a new block is stored as tip in the `LightState`.
Additionally:
- adds methods to `BlockChain` state to get frequently used valus: tip
height, tip hash, and tip mutator set; and use these methods
throughout the codebase
- Change interface of `Block::compose` to take an owned block since it
blocned the block in its callgraph anyway.
- Add documentation to `LightState` fields and methods.
commit 7489f4feb1e3532f8c05679cd88dfa1968b63d52
Author: 21cypher <twentyfirstcypherpunk@proton.me>
Date: Mon Mar 23 17:43:04 2026 +0700
Add tip mining time information to dashboard (#885)
* save vertical space: redundant empty lines at the end of the 'blockchain' block
* save vertical space #2: put 'Archive' and 'Mempool' block side-by-side
* added time-to-mine information to block on accepting it as tip. passing this value in overview-data RPC call, to display it in dashboard
* tweaked comments + added tests
* remove block_interval field in overview data which as far as I can see is never being initialized
* failed to debug; removing debug prints
* LightState now contains auxiliary data to the block
* impl Clone
* renamed field 'received_at' to 'accepted_at' as it better reflects the value
* added tests, removed accepted_at field since nobody is using it
* removing unnecessary clones and todos
* using set_new_tip on a test
* restored original order of two lines
* added tip_mut helper for tests so that they can change the block
* remove redundant clones
* preventing a lock acquisition
* simplifying LightState to not be an Arc + InnerStruct
* blockchainState now holds Box<LightState>
commit d923aedfdd9a107baab39b8511c8d13ffc262d10
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 23 09:14:09 2026 +0100
chore: Make 1.96 linter happy
Make nightly version of linter happy.
commit 93ffd4e1c41b492daf09ffd1f4e61280f61618f4
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 23 09:03:16 2026 +0100
devops: Disable `collapsible-match` clippy rule
The reason I'm disabling this is that its new enforcement in 1.96
(nightly) disturbs what I consider to be a nice predictable pattern of
"match" expressions and statements. I'm used to seeing this:
match enum {
variant0 => ...
variant1 => ...
...
}
So while collapsing an `if` into the branch selection rule does reduce
the number of decision points for codeblocks, it also disturbs what I
consider to be a canonical pattern of `match`. So collapsing the `if`
into the branch decision logic increases the mental overhead of parsing
this code, rather than reducing it.
commit 928b4939277d4d0db604ed02a6c5f9b168ddb63f
Author: 21cypher <twentyfirstcypherpunk@proton.me>
Date: Fri Mar 13 12:46:07 2026 +0200
ci: Validate changelog
Add CI test that will prevent incorrect changelogs in the future
commit 34fb2b31f341b0ed28189e24910231ddb023a5eb
Author: xvchris <56232580+xvchris@users.noreply.github.com>
Date: Fri Mar 13 17:21:05 2026 +0800
feat(sync): Add `--sync-dir` to override sync block storage location
Add new CLI argument: `--sync-dir` for specifying where to store unprocessed blocks downloaded during sync. By default the node asks the OS for a temporary directory and stores them there. Passing this argument overrides that default.
Closes #880
- Add CLI argument `--sync-dir`.
- Pass `sync_dir` argument to SyncLoopHandle::new in mock.
- Improve variable naming for clarity in `RapidBlockDownload`: rename temp_dir_base() to base_storage_dir() to avoid confusion, and rename temp_directory variable to storage_dir for consistency
- Add docstring explaining that the function `base_storage_dir()` returns either user-specified directory or OS temp directory, not necessarily temporary.
Co-authored-by: xvchris <xvchris@github.com>
commit 700e88daa0662bf9a29fb21740fc4fbcb9ac7ebf
Merge: f006fb90 23a3b509
Author: aszepieniec <alan@neptune.cash>
Date: Fri Mar 13 10:09:27 2026 +0100
ci: Fix red CI in PRs from forks
Original motivation:
The "Release" CI workflow fails for all PRs from forks because it doesn't have access to the ACTIONS_ID_TOKEN_REQUEST_URL env variable. There are currently 3 or so PRs that are failing the CI for this reason.
This disables the workflow from running on Pull Requests. It will allow better quick inspection of PRs, because all Red PRs will be red for a good reason, rather than a false-negative Red, which they have no fault in.
The only ones for which the Release workflow will succeed are those allowed to create PRs directly here, which I assume is the core team. If they still want to create the artifacts on the PR, as a compromise, they could trigger this workflow manually (or push a tag, which also triggers this workflow).
I like to see GREEN
---
AI-assisted summary:
The proposed change to `release.yml` replaces the automatic `pull_request` trigger with a manual `workflow_dispatch` trigger. This successfully eliminates "false-negative" CI failures on pull requests from forks, which occur because external contributors lack the security permissions required for build attestations. While this clears up "Red" PR noise and saves CI minutes, it removes the automated "dry run" that verifies the release process still works before code is merged. The primary release mechanism—pushing a versioned Git tag—remains intact and automated. Ultimately, manual execution via the new trigger will require maintainers to select a specific Git tag or branch, which may affect how the underlying cargo-dist logic identifies the release version.
commit 23a3b5093c839c0952ac3c24996821903090c14c
Author: 21cypher <twentyfirstcypherpunk@proton.me>
Date: Thu Mar 12 16:42:45 2026 +0200
don't run the 'Release' workflow in PRs, as it fails for forks regardless, because it requires secrets
commit f006fb90e8ce8f6af274f84921958f82829f1a08
Author: cypher21 <twentyfirstcypherpunk@proton.me>
Date: Tue Mar 10 20:56:58 2026 +0200
fix(CHANGELOG.md): Delete double-entry for v0.6.1
Also: move merge line to the version where it belongs (v0.6.1 not v0.6.0).
commit c817affdedc336536fd6556e1adbefea4b2af84b
Author: Alan Szepieniec <alan@neptune.cash>
Date: Tue Mar 10 17:25:57 2026 +0100
fix(wallet): Sleep before making backup on migration
LevelDB periodically prunes files that are not necessary (to save space). The
operating system likes to wait before deleting them (to save work). Backing up
the directory in a hurry is bound to stumble across no-longer-necessary-but-not-
deleted-yet files. Sleeping gives LevelDB / the OS a chance to complete the file
deletions.
Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
commit 632c2f5fc37611279c0e90cad732c069f93a868a
Author: Alan Szepieniec <alan@neptune.cash>
Date: Tue Mar 10 17:07:32 2026 +0100
log: Make error messages noisier
For recursive directory copying.
Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
commit e604f04b145e861d016818b4075f25db8425f0df
Author: Alan Szepieniec <alan@neptune.cash>
Date: Tue Mar 10 16:54:19 2026 +0100
log: Add info to logs on error event
Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
commit bd433a0ed6461524e96f9100475497d001cb8db0
Author: Alan Szepieniec <alan@neptune.cash>
Date: Tue Mar 10 15:33:09 2026 +0100
chore: Releave v0.7.0
commit 56b64762e52c519541b2de52f1180328b85833d7
Author: Alan Szepieniec <alan@neptune.cash>
Date: Tue Mar 10 15:31:12 2026 +0100
chore(cliff): Support pull request merge commit messages
commit 3b1207e1634661cff4af963a1fa23ab5ebddc70d
Merge: f02c3cfa c7ed82b5
Author: aszepieniec <alan@neptune.cash>
Date: Tue Mar 10 14:04:50 2026 +0100
Merge PR #876: Exchange Integration Endpoints
Closes #871.
See #871 for a more elaborate description.
This PR adds or modifies the following endpoints:
- `wallet::ValidateAddress`
- `wallet::ValidateAmount`
- `personal::GenerateAddress`
- `personal::GetBalance`
- `personal::SendTx`
- `personal::UnspentUtxos`
- `personal::IncomingHistory`
- `personal::CountSentTransactionsAtBlock`
- `personal::ClaimUtxo`
- `personal::OutgoingHistory`
- `utxo_index::WasMined`
Additionally, this PR:
- introduces RPC namespace "Personal" for endpoints related to your *own* wallet, as opposed to generic wallet endpoints;
- refactors the transaction initiation pipeline.
commit f02c3cfae025da9591917ec2c51f7659c36915df
Author: aszepieniec <alan@neptune.cash>
Date: Tue Mar 10 13:53:22 2026 +0100
feat: Allow out-of-process prover to relay job (#877)
This PR enables the out-of-process prover job to act as a proxy and relay the proof production job to an external command or executable. It does this by applying the following refactors:
Rename binary triton-vm-prover to neptune-prover, adhering to a consistent prefix rule for all binaries in this crate.
Add and use struct NeptuneProverJob to codify the input format for this process.
(In neptune-prover:) inspect environment variable NEPTUNE_PROVER_PROXY and if set, interpret the value of this variable as a command or binary and run that with identically the same input, and return identically the same output when it is done.
(In neptune-core:) since standard input and standard output channels are occupied for the task description and proof payloads, use stderr for logging. Intercept stderr messages with the prefix "INFO:" / "DEBUG:" etc and pass them to the equivalent tracing counterpart.
---
(Selection of) squashed commits:
* style: Rename binaries and env vars for consistent namespace prefix
Binaries renamed:
- `triton-vm-prover` -> `neptune-prover`
- `block-claims` -> `neptune-block-claims`
* refactor(neptune-prover): Codify input format
Add a new struct, `NeptuneProofJob`, which contains exactly the
information to be transmitted to the out-of-process prover. This
refactor displaces the custom encoders and decoders, which would
be difficult to maintain across changes to the output format.
Also:
- Add `neptune-prover` as an explicit binary in `Cargo.toml`.
This addition is necessary to inform Cargo to provide us with
this binary when running integration tests.
- Add an integration test for out-of-process proving.
- Remove inlined `#[cfg(not(test))]` decorations. The code decorated
thusly now is being tested.
- Remove inlined #[cfg(test)] decorations; move relevant code to
test module instead.
* refactor(neptune-prover): Proxy proof job
If environment variable NEPTUNE_PROVER_PROXY is set, neptune-prover will
relay its input to the given command or executable; and relay the
output back.
* log(neptune-prover): Pipe out-of-process stderr to tracing logs
This way, you can write in `neptune-prover.rs` or its delegatee
```
eprintln!("ERROR: failed to generate proof because silly.");
```
and it will generate an ERROR log message.
* ci: Update release workflow files following binary name change
Co-authored-by: sword-smith <thor@neptune.cash>
commit c7ed82b5d98a892fe5bbf662bbd234cb63408866
Author: Alan Szepieniec <alan@neptune.cash>
Date: Tue Mar 10 11:37:11 2026 +0100
docs: Clarify function and purpose of field `unrestricted`
commit bbce0a46d4c7767ce1d697e3b8121f7a80c7e735
Author: sword-smith <thor@neptune.cash>
Date: Fri Mar 6 11:30:29 2026 +0100
fix(sync_loop): Increase sync channel capacity to 100
The individual message types used in this channel do not exceed 1 block
in size, so they will not be more than the consensus size-limit for
blocks which is 8MB. So this change allows for 800MB to be used by the
sync loop. For archival nodes, I consider that acceptable.
This fixes the problem of a very slow test
`can_sync_with_moving_target_from_dynamic_set_of_flaky_syncing_peers`
which is slow because the channel capacity to the main loop is
exhausted. It should also improve production-code behavior as the sync
loop's channel to send to main loop will be less likely to be exhausted.
commit a765af9a8ec084c25012b811c446fca16d89acb1
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 9 15:32:58 2026 +0100
style: Simplify construction of MempoolTransactionInfo
Avoid creating then mutating, just create.
commit 4eb08648b6a5e10742b5668e397d9017b78c099b
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 9 12:05:44 2026 +0100
fix: Enumeration of TxProvingCapability
The lowest possible value is PrimitiveWitness, not LockScript (which is
still unimplemented). This change fixes this discrepancy.
commit 15e8b576754cde50dd0af1121c2e21db29f385a5
Author: sword-smith <thor@neptune.cash>
Date: Fri Mar 6 16:32:44 2026 +0100
refactor(json_rpc): Disallow 'Personal' namespace unless unsafe RPC is chosen
The `unsafe_rpc` CLI parameter was introduced to prevent 3rd parties from
making destructive calls to the client -- if not set, then the RPC
should be safe to open for 3rd parties. Its use is now extended to also
cover access to the node's wallet. If the unsafe_rpc value is not set,
then the user is prevented from activating the `Personal` namespace. It
should be obvious to the reader that access to `Personal::send` is worse
than a DOS attack. So if `unsafe_rpc` guards against DOS, it should
certainly also guard against theft.
commit 5338c3580f2ed0b5eb65856c2e84032952c6a2b1
Author: sword-smith <thor@neptune.cash>
Date: Fri Mar 6 15:41:27 2026 +0100
feat(json_rpc): Add `personal::ClaimUtxo`
Add an JSON RPC endpoint for registering UTXOs that come with off-chain
notifications.
commit 85a3bbd8b75169af0c321f4efa5b1acee0e7840d
Author: sword-smith <thor@neptune.cash>
Date: Fri Mar 6 11:57:07 2026 +0100
fix(tx_initiator): Avoid race condition during tx initialization
Addresses #874 (but does not close it) by fixing the race conditions
during transaction initialization. Race conditions during the
`consolidate` transaction initialization remain.
The key to this fix is to grab a lock on the global state *once* and
then only letting it go once a consistent transaction has been
constructed. This implies that methods in `TransactionInitiator` must
not call other methods on the same struct since each method is
responsible for taking its own locks.
This race condition was introduced in
bcc7153f40a0ba9291c3c426da5886c95393489b and made worse in
08190b07d2ea8d639daf904b73a18e3c6ed4e45f since an assert was introduced
that a transaction input, a UTXO, was not used as input in a mempool
transaction, and this assert occurred *after* the release of a read-lock
where the transaction inputs were collected.
Introduces two ways of constructing transactions through the
`TransactionInitiator`, one that may mutate state and one that may not.
commit 1ff93bf2ea25e9413030dc4b25dfd939e819e7ac
Author: sword-smith <thor@neptune.cash>
Date: Thu Mar 5 17:30:01 2026 +0100
refactor(json_rpc): Drop extra balance check
The transaction initiator already checks that the balance is sufficient
when contructing a transaction. No reason to perform that check again in
the RPC server.
commit 92c7747e12face30d090009b7328a99906a6566a
Author: sword-smith <thor@neptune.cash>
Date: Thu Mar 5 17:26:20 2026 +0100
refactor(json_rpc): Avoid mutating state in `send`
Do not bump the key derivation index during execution of the `send`
endpoint. Instead, bump the derivation index in main loop. This
honors the rule that the RPC server may not change any global state.
commit 07585a8c9a21d6ecaa3ec4a5484ae61b2c5532e9
Author: sword-smith <thor@neptune.cash>
Date: Wed Mar 4 18:43:40 2026 +0100
feat(json_rpc): send
Add an endpoint to the JSON/RPC server for initiating a transaction from
the node's own wallet.
This commit also allows for the injection of a custom wallet seed
through the `crate::inintialize` "master" function that starts a new
node. This injected custom wallet seed allows for the construction of
tests that have a non-negative balance at the genesis block. This allows
for easy tests of the `send` endpoint.
commit 936b63e19215d1631226f40b16e05ac0c6bbf26d
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 3 19:05:29 2026 +0100
feat(json_rpc): list unspent UTXOs
Add endpiont for listing basic information about unspent UTXOs.
Adds some fields to the `CoinWithPossibleTimeLock` data structure to
support this new endpoint.
No significant slowdown observed in the
wallet_state::coins_with_possible_timelocks_1000_4 benchmark:
This commit
$cargo criterion --bench wallet_state
wallet_state fastest │ slowest │ median │ mean │ samples │ iters
╰─ wallet_state │ │ │ │ │
├─ coins_with_possible_timelocks_1000_4 1.099 ms │ 2.252 ms │ 1.118 ms │ 1.265 ms │ 10 │ 10
Prior to this commit
$cargo criterion --bench wallet_state
wallet_state fastest │ slowest │ median │ mean │ samples │ iters
╰─ wallet_state │ │ │ │ │
├─ coins_with_possible_timelocks_1000_4 1.087 ms │ 1.915 ms │ 1.096 ms │ 1.194 ms │ 10 │ 10
Cf. #871.
commit 24261d641d5bedfcc29ec7dc1ede77b0bb53614d
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 3 18:08:50 2026 +0100
feat(json_rpc): incoming UTXO history
Add endpoint `personal::IncomingHistory`
Cf. #871.
commit b652e4fda2e0ae023531d76bc2be1fbaa8dd559e
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 3 14:17:18 2026 +0100
feat(json_rpc): Validate NAU amounts in string format
Similar to the validator of coin amounts but accepts the number of NAU
as string, instead of the number of coins. Gives end-users the
flexibility to chose their own unit of account.
Cf. #871.
commit 64890634dc2ab7bde2c31ab504517642ff7230ab
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 3 13:51:34 2026 +0100
feat(json_rpc): Verify string as valid amount
Verify that a string represents a valid NPT/native currency amount
number of coins. Rejects strings that cannot be parsed as numbers as
well as negative numbers.
Cf. #871.
commit fe99ccb2a34f8cc3b8ec88a911549c95d8a1125f
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 3 12:53:28 2026 +0100
feat(json_rpc): Add endpoint for validating an address
Takes a string and returns meta information about the address, if the
string represents an address format known to neptune-core.
Cf. #871.
Co-authored-by: huuhait <huuhadz2k@gmail.com>
Co-authored-by: Tuan Kiet (Lucky) <chtkkiet@gmail.com>
commit 7bc058e9c46cb2bd6ffe7bbf01a36a42608d7333
Author: sword-smith <thor@neptune.cash>
Date: Tue Mar 3 12:23:23 2026 +0100
feat(json_rpc): add endpoint for counting txs intiated at block
Implement `personal::CountSentTransactionsAtBlock` JSON RPC endpoint.
Used to count number of transactions initiated at e.g. current tip. Can
be used to check if a new transaction should be queue rather than
broadcasted on the network.
Cf. #871.
Co-authored-by: huuhait <huuhadz2k@gmail.com>
Co-authored-by: Tuan Kiet (Lucky) <chtkkiet@gmail.com>
commit abfc23409707effc9d6289900acd17a273390e41
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 2 22:40:35 2026 +0100
test(wallet_state): balance with n confirmations
Add a test that asserts that the tip-balance n blocks back matches the
tip balance with n confirmations.
Cf. https://github.com/Neptune-Crypto/neptune-core/pull/873#issuecomment-3984942212
Also add a very simple test that transactions in the mempool affect the
unconfirmed balance as expected.
commit ba4ecbee8a88ce6dad2f55257daf91104d786faf
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 2 18:52:21 2026 +0100
style: Make clippy happy
Drop doctest that is unmaintainable after refactor without IDE help.
Ain't nobody got time for that. Use one of the actual examples in
production code.
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
commit 6231a4573b5f647171fb130fe2324de67f99730e
Author: sword-smith <thor@neptune.cash>
Date: Mon Mar 2 18:31:31 2026 +0100
fix(wallet_state): Only recover MUTXOs if key is known
When recovering monitored UTXOs from `incoming_randomness.dat`, only
recover the UTXO to the wallet if a key is known for this UTXO. If a key
is not known, print an error message to inform the user that there might
be a discrepancy between `incoming_randomness.dat` and their wallet
seed, or that they might need to bump their derivation indices.
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
commit 08190b07d2ea8d639daf904b73a18e3c6ed4e45f
Author: Alan Szepieniec <alan@neptune.cash>
Date: Wed Feb 25 19:10:01 2026 +0100
refactor!(wallet): Add RPC support, explicit unlocking step
1. Refactor `WalletStatus`
1.a) Remove membership proofs from WalletStatus.
Motivation: performance. Accelerates assembling and doing operations
with the `WalletStatus`. Note that the archival node does not need the
membership proofs to live on `WalletStatus`; they are fetched from the
archival mutator set just in time for initiating transactions. The
light node does not have the archival mutator set and does need
membership proofs to initiate transactions but can afford to get them
from the monitored UTXOs database because, after all, the
`StrongUtxoKey`, or the data from which it can be derived, is
available.
1.b) Disable getting balances from `WalletState` some of the time and
from `WalletStatus` other times.
Motivation: confusing and error-prone distribution of responsibility.
The new one-way cascade `WalletState` -> `WalletStatus` -> balances
concentrates responsibility and logic into `WalletStatus`.
1.c) Expand `WalletStatus` with Mempool UTXOs.
Motivation: without knowledge of mempool UTXOs, the `WalletStatus`
cannot answer questions about unconfirmed balances -- which is why
some of the time the user had to get that info from `WalletState`.
1.d) Allow the user to specify block height to classify
balance-updates as confirmed versus unconfirmed.
Motivation: the RPC endpoint specification states that the user must
be able to specify a number of confirmations relative to which the
balance is calculated. In practice, the RPC handler fetches the block
height, subtracts the number-of-confirmations, and uses that threshold
parameter.
2. Add RPC support for getting balance with n confirmations.
Motivation: specified by RPC endpoints for exchange integration.
Exchanges have an incentive to a) avoid crediting users before n
confirmations; and b) to avoid repeating the bookkeeping done when
initating a withdrawal transaction or when
consolidating-to-cold-wallet UTXOs that may be reorganized away.
3. Rename `mempool_spent_utxos` and `mempool_unspent_utxos` (both
fields and functions living on `WalletState`) into
`mempool_outgoing_utxos` and `mempool_incoming_utxos`.
Motivation: the term "spent UTXOs" is ambiguous but certainly one of
its two senses succeeds to denote own UTXOs that are about to be spent
by transactions that now live in the mempool but are about to be
confirmed by a block. However, the term "unspent UTXOs" is the
opposite but on the wrong dimension. The UTXOs this term is supposed
to describe *are* unspent but *all mempool UTXOs are unspent* (and
"spent UTXOs" too). The thing denoted by this variable and function is
UTXOs that the wallet recognizes as its own, and that are generated by
transactions that now live in the mempool but are about to be
confirmed. The correct dimension for oppositional naming is not spent
versus unspent, but incoming versus outgoing.
4. Change type of `mempool_unspent_utxos_iter()` (now
`mempool_incoming_utxos_iter()`) from `(Utxo, AdditionRecord)` to
`(Utxo, Digest, Digest)` where the two digests are the sender
randomness and the receiver preimage, respectively.
Motivation: one of the responsibilities of the `WalletStatus` is to
hold UTXOs for sampling inputs for initiating payments. In a future
world where you can spend incoming mempool UTXOs (such as in a
chaining transaction), you will certainly need to demonstrate
knowledge of the preimage of the addition record. So the (item,
sender randomness, receiver postimage) should be considered
information needed to spend the UTXO. The jury is still out on whether
the receiver postimage suffices or whether the receiver preimage is
needed, so the choice for the latter option errs on the safe side.
Note that the addition record can be computed from the available data
with very little hashing.
5. Rename `InputSelectionPolicy` to `InputSelectionPriority`.
Motivation: "policy" is more general and will be used to include other
things, such as which inputs to filter out. The enum in question is
really only about how to sort inputs that are already established to
be admissible candidates.
6. Add `InputSelectionPriority` variant `ByAge`.
Motivation: sometimes, like when consolidating UTXOs, you want to
spend the oldest UTXOs first. The comment (now dropped) suggested it
was difficult to do based on block height. And that might be true, but
you do have access to the AOCL leaf index, which is used instead.
7. Add and use struct `InputSelectionPolicy` over
`InputSelectionPriority`.
Motivation: "policy" is a strict superset of "priority". Right now it
also contains a threshold number of confirmations below which to
disqualify inputs.
8. Separate input selection from unlocking.
Previously, the list of all spendable inputs were obtained, sent to
the transaction builder API, which proceeded to make a selection of
inputs and build a transaction based on that selection. The where and
how of unlocking those inputs was a little muddy and under-performant.
This refactor changes the data types and data flow to the following.
`WalletState` | User calls `get_wallet_status` to generate a wallet
| | status.
v
`WalletStatus` | `WalletStatus` contains a list of all UTXOs the
| | wallet knows about, both synced and unsynced fetched
| | from the database and unconfirmed fetched from the
| | mempool.
| | The user calls `spendable_inputs` to get a list of
| | candidate inputs for building a transaction. These
| | candidate inputs are not unlocked.
v
`InputCandidate` | (Renamed from `TxInput`.) The difference between a
| | `InputCandidate` and a raw UTXO known to be under
| | management by the wallet is metadata for enforcing
| | a transaction input selection policy.
v
`InputSelector` | (Renamed from `TxInputListBuilder`.) The
| | `InputSelector` makes a selection of the given
| | inputs suitable for the intended transaction.
v
`GlobalState` | The selected inputs need to be unlocked: extended
| | membership proofs and witness data that enable the
| | production of the proofs in the proof collection
| | relative to these inputs. The reason why this
| | process involves `GlobalState` is because the
| | `GlobalState` knows whether it is archival or light.
| | If it is a light node, then the membership proofs
| | are fetched from the wallet. If it is an archival
| | node, then the membership proofs are fetched from
| | the archival mutator set. In either case, the
| | remaining witness data are fetched from the wallet.
v
`UnlockedUtxo` | An `UnlockedUtxo` is ready for the proof collection
| | production step. But as a convenience layer it is
| | wrapped in a ...
v
`TxInputs` | (Renamed from `TxInputList`.) ... vector new type.
| The `TxInputs` is what lives on the
| `TransactionDetails` object.
This sequence cleanly separates *input selection* from the *input
unlocking*. All inputs that reach the selection stage are assumed to
be unlockable. Unlocking is relatively expensive, so you want to do
it on as few inputs as possible. By separating this step out and
putting it after input selection, we shrink the workload.
A note on filtering out inputs based on mempool expenditures: the
`WalletStatus` method `spendable_inputs` filters out synced inputs
based on whether their AOCL leaf indices are contained in a hash set
of AOCL leaf indices generated from mempool inputs. Previously this
filtering used the absolute index sets. However, the view of known
inputs to mempool transactions is already generated using absolute
index sets (so matching based on absolute index sets constitutes
repetition of work) and their AOCL leaf indices are available at that
point. This change just keeps those AOCL leaf indices handy until
they are used by the `WalletStatus`.
Also, rename:
- `TxInput` to `InputCandidate`;
- `TxInputList` to `TxInputs`;
- `TxInputListBuilder` to `InputSelector`.
9. Enforce required number of confirmations.
Motivation: required by RPC endpoint specification. You want to be
able to avoid redoing bookkeeping (in tems of allocating inputs to
transactions) when a reorganization happens and you need to
re-initiate.
Struct `InputCandidate` is extended with the meta-data, the number of
confirmations specifically, necessary to perform this filter. The
`InputSelector` now possesses the required information and applies the
filter.
Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
commit 02b466b28fa566d8183aeb32375c6d2d810addcf
Author: sword-smith <thor@neptune.cash>
Date: Fri Feb 27 14:22:05 2026 +0100
docs: Move JSON/RPC docmentation to endpoint
Cf. `neptune-core/src/application/json_rpc/README.md`. The endpoint
defined in the interface is the documentation that the RPC `client`
object sees. So it belongs as a docstring to the endpoint.
commit 9cfcef917f92dedcdd8fdda58891116e28289932
Author: sword-smith <thor@neptune.cash>
Date: Fri Feb 27 13:35:13 2026 +0100
feat(json_rpc): add endpoint `personal::OutgoingHistory`
This endpoint is used to fetch all of the sent transactions from the
database, with the ability to both filter and paginate the returned
outgoing transactions.
Cf. #871.
Co-authored-by: Tuan Kiet (Lucky) <chtkkiet@gmail.com>
Co-authored-by: huuhait <huuhadz2k@gmail.com>
commit 853028df31cf2de89a831a79c7b74e47060d33b5
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 25 17:48:52 2026 +0100
refactor(json_rpc): 'Personal' and 'Wallet' namespaces
Personal refers to endpoints pertaining to a node's own wallet. Wallet
refers to endpoints where the node can act as a server for other
wallets.
Co-authored-by: KaffinPX <KaffinPX@gmail.com>
commit 5b770015be0aed407969c574ca768a901d675d56
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 25 14:33:03 2026 +0100
feat(json_rpc): Add `Utxoindex::WasMined` endpoint
Answer the question if a block with a specific set of inputs/ouputs was
ever mined.
Ticks off another box in #871.
commit d7aa1861660a415dcfd6e43b5526e3f40afdbde3
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 25 14:25:32 2026 +0100
feat(UtxoIndex): Filter canonical blocks on in/outputs
Add new method to archival state that requires the presence of a UTXO
index: `canonical_block_heights_with_puts`.
Can be used to answer the question if a block with a specific set of
inputs/outputs was ever mined. Useful to match a node's sent transaction
with a block -- to answer if a specific initiated transaction was ever
mined.
To be used by the `Utxoindex::WasMined` endpoint cf. #871.
commit d49a79ea9a6af18f02920d5498355c729903ccb2
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 25 17:02:31 2026 +0100
ci: Disable wallet-db migration test on Windows
This test fails on Windows, like the other real migration tests, due to
LevelDB platform issues.
commit 0a8d536fa24e3d7c85dab7ff898cff415bc8cee5
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 25 14:29:22 2026 +0100
docs(json_rpc): Document where documentation goes
Such meta.
commit 0853e66a2aa9bf1d8e91990ea89570dabfdc4c53
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 25 11:49:49 2026 +0100
style: Don't camelcase namespaces
We want the `method` in the POST request to follow the form
namespace_endPointName. And if we use camelcase in the namespace, the
method field will contain two underscores. Without camelcase, the
underscore cleanly separates the namespace from the endpoint's name.
commit cd2bbab3535e8cf81bb7343ec3c13c65d84ee66d
Author: sword-smith <thor@neptune.cash>
Date: Tue Feb 24 18:44:39 2026 +0100
feat(json_rpc): Add endpoint to generate new address
Cf. #871.
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
commit 475f1e23ef2a92a8116ec2e7b9ce82afa2448ff4
Author: sword-smith <thor@neptune.cash>
Date: Tue Feb 24 18:33:39 2026 +0100
refactor(json_rpc): demarcate wallet endpoints with new namespaces
The `Wallet` namespace was actually two very distinct categories of
endpoints: one for acting as a server for other light clients/wallet
software and one for controlling the node's own wallet. To distinguish
these two classes of endpoints we demarcate the namespace into
`Walletserver` and `Ownwallet`.
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
commit 5123228a17da148c282273fd4d1a5fc752ddbb50
Author: sword-smith <thor@neptune.cash>
Date: Mon Feb 23 21:53:15 2026 +0100
test: Allow run even if node is running
A particular test, `client_responds_in_real_world_scenario`, didn't work
if an instance of the node was running -- which was quite annoying. This
commit allows the running of this test on machines where the node is
running.
commit 5d001f8b51fb0d810d63ce540360e728330dbf70
Author: sword-smith <thor@neptune.cash>
Date: Mon Feb 23 21:17:17 2026 +0100
test: Make proptest deterministic
Functions called by proptest test are not allowed to use true
randomness, only deterministic randomness.
Cf. #869.
Co-authored-by: skaunov <skaunov@disroot.org>
commit e6a9ea5776d56c880fbcfc84eeb267198277c0e9
Author: sword-smith <thor@neptune.cash>
Date: Mon Feb 23 16:18:26 2026 +0100
refactor(wallet): Rewrite archival/mutxo spend check logic
changelog: ignore
commit 1b1f2e62305891384c7c11d53002dabfeb8de30a
Author: sword-smith <thor@neptune.cash>
Date: Mon Feb 23 16:07:23 2026 +0100
log: Report when wallet-rescan is completed
Rescans might take a long time if no UTXO index is maintained. It's nice
for the user to get a log message when it is done.
commit aacc1659ade5ab34e5cf6d2fd971a3facae39088
Author: sword-smith <thor@neptune.cash>
Date: Thu Feb 19 17:49:46 2026 +0100
ci: Run wallet_state benchmark in CI/codspeed
Shamelessly copied from the https://github.com/TritonVM/triton-vm
repository.
commit a3aae7f88a49d94173c36e51ad872eca54bf3d3f
Author: sword-smith <thor@neptune.cash>
Date: Mon Feb 23 15:46:10 2026 +0100
docs: Clarify that rescan for incoming UTXOs must be followed by expenditure rescan
commit f5d5e131bfbe051ab504c5d2afad3c71f7bc6220
Author: sword-smith <thor@neptune.cash>
Date: Mon Feb 23 14:42:54 2026 +0100
test: Verify rescan of incoming also registers expenditures
Because of changed assumptions introduced in #859, we must ensure that
all monitored UTXOs are always scanned for expenditures after a new UTXO
is identified (e.g. after a rescan). This test verifies that this
expenditure rescan happens after a rescan for incoming UTXOs.
Moves some test-helper functions around.
commit 68e45119a51673eaf758c607793fb57f9b5ee68d
Author: sword-smith <thor@neptune.cash>
Date: Fri Feb 20 17:28:46 2026 +0100
refactor: wallet-db v3
- Change interface for rescanning for outgoing UTXOs -- allways scan all
monitored UTXOs, not only a specified range.
- Change `spent` field on monitored UTXOs to account for spends in
unknown block.
- Always scan all UTXOs for expenditures after scanning for incoming
UTXOs.
The key insight is that with the archival mutator set, it's very fast to
check if the absolute index set has been applied. Then the "spent
status" of a UTXO can be easily checked during a rescan. But only with
the UTXO index can you quickly find out in which block the UTXO was spent.
To account for this partial knowledge of a UTXO, we add an enum to
enumerate the spent status, with
`MonitoredUtxoSpentStatus::SpentInUnknownBlock` representing the new
possibility.
Even if a UTXO index is not present on the node, we can still check if
a particular monitored UTXO was spent or not, by probing the archival
mutator set. But in that case, it's impractical to find the block in
which the UTXO was spent. So we need a new option for the `spent` status
of a monitored UTXO: Spent in unknown block. This change justifies a
wallet database migration and when we're at it we might as well delete
all memembership proofs from the monitored UTXOs since we now, because
speed, generate them on the fly, cf. #859.
With the closing of #859 a subtle problem snuck in that this commit
fixes: #859 assumed that archival nodes have processed all blocks, so if
a UTXO has never been observed to be spent, it must be unspent. But this
assumption does not hold when a wallet has been restored through the
`rescan` endpoints. To account for this, we ensure that all monitored
UTXOs are always rescanned for expenditures whenever new UTXOs are added
through the rescanning endpoints. This way, the wallet should never end
up erroneously registering UTXOs as unspent.
Wallet database migration to v3 deletes all membership proofs from the
since archival nodes no longer need to maintain or know them.
commit 755c24afd4fc1a3407257f7bd2ebb9f8a4d73c6c
Author: Alan Szepieniec <alan@neptune.cash>
Date: Mon Feb 23 14:07:11 2026 +0100
feat: Sync from all peers who complete sync challenge
Previously, the only way in which a node can start syncing from more
than one peer is if that peer is discovered (through peer discovery)
after sync mode has started. This commit enables another pathway into
that configuration: if the peer connections are established immediately,
more than one challenge is issued, and more than one response comes
back. Previously, the first successful sync challenge response would
cause subsequent responses to be ignored. Now the sync loop is notified
of the late responder, and will proceed to sync from them as well.
commit 3bb5f9440dc4936e6fed1930f2bd3435d543b4f0
Author: Alan Szepieniec <alan@neptune.cash>
Date: Thu Feb 19 17:53:14 2026 +0100
fix(CLI): Sort UTXOs with released time-lock into `available` column
Fixes #864.
commit 37f344a2ac0a8c139cebab39405e123071c079ad
Merge: 21155a59 56b2efa8
Author: Thorkil Værge <thor@neptune.cash>
Date: Thu Feb 19 17:03:28 2026 +0100
Merge pull request #860 from Neptune-Crypto/dont-rebuild-nops
Dont rebuild nops
commit 56b2efa8ee73c4ae4cfb362dcccf193fce801fe3
Author: sword-smith <thor@neptune.cash>
Date: Wed Feb 18 01:11:57 2026 +0100
perf(mine_loop): Re-use nop and updated transaction across proposals
If a composer builds multiple block proposals for the same block height,
they might as well reuse the nop and updated transactions. This is done
by inserting those transactions into the mempool.
commit 21155a59dd6292fb6e8ea100aa1fe916168856c6
Merge: e6d4df76 0c8f6ff3
Author: Thorkil Værge <thor@neptune.cash>
Date: Thu Feb 19 16:54:56 2026 +0100
Merge pull request #859 from Neptune-Crypto/faster-wallet-with-archive
Faster wallet with archive
Speed up wallet operations by using archival state, when it's available.
```
`master`
Timer precision: 10 ns
wallet_state fastest │ slowest │ median │ mean │ samples │ iters
╰─ wallet_state │ │ │ │ │
├─ set_new_tip_1000_4 1.651 s │ 2.976 s │ 1.678 s │ 1.813 s │ 10 │ 10
├─ spendable_inputs_1000_4 2.354 s │…
4 participants
a transfer
current list of review points is tracked & processed in the following comment
reserves
here's a good draft of the Triton code
the discussion is below, will be get back to it after the transfer part