Add CLI Tools for Identifying Incoming UTXOs#783
Closed
aszepieniec wants to merge 5 commits intomasterfrom
Closed
Conversation
Add a CLI command to neptune-cli for obtaining the derivation index from a given address. Usage: > neptune-cli index-of <full-address> Example output: ``` key type: Generation index: 97 ```
Add a CLI command, along with an RPC function and application support, for re-scanning a particular block (identified by `BlockSelector`) for UTXOs bound a given address. The wallet database will be appraised of any new UTXOs found, and `incoming_randomness.dat` will get new entries for them. Usage: > neptune-cli rescan --block 1337 --address <address> Also!: deprecate `GenerationReceivingAddress::to_bech32m_abbreviated`. Reason: we already have `ReceivingAddress::to_display_bech32m_abbreviated`, which works for both `GenerationReceivingAddress` and `SymmetricKey` and it disagrees with this method.
Collaborator
Perhaps address should be optional, and if no address is specified then the block is scanned for all known wallet keys. (Though that could be further constrained by an optional --key-types argument that defaults to all.) |
Sword-Smith
requested changes
Nov 24, 2025
Member
Sword-Smith
left a comment
Sword-Smith
left a comment
There was a problem hiding this comment.
Looks OK to me. Let's see if we can't add a test or two though. Maybe one where block_n has already been processed, and then scan_for_utxos_announced_to_given_key is called, and you verify that nothing is returned? Such that you ensure that this recovery does not create duplicates.
Contributor
Author
|
@Sword-Smith: which parts, if any,of this PR are obviated by the new branch for faster wallet updates? If all, then let's close the PR without merging. |
Contributor
Author
|
@Sword-Smith: my understanding is that the code you wrote for rescanning requires a populated UTXO index whereas this branch does not. Is that accurate? If yes, then I would propose to refactor this PR such that this code is used as a fallback when the node has no access to a populated UTXO index. |
aszepieniec
added a commit
that referenced
this pull request
Feb 16, 2026
Adds two new CLI commands:
1. `neptune-cli index-of <address>` computes the derivation address from the
given address.
2. `neptune-cli rescan --block 1337 --address <address>` rescans block 1337 for
incoming UTXOs.
This commit is a squash of four commits, following a rebase on top of master
(currently, 957e2e2) of branch
`asz/better-incoming-utxo-tracking` which was the object of PR #783 [1].
[1]: #783
---
feat(CLI): `index-of`: Get the derivation index of the given address
Add a CLI command to neptune-cli for obtaining the derivation index from a given
address. Usage:
> neptune-cli index-of <full-address>
Example output:
```
key type: Generation
index: 97
```
---
feat(CLI): Allow abbreviated addresses in `index-of`
---
feat(CLI/RPC): Add command for re-scanning a particular block
Add a CLI command, along with an RPC function and application support, for
re-scanning a particular block (identified by `BlockSelector`) for UTXOs bound
a given address. The wallet database will be appraised of any new UTXOs found,
and `incoming_randomness.dat` will get new entries for them.
Usage:
> neptune-cli rescan --block 1337 --address <address>
Also!: deprecate `GenerationReceivingAddress::to_bech32m_abbreviated`. Reason:
we already have `ReceivingAddress::to_display_bech32m_abbreviated`, which works
for both `GenerationReceivingAddress` and `SymmetricKey` and it disagrees with
this method.
---
docs: Avoid HTML tags in CLI command docs
aszepieniec
added a commit
that referenced
this pull request
Feb 17, 2026
Adds two new CLI commands:
1. `neptune-cli index-of <address>` computes the derivation address from the
given address.
2. `neptune-cli rescan --block 1337 --address <address>` rescans block 1337 for
incoming UTXOs.
This commit is a squash of four commits, following a rebase on top of master
(currently, 957e2e2) of branch
`asz/better-incoming-utxo-tracking` which was the object of PR #783 [1].
[1]: #783
---
feat(CLI): `index-of`: Get the derivation index of the given address
Add a CLI command to neptune-cli for obtaining the derivation index from a given
address. Usage:
> neptune-cli index-of <full-address>
Example output:
```
key type: Generation
index: 97
```
---
feat(CLI): Allow abbreviated addresses in `index-of`
---
feat(CLI/RPC): Add command for re-scanning a particular block
Add a CLI command, along with an RPC function and application support, for
re-scanning a particular block (identified by `BlockSelector`) for UTXOs bound
a given address. The wallet database will be appraised of any new UTXOs found,
and `incoming_randomness.dat` will get new entries for them.
Usage:
> neptune-cli rescan --block 1337 --address <address>
Also!: deprecate `GenerationReceivingAddress::to_bech32m_abbreviated`. Reason:
we already have `ReceivingAddress::to_display_bech32m_abbreviated`, which works
for both `GenerationReceivingAddress` and `SymmetricKey` and it disagrees with
this method.
---
doc: Escape HTML markers
aszepieniec
added a commit
that referenced
this pull request
Feb 19, 2026
…nning * Merge PR #783: Add CLI Tools for Identifying Incoming UTXOs Adds two new CLI commands: 1. `neptune-cli index-of <address>` computes the derivation address from the given address. 2. `neptune-cli rescan --block 1337 --address <address>` rescans block 1337 for incoming UTXOs. This commit is a squash of four commits, following a rebase on top of master (currently, 957e2e2) of branch `asz/better-incoming-utxo-tracking` which was the object of PR #783 [1]. [1]: #783 --- feat(CLI): `index-of`: Get the derivation index of the given address Add a CLI command to neptune-cli for obtaining the derivation index from a given address. Usage: > neptune-cli index-of <full-address> Example output: ``` key type: Generation index: 97 ``` --- feat(CLI): Allow abbreviated addresses in `index-of` --- feat(CLI/RPC): Add command for re-scanning a particular block Add a CLI command, along with an RPC function and application support, for re-scanning a particular block (identified by `BlockSelector`) for UTXOs bound a given address. The wallet database will be appraised of any new UTXOs found, and `incoming_randomness.dat` will get new entries for them. Usage: > neptune-cli rescan --block 1337 --address <address> Also!: deprecate `GenerationReceivingAddress::to_bech32m_abbreviated`. Reason: we already have `ReceivingAddress::to_display_bech32m_abbreviated`, which works for both `GenerationReceivingAddress` and `SymmetricKey` and it disagrees with this method. --- doc: Escape HTML markers * refactor: Reuse same rescan logic Two different RPC endpoints gave very similar results and used similar code under the hood. - rescan-announced (master): calls `GlobalState::rescan_announced_incoming` asynchronously. This function uses the UTXO index to iterate quickly over the given range of blocks and scans them for UTXO notifications encryped to a set of keys, which the RPC handler sets to "all known keys" - rescan (earlier, but this PR): calls `WalletState::rescan` with a given block and set of key indices, and scans the block for UTXO notifications encrypted any of the given keys. The RPC user supplies the key derivation index. This PR modifies `WalletState::rescan`: - Rename to `rescan_block_for_announced_incoming`. - Take an iterator over keys instead of a slice of (index, key type) pairs. (The RPC handler of `rescan` is now tasked with producing the iterator over the one indicated key.) This PR also modifies `GlobalState::rescan_announced_incoming`: - Use the UTXO index, if it is available, to skip blocks if we can; otherwise proceed anyway (but slowly). - Use `WalletState::rescan_block_for_announced_incoming` internally, reusing code. Also, more deeply, in `WalletState::scan_block_for_utxos_announced_to_given_keys`: - Change type of argument `keys` from `&[SpendingKey]` to any concrete type that implements `IntoIterator<Item = &SpendingKey>>`. - Change return type from `Vec<IncomingUtxo>` to `HashMap<AdditionRecord,IncomingUtxo>`, for better alignment with downstream use. * ux!(CLI): Refactor rescan commands - Add option `address` to rescan-announced. If not set, use all known addresses. - Make last block optional. If not set, scan just the first block. - Split four rescan commands into four subcommands of `rescan`. - `neptune-cli rescan announced 13 --address nolgam1foobar` ~ ~ ~~~~~~~~~~~~~~~~~~~~~~ | | new, optional | last block ommitted: range is {13} no dash (mandatory, breaking) Note that the address is currently parsed but then ignored. A future commit will integrate this value into the RPC endpoints, thereby giving this option effect. * refactor!(RPC): Extend `rescan-announced` with key derivation path Add an optional key derivation path (of type `(KeyType, u64)` to the interface of the rescan-announced RPC endpoints (tarpc and json). Use this new field for the CLI subcommand `rescan announced` if the address is set (and if the client can determine the derivation path from the address). Breaking change because this change modifies the RPC interface. * fix(RPC): Allow rescan announced without UTXO index Remove safeguards. Method was adapted already. * refactor!(CLI): Drop redundant statistics commands The commands for statistics over block time and difficulty do not add much value besides the commands for listing the block times and difficulties. Pipe them through a python script if you must. Meanwhile, neptune-cli must face the pressure on the clap parser originating from too many commands leading to stack overflow errors while parsing. Good excuse to make sure that CLI commands are few but useful. Also: rearrange commands and clarify comments. Co-authored-by: Thorkil Schmidiger <thor@neptune.cash> * style(wallet-state): Unmix notions "derivation index" and "counter" Two related notions, "derivation index" and "counter" were used interchangeably and, indeed, erroneously. The *counter* tracks how many keys of the given type were generated by the wallet. The *derivation index* is the personalization integer used for pseudorandom key generation. The relation between these two concepts is that for new keys, the derivation index is set to the current counter value; and the counter value is bumped. This refactor renames functions and variables and clarifies comments to better separate these notions. * feat(RPC): Read and modify key derivation indices Add RPC support (both json-rpc and tarpc) for getting the current derivation index of a given key type, and for setting it. Setting comes with safeguards: you cannot roll back the index and you can only increase it by 100 at a time. Users who find this limitation problematic are assumed clever enough to solve it ;-) * feat(CLI): Read and modify key derivation indices Add CLI commands for reading and modifying the wallet's key derivation indices for a given key type. Specifically: - `neptune-cli get-derivation-index <key_type>` and - `neptune-cli set-derivation-index <key_type> <index>` * docs(CLI): Clarify rescan command Also: correct grammar error. Co-authored-by: Thorkil Schmidiger <thor@neptune.cash> * fix(wallet): Make rescan-announced work without UTXO index Also: fix: remove safeguards preventing the function from being used when no UTXO index is present. Also: clarify and correct docs. Also: simplify type signatures in favor of `HashSet<_>` in favor of `T: IntoIterator<..>` such that type system guarantees uniqueness. Co-authored-by: Thorkil Schmidiger <thor@neptune.cash> --------- Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
Contributor
Author
|
Closed by commands introduced in (commits leading up to) v0.7.0. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two CLI commands, the second one with RPC and application support:
neptune-cli index-of <address>Computes the derivation index from the given address.neptune-cli rescan --block 1337 --address <address>Re-scans block1337to look for output UTXOs that can be spent by the secret spending key associated with the given address, and if any are found, adds them to the wallet database and adds their randomness toincoming_randomness.dat.