KIP-227: Candidate and Validator Evaluation

[jiseongnoh]

Proposed changes

Propose a framework for quantitatively assessing the performance and stability of candidates and validators in the Kaia Chain network

Types of changes

Please put an x in the boxes related to your change.

• Bugfix
• KIP Proposal
• KIP Improvement

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• Used the suggested template: https://github.com/kaiachain/KIPs/blob/main/kip-template.md
• I have read the CLA and signed by comment I have read the CLA Document and I hereby sign the CLA in first time contribution
• I have added necessary documentation (if appropriate)
• Any dependent changes have been merged and published in downstream modules

Related issues

• Please leave the issue numbers or links related to this PR here.

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you proposed and what alternatives you have considered, etc.

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[jiseongnoh]

I have read the CLA Document and I hereby sign the CLA

[hyeonLewis]

@jiseongnoh I assigned 227 to this KIP according to the numbering rule in KIP-201. Thanks.

[hyunsooda]

I undertood that this KIP addresses how to measure the node's performance. How to exploit measured numbers is not the scope of this KIP and delegated for KIP-286(validator's lifecylcle). Is it correct?

[hyunsooda]

Does this timeout apply to each message type individually? Specifically, is there a 200ms limit for each stage: Pre-prepare, Prepare, and Commit?

[jiseongnoh]

Thanks for the question.
CANDIDATE_READY_TIMEOUT is evaluation-only and applies only to the VRank CandidateReady reception deadline. It is not related to IBFT consensus and does not apply to IBFT message types.

[hyunsooda]

Ah, I see. I previously assumed candidates participated in consensus without voting power, but I was mistaken. Is that message just a artifical message like ping-pong?

[jiseongnoh]

Yes, CandidateReady is synthetic in the sense that it exists purely for measurement. However, a reporter cannot fabricate a success because it requires a valid candidate signature bound to the target block.

[hyunsooda]

Understood. How are message transmission and reception processed? I couldn’t find a minimal protocol for this, so I assume message testing occurs over the full EPOCH_LENGTH (one day). Additionally, if these are just artificial 'ping-pong' messages, it would be difficult to infer a candidate's machine specifications. Verifying computational capability may not align with simple message transfers. Are there any major parts that I've missed?

[hyunsooda]

As I understood, only one of the validators measures the candidate, and the node is the next block proposer.

• Is it correct that only one validator measures the candidate's performance for each block?
• Why do the next block proposer measure, rather than the current proposer? (maybe relavant with the round change)

[jiseongnoh]

mfReport for block N is committed in block N+1 by the proposer of the next finalized block (N+1).
Reporters rotate every block, so candidates are observed by many validators over time, which makes per-reporter counts and TMFS top-F filtering meaningful against Byzantine reporters.
We commit in N+1 for consistency with pfReport and to avoid a timing race. CandidateReady has a strict deadline (200ms), and including it in block N would force delaying block production.

[hyunsooda]

Thanks for the details. I see that both are N+1 commit.

• mfReport: Wondering why it is committed N+1? Even if it's committed in N, I believe the distribution of diversity would be similar.
• pfReport: Got it. Is that specific number 200 derived from 250ms of the execution time because the response must be guaranteed to arrive before making the proposal?

[jiseongnoh]

We commit mfReport/crReport in N+1 to protect block time for N. If embedded in N, the proposer must wait up to CANDIDATE_READY_TIMEOUT before finalizing the header, which delays block time.
By committing in N+1, we keep block N production on time and treat candidate readiness as an asynchronous measurement artifact that can be recorded in the next block without delaying block time.

[jiseongnoh]

I undertood that this KIP addresses how to measure the node's performance. How to exploit measured numbers is not the scope of this KIP and delegated for KIP-286(validator's lifecylcle). Is it correct?

Yes, that is correct.
KIP-227 specifies (1) what data is reported in the block header (vrank), and (2) how performance metrics (PFS/TMFS/CMFS) are measured and computed in a verifiable way. It does not define operational policies such as promotion, demotion, validator lifecycle transitions, or penalty actions.

How these measured metrics are used for validator/candidate lifecycle decisions is out of scope for this KIP and is expected to be defined in a separate proposal.

[ian0371]

validators must validate the newly added vrank field

Validate what? For pfReport, validate that it only contains the proposers that caused RC?
How about mfReport? signature validation?

[jiseongnoh]

Let’s add explicit validation rules for vrank after FORK_BLOCK
(i) pfReport MUST be a deterministic, verifiable list of round-change proposers in round order
(ii) crReport MUST contain at most one entry per candidate and each entry MUST carry a valid CandidateReady signature bound to (block_number, proposal_hash).

[ian0371]

Suggested change

	The VRank framework evaluates node performance using three independent metrics. These metrics apply separately to validators and candidates, allowing for a more focused assessment of each role's responsibilities.
	The VRank framework evaluates node performance using three independent metrics. Each metric measures events occurred in an epoch, and resets at the epoch start block. These metrics apply separately to validators and candidates, allowing for a more focused assessment of each role's responsibilities.

[ian0371]

I think we should record all round changes of block N in the block N (not block N+1), so we can remove it from the council when generating block N+1. cc @hyeonLewis

[ian0371]

Let's define CandidateReady something like this:

message CandidateReady {
	uint64 block_number;
	byte[32] proposal_hash;
	signature sig; // sign_ecdsa(proposal_hash)
}

[ian0371]

Suggested change

	**Consensus Method**: The proposer of block `N` MUST record the proposal failure information of block `N` in the `pfReport` field of the block header at height `N`, as an ordered list of `(round number, proposer)` for each failed round. Validators MUST compare `pfReport(N)` with their local round-change record for block `N` to reach consensus.
	**Consensus Method**: `pfReport` is not recorded in a block. Instead, a validator must derive `pfReport` from its round number and previous rounds' proposers.

[ian0371]

Suggested change

	**Score (Aggregation)**: For epoch index `k`, PFS MUST be computed from headers `H ∈ [k*EPOCH_LENGTH, (k+1)*EPOCH_LENGTH - 1]` by counting each validator's proposal failures from `pfReport`.
	**Score (Aggregation)**: For epoch index `k`, PFS MUST be computed from headers `H ∈ [k*EPOCH_LENGTH, (k+1)*EPOCH_LENGTH - 1]` by counting each validator's proposal failures from round information of `header.ExtraData`.

[ian0371]

Suggested change

	### 2\. Message Transmission Failure Score (MFS)
	### 2\. Candidate Failure Score (CFS)

[ian0371]

Suggested change

	The Message Transmission Failure Score is divided into two components:
	The Candidate Failure Score is divided into two components:

[ian0371]

Suggested change

	**Definition**: Measures the number of times a candidate fails to transmit the expected `CandidateReady` message during a block proposal cycle, after removing the highest `F` of the failure counts to address measurement distortions.
	**Definition**: Measures the number of times a candidate fails to transmit the expected `VRankCandidate` message during a block proposal cycle, after removing the highest `F` of the failure counts to address measurement distortions.