Skip to content

feat(api-compare): report tested API versions in conformance report#7283

Open
0xDevNinja wants to merge 1 commit into
ChainSafe:mainfrom
0xDevNinja:0xdevninja/issue-6509-conformance-api-versions
Open

feat(api-compare): report tested API versions in conformance report#7283
0xDevNinja wants to merge 1 commit into
ChainSafe:mainfrom
0xDevNinja:0xdevninja/issue-6509-conformance-api-versions

Conversation

@0xDevNinja

Copy link
Copy Markdown
Contributor

Summary of changes

The conformance/api compare report gave no indication of which API version (v0/v1/v2) a method was tested under. This adds that information.

Changes introduced in this pull request:

  • Track the API path of every executed test per method in ReportBuilder.
  • Expose it as a tested_api_versions field in the JSON report (omitted when empty, so existing consumers stay compatible).
  • Add an "API Versions" column to the printed summary table.

I went with both the JSON field and the summary column to cover the two options mentioned in the issue.

Reference issue to close (if applicable)

Closes #6509

Other information and links

The version set is a BTreeSet<ApiPaths> so the output is deterministic (v0, v1, v2) regardless of test execution order. Added a unit test for the formatting.

I didn't add a CHANGELOG entry since forest-tool api compare is a developer/CI conformance tool rather than a node-operator-facing feature, but happy to add one if you'd prefer.

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

  • I have read and agree to the CONTRIBUTING document.
  • I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

The conformance report previously gave no indication of which API
version each method was exercised under. Track the `v0`/`v1`/`v2` path of
every executed test per method, expose it as a `tested_api_versions`
field in the JSON report, and add an "API Versions" column to the
printed summary table.

Closes ChainSafe#6509
@0xDevNinja 0xDevNinja requested a review from a team as a code owner July 2, 2026 11:04
@0xDevNinja 0xDevNinja requested review from akaladarshi and sudo-shashank and removed request for a team July 2, 2026 11:04
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da540af7-d100-4328-8713-2e00a7230c47

📥 Commits

Reviewing files that changed from the base of the PR and between ef3a30b and b376ced.

📒 Files selected for processing (2)
  • src/tool/subcommands/api_cmd/api_compare_tests.rs
  • src/tool/subcommands/api_cmd/report.rs
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • filecoin-project/lotus (manual)
👮 Files not reviewed due to content moderation or server errors (2)
  • src/tool/subcommands/api_cmd/report.rs
  • src/tool/subcommands/api_cmd/api_compare_tests.rs

Warning

Walkthrough skipped

File diffs could not be summarized.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
✨ Simplify code
  • Create PR with simplified code

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.16667% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.11%. Comparing base (ef3a30b) to head (b376ced).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/tool/subcommands/api_cmd/report.rs 79.16% 5 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/tool/subcommands/api_cmd/report.rs 24.21% <79.16%> (+5.76%) ⬆️

... and 5 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef3a30b...b376ced. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Monthly conformance reports should report API version

1 participant