ENSDb Compare tool for database exports and comparisons

[djstrong]

• Introduced a new CLI tool for comparing ENS databases using Drizzle ORM and PostgreSQL.
• Implemented features for exporting tables in various formats (CSV, JSON, binary) with compression options.
• Added functionality for benchmarking export performance and generating comparison reports.

Questions:

1. Should we compare only some tables and some columns like in graphql diff tool? Or compare everything?
2. Should the tool work on databases with different number of blocks indexed? If yes then open question is how to export or compare data to a given block number.

I have run comparison on the same database exported with some time delay:

Comparison Summary:
Database A: export2drizzle/
Database B: export3drizzle/
Tables Compared: 29
Total Differences: 49
Comparison Time: 22266ms

Table Results:
╔══════════════════════════════╤═════════╤═════════╤═══════════╤═══════════╤════════╤════════╗
║ Table                        │ Total A │ Total B │ Identical │ Different │ Only A │ Only B ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ abi_changed                  │ 125     │ 125     │ 125       │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ accounts                     │ 11180   │ 11182   │ 11180     │ 0         │ 0      │ 2      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ addr_changed                 │ 22881   │ 22884   │ 22881     │ 0         │ 0      │ 3      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ authorisation_changed        │ 0       │ 0       │ 0         │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ contenthash_changed          │ 2985    │ 2985    │ 2985      │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ domains                      │ 81256   │ 81260   │ 81252     │ 4         │ 0      │ 4      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ expiry_extended              │ 87      │ 87      │ 87        │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ ext_resolver_address_records │ 21563   │ 21566   │ 21563     │ 0         │ 0      │ 3      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ ext_resolver_text_records    │ 178147  │ 178147  │ 178147    │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ fuses_set                    │ 1006    │ 1006    │ 1006      │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ interface_changed            │ 233     │ 233     │ 233       │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ multicoin_addr_changed       │ 22928   │ 22931   │ 22928     │ 0         │ 0      │ 3      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ name_changed                 │ 20163   │ 20164   │ 20163     │ 0         │ 0      │ 1      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ name_registered              │ 68334   │ 68336   │ 68334     │ 0         │ 0      │ 2      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ name_renewed                 │ 1172    │ 1172    │ 1172      │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ name_transferred             │ 2501    │ 2502    │ 2501      │ 0         │ 0      │ 1      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ name_unwrapped               │ 824     │ 824     │ 824       │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ name_wrapped                 │ 67373   │ 67374   │ 67373     │ 0         │ 0      │ 1      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ new_owners                   │ 90200   │ 90206   │ 90200     │ 0         │ 0      │ 6      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ new_resolvers                │ 72518   │ 72524   │ 72518     │ 0         │ 0      │ 6      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ new_ttls                     │ 14      │ 14      │ 14        │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ pubkey_changed               │ 1064    │ 1065    │ 1064      │ 0         │ 0      │ 1      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ registrations                │ 68181   │ 68183   │ 68181     │ 0         │ 0      │ 2      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ resolvers                    │ 85943   │ 85949   │ 85943     │ 0         │ 0      │ 6      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ text_changed                 │ 186524  │ 186524  │ 186524    │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ transfers                    │ 2136    │ 2138    │ 2136      │ 0         │ 0      │ 2      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ version_changed              │ 62      │ 62      │ 62        │ 0         │ 0      │ 0      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ wrapped_domains              │ 66549   │ 66550   │ 66549     │ 0         │ 0      │ 1      ║
╟──────────────────────────────┼─────────┼─────────┼───────────┼───────────┼────────┼────────╢
║ wrapped_transfers            │ 70768   │ 70769   │ 70768     │ 0         │ 0      │ 1      ║
╚══════════════════════════════╧═════════╧═════════╧═══════════╧═══════════╧════════╧════════╝

Database B is the later one so it has more data and almost all is identical except 4 domains where subdomain_count has increased.
comparison_report.json
So if we want to compare databases with different number of blocks indexed then at least we need to add some special comparison method for subdomain_count but there will be more issues.

Export fails on larger databases.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Project	Deployment	Preview	Comments	Updated (UTC)
admin.ensnode.io	❌ Failed			Aug 13, 2025 2:05pm
ensnode.io	❌ Failed			Aug 13, 2025 2:05pm
ensrainbow.io	❌ Failed			Aug 13, 2025 2:05pm

[changeset-bot]

⚠️ No Changeset found

Latest commit: 99daddd

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR