Open
Conversation
- Introduced `convert-csv` command for converting CSV files to .ensrainbow format. - Added support for single and two-column CSV formats. - Implemented error handling for invalid CSV data. - Created tests for various CSV scenarios, including special characters and invalid formats. - Updated package dependencies to include `csv-simple-parser` for CSV parsing.
- Introduced `--existing-db-path` option to filter out existing labels from an ENSRainbow database during CSV conversion. - Enhanced conversion process to skip duplicate labels within the same CSV file. - Updated logging to include statistics on filtered labels. - Added comprehensive tests for filtering functionality and updated documentation to reflect new features.
- Added new command-line options for CSV conversion: `--silent`, `--disable-dedup`, `--cache-size`, `--use-bloom-filter`, and `--bloom-filter-size`. - Implemented a deduplication database using ClassicLevel with optional Bloom filter for faster processing. - Updated the conversion process to support deduplication and improved memory management. - Enhanced logging for large file processing and added tests for new deduplication features.
- Added a function to estimate memory usage of Maps for better tracking. - Reduced default cache size in DeduplicationDB from 10000 to 1000. - Enhanced backpressure handling during CSV writing to prevent memory overflow. - Updated logging to include output backpressure events and improved performance for large files. - Streamlined the CSV processing to operate in a completely sequential manner.
- Removed unused command-line options for deduplication and Bloom filter from the CLI interface. - Updated default progress interval from 10000 to 50000 records for improved performance. - Enhanced logging for file processing and memory management during CSV conversion. - Cleaned up code for better readability and maintainability.
…s, adding references to the glossary for clarity. Update environment variable descriptions in `.env.local.example` files and improve comments in scripts to ensure consistent understanding of label set concepts.
…ipts to enhance clarity on label sets and versions. Add glossary references for better understanding across multiple files.
…criptions in Terraform variables for clarity on label sets and versions. Add references to the glossary for improved understanding.
…sh and add references to the ENSRainbow Glossary for comprehensive understanding.
…l" in the ENSRainbow overview and API client comments, improving clarity and user understanding.
…rd" in multiple files, improving clarity and user understanding of the data model and related concepts.
…VERSION references and enhancing clarity in environment variable definitions. Update examples for consistency and improve overall readability.
…set reference links, ensuring consistency and clarity in documentation. Enhance user understanding of environment variables related to label sets.
Updated the URL for the labelhash glossary reference.
…creating `.ensrainbow` files from both SQL and CSV sources. Update the description of the `searchlight` label set to reflect its availability and improved dataset features. Revise performance metrics for the `searchlight` dataset and include usage examples for downloading the extended discovery dataset.
…sed dependencies from package.json
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive documentation explaining the fundamental problem of unknown labels in ENS and how ENSRainbow solves it.
Changes:
- Added a new "Unknown Labels" concept documentation page that explains what unknown labels are, why they exist, their impact on indexing and applications, and how ENSRainbow's healing service addresses them
- Updated the main ENSRainbow overview page to include a link to the new unknown labels documentation
- Updated the concepts index page to include the unknown labels page in the learning path and navigation
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| docs/ensnode.io/src/content/docs/ensrainbow/index.mdx | Added a reference link to the new unknown labels documentation page for comprehensive explanation |
| docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx | New comprehensive documentation explaining unknown labels, their causes, impacts, and how ENSRainbow's healing service works to mitigate them |
| docs/ensnode.io/src/content/docs/ensrainbow/concepts/index.mdx | Added the unknown labels page to the concepts navigation and inserted it as step 2 in the recommended learning path |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
djstrong
changed the title
Member
lightwalker-eth
left a comment
lightwalker-eth
left a comment
There was a problem hiding this comment.
@djstrong Thanks. Reviewed and shared feedback.
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
Co-authored-by: lightwalker.eth <126201998+lightwalker-eth@users.noreply.github.com>
📝 WalkthroughWalkthroughAdds a changeset for a patch to Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx`:
- Around line 86-90: Standardize usage by changing "onchain" to "on‑chain" and
use the verb form "look up" (not "lookup") when describing actions; specifically
update the phrase "onchain events" to "on‑chain events" and change "attempts to
lookup the label" (and any other instances where "lookup" is used as a verb) to
"attempts to look up the label" — apply the same edits for the other occurrence
referenced (around the "ENSRainbow lookup" / "labelhash" / "label" text).
- Line 18: The sentence "ENSRainbow serves the goal of minimizing the number of
unknown labels that exist, and therefore minimizing the probability that users
experience interacting them." is ungrammatical; update the sentence (in the
content containing the string "ENSRainbow serves the goal of minimizing the
number of unknown labels...") to a clearer wording such as: "ENSRainbow aims to
minimize the number of unknown labels and thus reduce the probability that users
encounter them." Replace the original sentence with this or an equivalent
grammatically correct variant.
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx`:
- Around line 72-75: The text currently claims ENSNode “heal 100% of subnames
under `addr.reverse`”, which is an absolute guarantee; update the copy to soften
this—replace the phrase in the paragraph referencing `addr.reverse` and ENSNode
so it reads something like “near-complete healing of subnames under
`addr.reverse`” or “currently covers all observed subnames under `addr.reverse`”
(or similar qualifying wording) and ensure the surrounding bullets referencing
ENSNode and `addr.reverse` are consistent with the toned-down claim.
Comment on lines
+72
to
+75
| - **Contract events**: Some contracts (like `ETHRegistrarController` or `NameWrapper`) emit the human-readable label in their events | ||
| - **[Rainbow table](/ensrainbow/concepts/glossary#rainbow-table) lookups**: The human-readable label for a given labelhash can be determined via customizable and massive rainbow table lookups through ENSRainbow | ||
| - **Heuristics for addr.reverse**: ENSNode uses specialized heuristics that heal 100% of subnames under `addr.reverse`, which represent reverse ENS records | ||
| - **Intelligent expansion strategies**: ENSNode has implemented strategies to intelligently expand the set of rainbow tables in ENSRainbow over time, ensuring healing coverage improves continuously |
There was a problem hiding this comment.
Avoid absolute “100%” claim unless guaranteed.
“heal 100% of subnames under addr.reverse” is a hard guarantee; consider softening (e.g., “near‑complete” / “currently covers all observed”) unless this is strictly enforced and invariant.
🤖 Prompt for AI Agents
In `@docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx`
around lines 72 - 75, The text currently claims ENSNode “heal 100% of subnames
under `addr.reverse`”, which is an absolute guarantee; update the copy to soften
this—replace the phrase in the paragraph referencing `addr.reverse` and ENSNode
so it reads something like “near-complete healing of subnames under
`addr.reverse`” or “currently covers all observed subnames under `addr.reverse`”
(or similar qualifying wording) and ensure the surrounding bullets referencing
ENSNode and `addr.reverse` are consistent with the toned-down claim.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lite PR
Summary
Why
This documentation fills a gap in explaining the core problem that ENSRainbow addresses. The unknown labels concept is fundamental to understanding ENSRainbow's purpose, but was previously only briefly mentioned without comprehensive explanation. This new page provides detailed coverage of what unknown labels are, why they exist, how they impact indexing and applications, and how ENSRainbow mitigates the problem.
Testing
Notes for Reviewer (Optional)
This is documentation-only; no code or runtime behavior changes.
Checklist