Add v3 scoring model (Diet Egg) and Fresh Egg advisory#48
Add v3 scoring model (Diet Egg) and Fresh Egg advisory#48jeffreyksmithjr wants to merge 4 commits intomainfrom
Conversation
v3 uses alltime merge rate as the sole scoring input, dropping graph score and log account age from the v2 logistic regression. Fresh accounts (under 365 days) get an advisory surfaced in all output formats without affecting the score. Bots are excluded from the advisory since their synthetic profiles have unreliable age data. - Add FreshAccountAdvisory model and TrustScore.fresh_account field - Add _score_v3() using merge_rate as raw and normalized score - Add _build_fresh_account_advisory() called in all scoring paths - Change default scoring_model from v1 to v3 - Add "Diet Egg" brand name for v3 in all formatters - Widen v2 component breakdown checks to include v3 - Add v3 to CLI, action, and MCP server validation - Update tests: 332 passing, 94% coverage
🥚 Good Egg: HIGH TrustScore: 81% Top Contributions
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a new v3 scoring model for evaluating trust, simplifying the scoring mechanism to rely solely on merge rate. It also incorporates a 'Fresh Egg' advisory to provide additional context for newer accounts, enhancing the overall insight into contributor trustworthiness. The default scoring model is updated to v3, and bots are excluded from the fresh account advisory. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Point the Good Egg workflow at ./ instead of the published @v1 tag so the PR runs the branch code. Set scoring-model to v3.
Remove the local checkout used for testing. Drop the explicit scoring-model since v3 is now the default.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the v3 scoring model ('Diet Egg'), which simplifies scoring to be based solely on the contributor's merge rate, and adds a 'Fresh Egg' advisory for accounts under a year old. The default model is also updated to v3. The changes are well-structured, logical, and supported by a comprehensive set of new tests. My review includes a couple of minor suggestions to improve code maintainability by updating a docstring and centralizing a configuration value to avoid duplication.
| @@ -36,7 +36,11 @@ def _existing_contributor_context(score: TrustScore) -> tuple[int, str]: | |||
|
|
|||
| def _brand_name(score: TrustScore) -> str: | |||
| """Return 'Better Egg' for v2, 'Good Egg' for v1.""" | |||
There was a problem hiding this comment.
The docstring for _brand_name is now outdated with the addition of the 'v3' scoring model. Please update it to reflect the new logic and include 'Diet Egg'.
| """Return 'Better Egg' for v2, 'Good Egg' for v1.""" | |
| """Return 'Diet Egg' for v3, 'Better Egg' for v2, or 'Good Egg' for v1.""" |
| """Build a fresh account advisory from user profile data.""" | ||
| age_days = user_data.profile.account_age_days | ||
| return FreshAccountAdvisory( | ||
| is_fresh=age_days < 365, |
There was a problem hiding this comment.
The threshold 365 is hardcoded here, which duplicates the default value from the FreshAccountAdvisory model in models.py. This could lead to inconsistencies if the threshold is ever changed in one place but not the other.
To ensure a single source of truth, I recommend retrieving the default value from the model's definition rather than repeating the magic number. For example:
threshold = FreshAccountAdvisory.model_fields['threshold_days'].default
# ...
is_fresh=age_days < threshold- README: update scoring models table, add Fresh Egg section, update description to reflect v3 as default - docs/configuration.md: add v3 to scoring model section, YAML schema, env var table, and scoring_model description - docs/github-action.md: update inputs/outputs tables, replace "Using Better Egg" with general "Selecting a Scoring Model" section, add diet-egg-workflow to example list - docs/library.md: replace v2-only config section with general scoring model selection, add fresh_account to TrustScore field table, update raw_score description - docs/mcp-server.md: update all parameter tables and return examples to show v3 as default - docs/methodology.md: add Diet Egg (v3) section with motivation, Fresh Egg advisory, and component scores; update scoring section to lead with v3 - docs/troubleshooting.md: add v3 and Fresh Egg troubleshooting sections - examples: add diet-egg-workflow.yml, update .good-egg.yml with v3 comments, update library_usage.py with v3 and fresh_account usage - assets: replace PR comment screenshot with Diet Egg version
1 participant
Summary
Changes
models.pyFreshAccountAdvisorymodel,fresh_accountfield onTrustScoreconfig.pyscoring_modelLiteral extended to includev3, default changed tov3scorer.py_score_v3(),_build_fresh_account_advisory(), routing, bot exclusionformatter.pycli.pyclick.Choiceaction.pymcp_server.pytest_scorer.pyTestV3Scoring(9 tests),TestFreshAccountAdvisory(7 tests), existing tests fixed for new defaulttest_formatter.pyTestDietEggFormatting(6 tests),TestFreshAccountFormatting(9 tests)test_models.pyFreshAccountAdvisoryconstruction and serialization teststest_config.pyLive testing results
Tested against real GitHub profiles from the blog post validation set.
Established contributors
Less established / unknown contributors
Edge cases
Test plan
uv run pytest --cov=good_egg -v-- 332 passed, 94% coverageuv run ruff check src/ tests/ scripts/-- cleanuv run mypy src/good_egg/-- cleanCloses #46, closes #47