Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/good-egg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ jobs:
- uses: 2ndSetAI/good-egg@v1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
scoring-model: v2
skip-known-contributors: 'false'
29 changes: 18 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
Good Egg
</h1>

Trust scoring for GitHub PR authors using graph-based analysis of
contribution history.
Trust scoring for GitHub PR authors based on contribution history.

## Why

Expand Down Expand Up @@ -122,23 +121,31 @@ See [docs/mcp-server.md](https://github.com/2ndSetAI/good-egg/blob/main/docs/mcp

## Scoring Models

Good Egg supports two scoring models:
Good Egg supports three scoring models:

| Model | Name | Description |
|-------|------|-------------|
| `v1` | Good Egg (default) | Graph-based scoring from contribution history |
| `v3` | Diet Egg (default) | Alltime merge rate as sole signal |
| `v2` | Better Egg | Graph score + merge rate + account age via logistic regression |
| `v1` | Good Egg | Graph-based scoring from contribution history |

To use v2, set `scoring_model: v2` in your `.good-egg.yml`, pass
`--scoring-model v2` on the CLI, or set `scoring-model: v2` in the action
input. See [Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md#better-egg-v2) for how the
v2 model works.
v3 is the default. To use an older model, set `scoring_model: v1` or
`scoring_model: v2` in your `.good-egg.yml`, pass `--scoring-model v1` on
the CLI, or set `scoring-model: v1` in the action input. See
[Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md) for how each model works.

### Fresh Egg Advisory

Accounts less than 365 days old receive a "Fresh Egg" advisory in the
output. This is informational only and does not affect the score. Fresh
accounts correlate with lower merge rates in the validation data.

## How It Works

Good Egg builds a weighted contribution graph from a user's merged PRs and
runs personalized graph scoring to produce a trust score relative to your
project. See [Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md) for details.
The default v3 model (Diet Egg) scores contributors by their alltime merge
rate: merged PRs divided by total PRs (merged + closed). Older models (v1,
v2) build a weighted contribution graph and run personalized graph scoring.
See [Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md) for details.

## Trust Levels

Expand Down
4 changes: 2 additions & 2 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ inputs:
required: false
default: 'false'
scoring-model:
description: 'Scoring model to use (v1 or v2)'
description: 'Scoring model to use (v1, v2, or v3)'
required: false
skip-known-contributors:
description: 'Skip scoring for authors with merged PRs in the repo (true/false)'
Expand All @@ -40,7 +40,7 @@ outputs:
description: 'GitHub username that was scored'
value: ${{ steps.score.outputs.user }}
scoring-model:
description: 'Scoring model that was used (v1 or v2)'
description: 'Scoring model that was used (v1, v2, or v3)'
value: ${{ steps.score.outputs.scoring-model }}
skipped:
description: 'Whether scoring was skipped for an existing contributor (true/false)'
Expand Down
Binary file modified assets/pr-comment-screenshot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
36 changes: 22 additions & 14 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,23 +27,24 @@ Configuration values are resolved in this order (highest priority first):

## Scoring Model

Good Egg supports two scoring models. Set the model at the top level of the
config file:
Good Egg supports three scoring models. Set the model at the top level of
the config file:

```yaml
scoring_model: v1 # default -- graph-based scoring only
scoring_model: v3 # default -- Diet Egg -- alltime merge rate as sole signal
scoring_model: v2 # Better Egg -- graph + external features via logistic regression
scoring_model: v1 # Good Egg -- graph-based scoring only
```

When using v2, PR comments are branded "Better Egg" instead of "Good Egg".
See [methodology.md](methodology.md#better-egg-v2) for how the v2 model
PR comments are branded "Diet Egg", "Better Egg", or "Good Egg" depending
on the model. See [methodology.md](methodology.md) for how each model
works.

## Full YAML Schema

```yaml
# Scoring model selection: v1 (default) or v2
scoring_model: v1
# Scoring model selection: v3 (default), v2, or v1
scoring_model: v3

# Skip scoring for authors who already have merged PRs in the target repo.
# When true (the default), existing contributors get an EXISTING_CONTRIBUTOR
Expand Down Expand Up @@ -127,12 +128,19 @@ v2:

### scoring_model

Selects the scoring model. Set to `v1` (default) for graph-only scoring or
`v2` for the Better Egg combined model. When set to `v2`, the parameters
under the `v2:` block are used and the graph construction is simplified
(no self-contribution penalty, no language normalization in repo quality, no
diversity/volume adjustment). Language match personalization weighting
(`same_language_weight`) is retained in v2.
Selects the scoring model. `v3` (default, Diet Egg) uses alltime merge rate
as the sole signal with no graph construction. `v2` (Better Egg) combines a
simplified graph score with merge rate and account age via logistic
regression. `v1` (Good Egg) uses graph-based scoring only.

When set to `v2`, the parameters under the `v2:` block are used and the
graph construction is simplified (no self-contribution penalty, no language
normalization in repo quality, no diversity/volume adjustment). Language
match personalization weighting (`same_language_weight`) is retained in v2.

v3 does not use graph construction, so the `graph_scoring`, `recency`,
`edge_weights`, and `language_normalization` sections have no effect. The
`thresholds` section still controls trust level classification.

### v2 (Better Egg)

Expand Down Expand Up @@ -215,7 +223,7 @@ The following environment variables override individual config values:
| `GOOD_EGG_HIGH_TRUST` | `thresholds.high_trust` | float |
| `GOOD_EGG_MEDIUM_TRUST` | `thresholds.medium_trust` | float |
| `GOOD_EGG_HALF_LIFE_DAYS` | `recency.half_life_days` | int |
| `GOOD_EGG_SCORING_MODEL` | `scoring_model` | str (`v1` or `v2`) |
| `GOOD_EGG_SCORING_MODEL` | `scoring_model` | str (`v1`, `v2`, or `v3`) |
| `GOOD_EGG_SKIP_KNOWN_CONTRIBUTORS` | `skip_known_contributors` | bool (`true`/`false`) |

## Programmatic Configuration
Expand Down
19 changes: 11 additions & 8 deletions docs/github-action.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This posts a trust score comment on each pull request:
| `comment` | No | `true` | Post a PR comment with the trust score |
| `check-run` | No | `false` | Create a check run with the trust score |
| `fail-on-low` | No | `false` | Fail the action if trust level is LOW |
| `scoring-model` | No | `v1` | Scoring model: `v1` (Good Egg) or `v2` (Better Egg) |
| `scoring-model` | No | `v3` | Scoring model: `v3` (Diet Egg), `v2` (Better Egg), or `v1` (Good Egg) |
| `skip-known-contributors` | No | `true` | Skip scoring for authors with merged PRs in the repo |

## Outputs
Expand All @@ -47,7 +47,7 @@ This posts a trust score comment on each pull request:
| `score` | Normalized trust score (0.0 - 1.0) |
| `trust-level` | Trust level: HIGH, MEDIUM, LOW, UNKNOWN, BOT, or EXISTING_CONTRIBUTOR |
| `user` | GitHub username that was scored |
| `scoring-model` | Scoring model used: `v1` (Good Egg) or `v2` (Better Egg) |
| `scoring-model` | Scoring model used: `v3` (Diet Egg), `v2` (Better Egg), or `v1` (Good Egg) |
| `skipped` | Whether scoring was skipped for an existing contributor (`true`/`false`) |

## Custom Configuration
Expand Down Expand Up @@ -179,9 +179,10 @@ jobs:

You can check whether scoring was skipped via the `skipped` output.

## Using Better Egg (v2)
## Selecting a Scoring Model

To use the v2 scoring model, set the `scoring-model` input:
The default model is v3 (Diet Egg), which scores by alltime merge rate. To
use an older model, set the `scoring-model` input:

```yaml
jobs:
Expand All @@ -191,12 +192,12 @@ jobs:
- uses: 2ndSetAI/good-egg@v0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
scoring-model: v2
scoring-model: v2 # or v1
```

When using v2, PR comments are branded "Better Egg" and include a component
score breakdown showing graph score, merge rate, and account age
contributions. The `scoring-model` output reflects which model was used.
PR comments are branded according to the model: "Diet Egg" for v3, "Better
Egg" for v2, "Good Egg" for v1. v2 and v3 include a component score
breakdown. The `scoring-model` output reflects which model was used.

You can also set `scoring-model` via the `GOOD_EGG_SCORING_MODEL` environment
variable, but the input takes precedence.
Expand All @@ -209,5 +210,7 @@ See the [examples/](../examples/) directory for complete workflow files:
that posts a PR comment
- [strict-workflow.yml](../examples/strict-workflow.yml) -- comment, check
run, and fail-on-low
- [diet-egg-workflow.yml](../examples/diet-egg-workflow.yml) -- v3
scoring model (default) with component breakdown
- [better-egg-workflow.yml](../examples/better-egg-workflow.yml) -- v2
scoring model with component breakdown
44 changes: 21 additions & 23 deletions docs/library.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,26 +114,22 @@ result = await score_pr_author(
)
```

### v2 (Better Egg) Configuration
### Scoring Model Selection

To use the v2 scoring model, set `scoring_model` on the config:
The default model is v3 (Diet Egg). To use an older model, set
`scoring_model` on the config:

```python
from good_egg import GoodEggConfig, score_pr_author

config = GoodEggConfig(
scoring_model="v2",
v2={
"graph": {"half_life_days": 180, "max_age_days": 730},
"features": {"merge_rate": True, "account_age": True},
"combined_model": {
"intercept": -0.8094,
"graph_score_weight": 1.9138,
"merge_rate_weight": -0.7783,
"account_age_weight": 0.1493,
},
},
)
# v3 (default) -- merge rate only
config = GoodEggConfig()

# v2 -- graph + merge rate + account age
config = GoodEggConfig(scoring_model="v2")

# v1 -- graph only
config = GoodEggConfig(scoring_model="v1")

result = await score_pr_author(
login="octocat",
Expand All @@ -142,12 +138,13 @@ result = await score_pr_author(
config=config,
)

# v2 results include component scores
# v3 and v2 results include component scores
if result.component_scores:
print(f"Graph score: {result.component_scores['graph_score']:.3f}")
print(f"Merge rate: {result.component_scores['merge_rate']:.3f}")
print(f"Log account age: {result.component_scores['log_account_age']:.3f}")
print(f"Normalized score: {result.normalized_score:.3f}")
print(f"Merge rate: {result.component_scores.get('merge_rate')}")

# v3 includes a fresh account advisory
if result.fresh_account and result.fresh_account.is_fresh:
print(f"Fresh account: {result.fresh_account.account_age_days} days old")

print(f"Scoring model: {result.scoring_model}")
```
Expand Down Expand Up @@ -176,7 +173,7 @@ the following fields:
|-------|------|-------------|
| `user_login` | `str` | GitHub username that was scored |
| `context_repo` | `str` | Repository used as scoring context |
| `raw_score` | `float` | Pre-normalization score: graph score (v1) or logit (v2) |
| `raw_score` | `float` | Pre-normalization score: merge rate (v3), logit (v2), or graph score (v1) |
| `normalized_score` | `float` | Normalized score (0.0 - 1.0) |
| `trust_level` | `TrustLevel` | HIGH, MEDIUM, LOW, UNKNOWN, BOT, or EXISTING_CONTRIBUTOR |
| `account_age_days` | `int` | Age of the GitHub account in days |
Expand All @@ -185,9 +182,10 @@ the following fields:
| `top_contributions` | `list[ContributionSummary]` | Top repositories contributed to |
| `language_match` | `bool` | Whether the user's top language matches the context repo |
| `flags` | `dict[str, bool]` | Flags (is_bot, is_new_account, etc.) |
| `scoring_model` | `str` | Scoring model used: `v1` or `v2` |
| `component_scores` | `dict[str, float] \| None` | Component breakdown (v2 only): `graph_score`, `merge_rate`, `log_account_age` |
| `scoring_model` | `str` | Scoring model used: `v1`, `v2`, or `v3` |
| `component_scores` | `dict[str, float]` | Component breakdown (v3: `merge_rate`; v2: `graph_score`, `merge_rate`, `log_account_age`) |
| `scoring_metadata` | `dict[str, Any]` | Internal scoring details |
| `fresh_account` | `FreshAccountAdvisory \| None` | Advisory for accounts under 365 days old (None for bots and existing contributors) |

`TrustScore` is a Pydantic model, so you can serialize it:

Expand Down
43 changes: 25 additions & 18 deletions docs/mcp-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,17 @@ Returns the full trust score as JSON, including all fields from the
|------|------|----------|-------------|
| `username` | `string` | Yes | GitHub username to score |
| `repo` | `string` | Yes | Target repository in `owner/repo` format |
| `scoring_model` | `string` | No | Scoring model: `v1` (Good Egg, default) or `v2` (Better Egg) |
| `scoring_model` | `string` | No | Scoring model: `v3` (Diet Egg, default), `v2` (Better Egg), or `v1` (Good Egg) |
| `force_score` | `boolean` | No | Force full scoring even for known contributors (default: `false`) |

**Returns:** Full `TrustScore` JSON with all fields (user_login,
context_repo, raw_score, normalized_score, trust_level,
account_age_days, total_merged_prs, unique_repos_contributed,
top_contributions, language_match, flags, scoring_model, component_scores,
scoring_metadata). When `scoring_model` is `v2`, the response includes
`component_scores` with graph_score, merge_rate, and log_account_age.
scoring_metadata, fresh_account). v3 includes `component_scores` with
`merge_rate`. v2 includes `graph_score`, `merge_rate`, and
`log_account_age`. The `fresh_account` field contains a Fresh Egg advisory
for accounts under 365 days old (null for bots and existing contributors).

### check_pr_author

Expand All @@ -102,17 +104,21 @@ Returns a compact summary suitable for quick checks.
|------|------|----------|-------------|
| `username` | `string` | Yes | GitHub username to check |
| `repo` | `string` | Yes | Target repository in `owner/repo` format |
| `scoring_model` | `string` | No | Scoring model: `v1` (Good Egg, default) or `v2` (Better Egg) |
| `scoring_model` | `string` | No | Scoring model: `v3` (Diet Egg, default), `v2` (Better Egg), or `v1` (Good Egg) |
| `force_score` | `boolean` | No | Force full scoring even for known contributors (default: `false`) |

**Returns (v1):**
**Returns (v3, default):**

```json
{
"user_login": "octocat",
"trust_level": "HIGH",
"normalized_score": 0.82,
"total_merged_prs": 47
"total_merged_prs": 47,
"scoring_model": "v3",
"component_scores": {
"merge_rate": 0.82
}
}
```

Expand Down Expand Up @@ -143,18 +149,18 @@ Returns an expanded breakdown with contributions, flags, and metadata.
|------|------|----------|-------------|
| `username` | `string` | Yes | GitHub username to analyse |
| `repo` | `string` | Yes | Target repository in `owner/repo` format |
| `scoring_model` | `string` | No | Scoring model: `v1` (Good Egg, default) or `v2` (Better Egg) |
| `scoring_model` | `string` | No | Scoring model: `v3` (Diet Egg, default), `v2` (Better Egg), or `v1` (Good Egg) |
| `force_score` | `boolean` | No | Force full scoring even for known contributors (default: `false`) |

**Returns (v1):**
**Returns (v3, default):**

```json
{
"user_login": "octocat",
"context_repo": "octocat/Hello-World",
"trust_level": "HIGH",
"normalized_score": 0.82,
"raw_score": 0.0045,
"raw_score": 0.82,
"account_age_days": 3650,
"total_merged_prs": 47,
"unique_repos_contributed": 12,
Expand All @@ -171,19 +177,26 @@ Returns an expanded breakdown with contributions, flags, and metadata.
"is_bot": false,
"is_new_account": false
},
"scoring_metadata": {}
"scoring_model": "v3",
"component_scores": {
"merge_rate": 0.82
},
"scoring_metadata": {
"closed_pr_count": 10
},
"fresh_account": null
}
```

**Returns (v2):**
**Returns (v1):**

```json
{
"user_login": "octocat",
"context_repo": "octocat/Hello-World",
"trust_level": "HIGH",
"normalized_score": 0.82,
"raw_score": 0.2871,
"raw_score": 0.0045,
"account_age_days": 3650,
"total_merged_prs": 47,
"unique_repos_contributed": 12,
Expand All @@ -200,12 +213,6 @@ Returns an expanded breakdown with contributions, flags, and metadata.
"is_bot": false,
"is_new_account": false
},
"scoring_model": "v2",
"component_scores": {
"graph_score": 0.78,
"merge_rate": 0.91,
"log_account_age": 3.45
},
"scoring_metadata": {}
}
```
Expand Down
Loading