Skip to content

Auth + API key creation when using bt setup#110

Open
ViaDézo1er / cedric (viadezo1er) wants to merge 24 commits intomainfrom
cedric-better-dx-pr3
Open

Auth + API key creation when using bt setup#110
ViaDézo1er / cedric (viadezo1er) wants to merge 24 commits intomainfrom
cedric-better-dx-pr3

Conversation

@viadezo1er
Copy link
Copy Markdown
Contributor

@viadezo1er ViaDézo1er / cedric (viadezo1er) commented Apr 13, 2026

The OAuth-related changes, isolated from the wizard.

  • ensure_auth rewrite (credential recovery flow, is_missing_credential_error)
  • maybe_create_api_key_for_oauth (auto-create key, print-once)
  • login_interactive_oauth made pub(crate)
  • #[allow(dead_code)] on login_interactive_api_key (or just remove it)
  • Remove login_interactive (the old method-chooser)
  • Auth output tweaks (warning→info, spacing)

Part 3 of #94

Rename/restructure CLI flags to improve UX without changing core behaviour:

BaseArgs:
- Remove --quiet/-q (BRAINTRUST_QUIET) and --no-input; quiet output is
  now the global default (set_quiet(true) in configure_output).
- Add verbose: bool as #[arg(skip)] — subcommands set it at runtime;
  set up output messages are shown only when verbose is active.

SetupArgs:
- --no-mcp-skill → --no-skills + --no-mcp (orthogonal opt-outs)
- Add --no-instrument (opt-out; instrument is the new default)
- Add --tui / --background (replace --interactive / --quiet in wizard)
- Add --language (repeatable), --interactive/-i, --verbose/-v at top level

AgentsSetupArgs:
- --agent (repeatable Vec) → --agent (single Option); drop AgentArg::All
- --yes removed from CLI (#[arg(skip)]); programmatic callers still set yes:true
- Add --no-workflow (conflicts with --workflow)

AgentsMcpSetupArgs:
- Same agents→agent and yes→skip treatment

InstrumentSetupArgs:
- --quiet → --background (run agent non-interactively, show spinner)
- --interactive → --tui (run agent in interactive TUI mode)
- --yes → #[arg(skip)]
- Add skip_launch: bool (#[arg(skip)]) for future wizard use

resolve_selected_agents: signature changes from &[AgentArg] to
Option<AgentArg>; None auto-detects all installed agents as before.

All call sites, struct literals in tests, and BaseArgs literals in
auth/switch/traces/functions updated to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- `bt setup instrument --interactive` now accepted as alias for --tui
- `bt setup instrument --quiet` now accepted as alias for --background
- `bt setup --no-mcp-skill` now accepted as hidden alias for --no-skills --no-mcp
- Explicit error when --json and --tui are both passed
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 13, 2026

Latest downloadable build artifacts for this PR commit 294b778826fc:

Available artifact names
  • ``artifacts-build-global
  • ``artifacts-build-local-x86_64-apple-darwin
  • ``artifacts-build-local-x86_64-pc-windows-msvc
  • ``artifacts-build-local-x86_64-unknown-linux-musl
  • ``artifacts-build-local-x86_64-unknown-linux-gnu
  • ``artifacts-build-local-aarch64-unknown-linux-musl
  • ``artifacts-build-local-aarch64-apple-darwin
  • ``artifacts-build-local-aarch64-unknown-linux-gnu
  • ``artifacts-plan-dist-manifest
  • ``cargo-dist-cache

move CLI coverage to integration tests
auth.rs:
- Remove login_interactive (method-chooser) and login_interactive_api_key
- Make login_interactive_oauth pub(crate) so setup can call it directly
- Auth output tweaks: Warning→Info in maybe_warn_api_key_override, drop the
  else branch (no-profile case doesn't need a warning when an API key is used)
- Add blank lines around the OAuth authorization URL for readability
- Add org description hint before the org selection prompt
- Add .report(false) to the OAuth callback manual-paste input
- Add set_quiet(false) in run() so bt auth output is never silenced by the
  global quiet default (unless --json is active)
- Add pub(crate) ensure_auth: credential recovery flow that falls back to
  OAuth when a stored profile's credentials are inaccessible
- Add pub(crate) is_missing_credential_error helper used by ensure_auth

setup/mod.rs:
- Replace local ensure_auth with auth::ensure_auth
- Remove local LoginContext import (no longer needed)
- Add maybe_create_api_key_for_oauth: after an OAuth login, auto-create a
  permanent API key, expose it via BRAINTRUST_API_KEY, and print it once
  (the key is never retrievable again from the API)
- Call maybe_create_api_key_for_oauth in run_setup_wizard after auth step

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@viadezo1er ViaDézo1er / cedric (viadezo1er) changed the title Cedric better dx pr3 Auth + API key creation when using bt setup Apr 14, 2026
@viadezo1er ViaDézo1er / cedric (viadezo1er) marked this pull request as ready for review April 14, 2026 00:52
Comment thread src/auth.rs
Comment thread src/auth.rs Outdated
Comment thread src/setup/mod.rs Outdated
bt does not need an api key but the agent executing the code with instrumentation needs it
ViaDézo1er / cedric (viadezo1er) added a commit that referenced this pull request Apr 14, 2026
Self-contained infra changes that the wizard depends on.
- `tty_term()` function in `src/ui/select.rs` (`/dev/tty` fallback)
- `fuzzy_select` using `interact_on(&term)`
- `detect_runnable_agents()` (PATH-only check)
- `detect_agents` simplification (drop config-dir heuristics)
- `resolve_unambiguous_instrument_agent`
- `/dev/tty` stdin fallback in `run_agent_invocation` for interactive
mode

Cursor skill path detection, global quiet mode are fixed in the [3rd
PR](#110).

PATH only detection of the agent seem ok, I don't think people would
want to install skills/mcp without having the agent.


Part 2 of #94

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Abhijeet Prasad <abhijeet@braintrustdata.com>
@viadezo1er
Copy link
Copy Markdown
Contributor Author

ViaDézo1er / cedric (viadezo1er) commented Apr 15, 2026

Auth spec

For case where the user should be prompted but bt setup runs in non interactive mode (--no-input, --json), stop the setup.
-o/--org/-p/--project/--profile/'~/.bt/config.json' (config.json configures org and project fields)and their respective env variables: flags > env variables > config file in case of conflict between 2 indicators of the same field (like --org and BRAINTRUST_ORG_NAME).
When populating a profile , use the same behavior as if 'bt auth login' was run. If the profile to use is not specified:

  • For OAuth login, if a profile with the same auth_kind, api_url, app_url, org_name, user_name and email already exist (profile name does not matter), just update the oauth_client_id, oauth_access_expires_at and the stored secret without prompting the user.
    • If there are multiple OAuth profiles matching org/user, update the one that has the largest oauth_access_expires_at (ie the most recently created/updated).
  • For API key login, if a profile with the same auth_kind, api_url, and org_name already exist (profile name does not matter), create a new profile with the new api_key_hint and stored secret.

When trying to login with OAuth, first try to use the cached OAuth token, if it's missing/stale and there's a valid refresh token use it to get a new OAuth token, else browser-based OAuth workflow.
When a variable should be checked but there is an error (for example, --profile incorrect-profile-name), the setup fails and print an error.
A removed/revoked API key is treated as if it does not match any org.
In case of network error, 5xx from Braintrust, wrong API call, print an error and stop.
To check if a project is in an org, list the projects of the org with

curl --request GET \
  --url https://api.braintrust.dev/v1/project \
  --header 'Authorization: Bearer <token>'

then search the returned project list for the requested project name.

'goto aaa' means 'continue the program the line after the [aaa] line'.
Like in python, if there is no else statement just continue to the next line.
'If XXX is passed' means 'if XXX is in flags or env variables or config file'.
Decision tree:

if need authentication and instrumenting:
  All authentication use cases need an API key
  if API key passed:
    if profile passed:
      goto profile-passed-1
    else if org passed:
      if prefer-profile passed and 0 of the available profile's org matches the passed org:
        fail, end
      if API key's org matches passed org:
        [project-passed-1]
        if project is passed:
          if the project is in the passed org:
            use API key, end
          else:
            fail, end
        else:
          use API key, end
      else:
        OAuth, create API key, goto project-passed-1
    else if prefer-profile passed:
      if the number of available profile is 0:
        use API key, add a profile as if bt auth login with API key, end
      else if 1:
        if API key's org and profile's org match:
          use API key, end
        else:
          OAuth, create API key, end
      else if number of profiles' org matching the API key's org is 0:
        OAuth, create API key, end
      else if >=1:
        use API key, end
  goto no-org-no-profile-prefer-profile
end

[no-org-no-profile-prefer-profile]
if API key passed:
  if project is in API key's org:
    use API key, end
OAuth, create API key, end

[profile-passed-1]
if org passed:
  [profile-passed-2]
  if org and profile's org match:
    if API key passed:
      if the API key's org matches the profile's org:
        if project is passed and does not exist in org:
          fail, end
        else:
          use API key, end
    OAuth login.
    if project is passed and is not in the passed org:
      fail, end
    else:
      create API key, end
  else:
    fail, end
else:
  consider that passed org is the same as the profile's org
  goto profile-passed-2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants