Skip to content

fix: validate Slack credentials before enabling channels#4582

Merged
cv merged 5 commits into
mainfrom
fix/1912-slack-token-validation
Jun 1, 2026
Merged

fix: validate Slack credentials before enabling channels#4582
cv merged 5 commits into
mainfrom
fix/1912-slack-token-validation

Conversation

@ericksoa
Copy link
Copy Markdown
Contributor

@ericksoa ericksoa commented May 31, 2026

Summary

  • add Slack Web API validation for bot tokens via auth.test and app tokens via apps.connections.open
  • require Slack validation before onboarding saves credentials or keeps Slack enabled
  • require Slack validation before channels add slack persists credentials or registers providers

Tests

  • npm run build:cli
  • npm test -- test/onboard-messaging.test.ts src/lib/onboard/messaging-channel-setup.test.ts src/lib/onboard/messaging-token.test.ts src/lib/onboard/slack-validation.test.ts test/channels-add-preset.test.ts
  • npm run typecheck:cli
  • git diff --check

Fixes #1912

Summary by CodeRabbit

  • New Features

    • Centralized Slack credential validation during onboarding and channel-adding; reuses existing tokens and avoids duplicate prompts.
  • Bug Fixes

    • Invalid or indeterminate Slack credentials are not persisted, cause Slack to be omitted from enabled channels, and abort channel-add flows with a clear, redacted message; successful-but-skipped checks emit warnings.
  • Docs

    • Added env flag to skip live Slack auth validation.
  • Tests

    • Expanded unit, integration, and e2e tests for Slack validation, fake-token skip, sequencing, and preset/channel flows.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 31, 2026

Review Change Stack

Important

Review skipped

This PR was authored by the user configured for CodeRabbit reviews. CodeRabbit does not review PRs authored by this user. It's recommended to use a dedicated user account to post CodeRabbit review feedback.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a44bcf78-aeeb-459b-b859-6f1c0852fead

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds Slack bot/app token probing and validation, integrates it into interactive and non-interactive onboarding and sandbox channel addition, and updates unit, integration, and e2e tests and harnesses to assert probe→persist ordering and prevent token leakage.

Changes

Slack Token Validation Integration

Layer / File(s) Summary
Slack validation module + helpers
src/lib/onboard/slack-validation.ts
Adds types, skip-flag, curl-probe helpers, validateSlackBotToken, validateSlackAppToken, validateSlackCredentials, failure formatting, and filterSlackSelectionByValidation.
Slack validation unit tests
src/lib/onboard/slack-validation.test.ts
Tests for probe building, bot/app endpoints, rejection/transient classification, sequencing, unreadable/network failure handling, skip-flag behavior, and token-redaction assertions.
Interactive setup: collect & validate Slack tokens
src/lib/onboard/messaging-channel-setup.ts
Adds setupSlackTokens flow: reuse/env or prompt, format-validate slots, pair-validate, persist only on success, and remove Slack from enabled set on failure.
Interactive setup tests
src/lib/onboard/messaging-channel-setup.test.ts
Mocks validation to assert no credential persistence or token leakage on rejected/indeterminate validation and verifies Slack removal from enabled channels.
Non-interactive selection filtering
src/lib/onboard.ts, test/onboard-messaging.test.ts
Applies filterSlackSelectionByValidation after Telegram reachability filtering to drop Slack when validation fails; tests ensure Slack is omitted and tokens are not printed.
Sandbox/API channel addition
src/lib/actions/sandbox/policy-channel.ts, src/lib/actions/sandbox/slack-channel-validation.ts
Adds ChannelDef key checks, required-token presence, per-entry format checks, and calls validateSlackCredentials; aborts before persisting/applying if validation fails (messages redacted).
Channels add preset integration tests & harness
test/channels-add-preset.test.ts
Extends child-process preamble to inject Slack envs, capture credentialSaveCalls, slackProbeCalls, provider upserts, and call order; adds success, skip, invalid, and indeterminate validation tests.
E2E scripts: fake-token skip gates
test/e2e/*.sh
Adds is_fake_slack_token() helpers and sets NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION=1 when fake/test tokens are detected to skip live Slack validation in E2E fake-token runs.
Docs: skip env var
docs/reference/commands.mdx
Documents NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION environment flag used to skip live Slack credential probes while still enforcing token format checks.

Sequence Diagram

sequenceDiagram
  participant User as User/Setup Flow
  participant Collector as setupSlackTokens
  participant Validator as validateSlackCredentials
  participant ProbeBot as validateSlackBotToken
  participant ProbeApp as validateSlackAppToken
  participant SlackAPI as Slack API
  participant Saver as saveCredential
  User->>Collector: collect bot token (prompt/env)
  Collector->>Collector: collect app token (prompt/env)
  Collector->>Validator: validate botToken + appToken
  Validator->>ProbeBot: validate bot token (auth.test)
  ProbeBot->>SlackAPI: auth.test with Bearer botToken
  SlackAPI-->>ProbeBot: response
  ProbeBot-->>Validator: SlackTokenValidationResult
  alt Bot valid
    Validator->>ProbeApp: validate app token (apps.connections.open)
    ProbeApp->>SlackAPI: apps.connections.open with Bearer appToken
    SlackAPI-->>ProbeApp: response
    ProbeApp-->>Validator: SlackTokenValidationResult
    Validator-->>Collector: { ok: true }
    Collector->>Saver: saveCredential(SLACK_BOT_TOKEN, botToken)
    Collector->>Saver: saveCredential(SLACK_APP_TOKEN, appToken)
    Collector-->>User: credentials saved, process.env updated
  else Validation fails
    Validator-->>Collector: { ok: false, credential: 'bot'|'app', message }
    Collector-->>User: remove slack from enabled, log redacted error
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#4403: Both PRs modify onboarding/sandbox flows around addSandboxChannel and provider/registry registration ordering.
  • NVIDIA/NemoClaw#4200: Both PRs adjust handling of preconfigured Slack tokens and gate onboarding on validation/format checks.

Suggested labels

NemoClaw CLI

"🐰 I hopped through code to vet each Slack key,
Bot and app tokens probed, no secrets to see.
Probes first, then save — bogus strings don't stay,
Warnings redacted, setup keeps tokens away.
Hop, test, and ship — the rabbit cheers hooray!"

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.92% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: validating Slack credentials before enabling channels, which directly addresses the core objective of the PR.
Linked Issues check ✅ Passed The PR implements all objectives from issue #1912: validates Slack tokens via Web API (auth.test for bot, apps.connections.open for app), enforces token format validation, blocks credential persistence and channel enabling on validation failure, and surfaces validation errors.
Out of Scope Changes check ✅ Passed All changes are scoped to Slack credential validation during onboarding and channel setup. E2E test updates consistently implement fake-token detection to skip validation, which supports the primary feature without introducing unrelated functionality.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@ericksoa ericksoa changed the title Validate Slack credentials before enabling channels fix: validate Slack credentials before enabling channels May 31, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 31, 2026

E2E Advisor Recommendation

Required E2E: messaging-providers-e2e, hermes-slack-e2e, openclaw-slack-pairing-e2e, token-rotation-e2e, channels-stop-start-e2e
Optional E2E: channels-add-remove-e2e, network-policy-e2e

Dispatch hint: messaging-providers-e2e,hermes-slack-e2e,openclaw-slack-pairing-e2e,token-rotation-e2e,channels-stop-start-e2e

Auto-dispatched E2E: messaging-providers-e2e, hermes-slack-e2e, openclaw-slack-pairing-e2e, token-rotation-e2e, channels-stop-start-e2e via nightly-e2e.yaml at 13e2fb9afd1d76508be164f91b7fc574183e6ecfnightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • messaging-providers-e2e (high): Best required coverage for the changed Slack credential/provider path: validates Slack provider creation, placeholder rewrite, secret isolation, channel config, and fake Slack API plumbing after onboarding with Slack enabled.
  • hermes-slack-e2e (medium): Required because the PR changes Slack onboarding validation and updates the Hermes Slack E2E skip behavior; this validates Hermes Slack policy, providers, credential rewrite, and sandbox reachability.
  • openclaw-slack-pairing-e2e (medium): Required to protect the OpenClaw Slack real user flow affected by Slack credential gating: hermetic Socket Mode pairing, provider placeholder rewrite, and connect-shell approval.
  • token-rotation-e2e (medium): Required because Slack validation now runs before saving/using credentials and token rotation depends on correct credential hash detection and propagation for Slack bot/app tokens.
  • channels-stop-start-e2e (high): Required because the PR changes channel lifecycle code and this E2E exercises Slack stop/start across OpenClaw and Hermes, including cached credentials, provider reattachment, registry state, and policy persistence.

Optional E2E

  • channels-add-remove-e2e (high): Optional adjacent confidence for the common channels add/remove and policy-preset lifecycle. It is Telegram-only, so it does not directly validate the new Slack auth gate, but it can catch regressions in shared addSandboxChannel plumbing.
  • network-policy-e2e (medium): Optional adjacent coverage because Slack channel enablement depends on the Slack policy preset and egress boundaries; the PR touches policy-channel code but does not change policy YAML directly.

New E2E recommendations

  • Slack channels add credential validation (high): Existing channels-add-remove-e2e is Telegram-only, while this PR adds Slack-specific live validation before nemoclaw <sandbox> channels add slack persists tokens or registers providers. The unit/integration tests cover this with stubs, but there is no real sandbox E2E for the explicit Slack channels-add path.
    • Suggested test: Add a selective E2E that runs nemoclaw <sandbox> channels add slack with fake Slack tokens plus NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION=1, then verifies provider registration, Slack preset application, rebuild behavior, and no token leakage; include a hermetic rejected-validation negative case if feasible without live Slack secrets.
  • Live Slack auth validation negative path (medium): The new fail-closed behavior distinguishes rejected and indeterminate Slack API validation, but current E2E coverage mostly uses fake tokens with skip mode to preserve hermeticity.
    • Suggested test: Add a hermetic Slack API probe mock or dependency-injected curl endpoint E2E to verify onboard/channels-add abort before credential persistence when Slack validation returns invalid_auth or times out.

Dispatch hint

  • Workflow: .github/workflows/nightly-e2e.yaml
  • jobs input: messaging-providers-e2e,hermes-slack-e2e,openclaw-slack-pairing-e2e,token-rotation-e2e,channels-stop-start-e2e

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 31, 2026

E2E Scenario Advisor Recommendation

Required scenario E2E: ubuntu-repo-cloud-openclaw-slack
Optional scenario E2E: ubuntu-repo-cloud-hermes-slack

Dispatch required scenario E2E:

  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-slack

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required scenario E2E

  • ubuntu-repo-cloud-openclaw-slack: Primary scenario coverage for Slack messaging onboarding/configuration. The PR changes Slack credential validation during onboarding and channels add, including live auth probes and skip behavior, which can affect the OpenClaw Slack messaging scenario.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-slack

Optional scenario E2E

  • ubuntu-repo-cloud-hermes-slack: Adjacent Slack onboarding coverage for the Hermes agent. Useful to confirm the new Slack validation behavior does not regress the Hermes Slack profile, but OpenClaw Slack is the primary targeted scenario.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-hermes-slack

Relevant changed files

  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/actions/sandbox/slack-channel-validation.ts
  • src/lib/onboard.ts
  • src/lib/onboard/messaging-channel-setup.ts
  • src/lib/onboard/slack-validation.ts

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 31, 2026

PR Review Advisor

Findings: 0 needs attention, 2 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 2 still apply, 0 new items found

Review findings

🛠️ Needs attention

  • None.

🔎 Worth checking

  • Runtime validation is still recommended for invalid Slack credential host-glue paths (src/lib/actions/sandbox/policy-channel.ts:829): The PR adds strong unit and subprocess coverage for Slack credential validation and verifies that mocked credential saves, provider registration, registry updates, and preset application do not happen after rejected or indeterminate Slack validation. The changed behavior still crosses runtime-sensitive host-glue boundaries: Slack probes, credential persistence, OpenShell provider records, NemoClaw registry/session state, policy preset application, and sandbox creation. The E2E fake-token paths added here explicitly set NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION, so they do not prove the real invalid-credential path leaves no Slack provider, channel state, registry update, session preset, or active policy in an actual runtime.
    • Recommendation: Add or identify a targeted runtime/integration validation that exercises invalid Slack credentials through onboarding and `channels add slack`, then observes real gateway/provider, registry/session, and policy state to confirm Slack is not enabled and no provider or preset is created.
    • Evidence: `validateSlackChannelCredentials()` is called before `persistChannelTokens()` and `applyChannelAddToGatewayAndRegistry()`, and tests assert mocked `credentialSaveCalls`, `providerCalls`, `registryUpdates`, and `appliedCalls` stay empty on failed validation. Deterministic test-depth context still reports `runtime_validation_recommended` for the changed host-glue files; E2E scripts set `NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION=1` for fake Slack tokens.
  • Slack auth-validation skip can still enable invalid Slack credentials (src/lib/onboard/slack-validation.ts:43): `NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION` is documented for restricted networks and hermetic tests, and this revision narrows the accepted values to explicit truthy strings. However, when it is set, Slack setup and `channels add slack` intentionally skip the live `auth.test` and `apps.connections.open` checks and can proceed to save/register credentials that Slack would reject. This is not a token leak or sandbox escape, but it weakens the validation guarantee if the environment variable is inherited in a real user or CI environment.
    • Recommendation: Keep the warning highly visible and consider further narrowing the bypass, for example requiring interactive confirmation outside non-interactive/test contexts, using a more test-scoped bypass name, or documenting operational cleanup when the flag is used.
    • Evidence: `shouldSkipSlackAuthValidation()` accepts `1`, `true`, `yes`, or `on`; `validateSlackCredentials()` returns an ok skipped result before probing Slack; tests assert skip mode performs no Slack probe and still allows credentials to be saved/registered.

🌱 Nice ideas

  • None.
Since last review details

Current findings:

  • Runtime validation is still recommended for invalid Slack credential host-glue paths (src/lib/actions/sandbox/policy-channel.ts:829): The PR adds strong unit and subprocess coverage for Slack credential validation and verifies that mocked credential saves, provider registration, registry updates, and preset application do not happen after rejected or indeterminate Slack validation. The changed behavior still crosses runtime-sensitive host-glue boundaries: Slack probes, credential persistence, OpenShell provider records, NemoClaw registry/session state, policy preset application, and sandbox creation. The E2E fake-token paths added here explicitly set NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION, so they do not prove the real invalid-credential path leaves no Slack provider, channel state, registry update, session preset, or active policy in an actual runtime.
    • Recommendation: Add or identify a targeted runtime/integration validation that exercises invalid Slack credentials through onboarding and `channels add slack`, then observes real gateway/provider, registry/session, and policy state to confirm Slack is not enabled and no provider or preset is created.
    • Evidence: `validateSlackChannelCredentials()` is called before `persistChannelTokens()` and `applyChannelAddToGatewayAndRegistry()`, and tests assert mocked `credentialSaveCalls`, `providerCalls`, `registryUpdates`, and `appliedCalls` stay empty on failed validation. Deterministic test-depth context still reports `runtime_validation_recommended` for the changed host-glue files; E2E scripts set `NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION=1` for fake Slack tokens.
  • Slack auth-validation skip can still enable invalid Slack credentials (src/lib/onboard/slack-validation.ts:43): `NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION` is documented for restricted networks and hermetic tests, and this revision narrows the accepted values to explicit truthy strings. However, when it is set, Slack setup and `channels add slack` intentionally skip the live `auth.test` and `apps.connections.open` checks and can proceed to save/register credentials that Slack would reject. This is not a token leak or sandbox escape, but it weakens the validation guarantee if the environment variable is inherited in a real user or CI environment.
    • Recommendation: Keep the warning highly visible and consider further narrowing the bypass, for example requiring interactive confirmation outside non-interactive/test contexts, using a more test-scoped bypass name, or documenting operational cleanup when the flag is used.
    • Evidence: `shouldSkipSlackAuthValidation()` accepts `1`, `true`, `yes`, or `on`; `validateSlackCredentials()` returns an ok skipped result before probing Slack; tests assert skip mode performs no Slack probe and still allows credentials to be saved/registered.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard.ts`:
- Around line 5591-5610: The function validateSlackSelection should be moved out
of src/lib/onboard.ts into a new or existing onboarding submodule under
src/lib/onboard/ (e.g., src/lib/onboard/validateSlackSelection.ts) and exported
from that module; in onboard.ts replace the function body with an import and
keep only the call site. Update any imports: export default or named export of
validateSlackSelection from the new file and import it in onboard.ts, and ensure
references to helper utilities used inside (getValidatedMessagingTokenByEnvKey,
MESSAGING_CHANNELS, validateSlackCredentials, formatSlackValidationFailure) are
available via imports in the new module. Also run or update any tests/refs that
import validateSlackSelection so CI stops failing the growth guardrail.

In `@src/lib/onboard/slack-validation.ts`:
- Around line 25-35: Update the TRANSIENT_SLACK_ERRORS set to match Slack’s
documented Web API error codes: keep only "ratelimited" (note spelling) and
"request_timeout" as transient errors and remove "rate_limited", "timeout",
"internal_error", "fatal_error", "service_unavailable", etc.; leave the existing
SLACK_AUTH_TEST_URL and SLACK_APPS_CONNECTIONS_OPEN_URL constants as-is (they
are correct). Use the constant names TRANSIENT_SLACK_ERRORS,
SLACK_AUTH_TEST_URL, and SLACK_APPS_CONNECTIONS_OPEN_URL to locate and modify
the set accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0a8610d9-5f1d-49ff-9b37-0f21e9f4c940

📥 Commits

Reviewing files that changed from the base of the PR and between 9641ce0 and e5b79b4.

📒 Files selected for processing (8)
  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/onboard.ts
  • src/lib/onboard/messaging-channel-setup.test.ts
  • src/lib/onboard/messaging-channel-setup.ts
  • src/lib/onboard/slack-validation.test.ts
  • src/lib/onboard/slack-validation.ts
  • test/channels-add-preset.test.ts
  • test/onboard-messaging.test.ts

Comment thread src/lib/onboard.ts Outdated
Comment thread src/lib/onboard/slack-validation.ts Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26717554171
Target ref: e5b79b493b06de91f06b23a5749837ba6657d947
Workflow ref: main
Requested jobs: messaging-providers-e2e,channels-stop-start-e2e,token-rotation-e2e
Summary: 0 passed, 1 failed, 0 skipped

Job Result
channels-stop-start-e2e ⚠️ cancelled
messaging-providers-e2e ❌ failure
token-rotation-e2e ⚠️ cancelled

Failed jobs: messaging-providers-e2e. Check run artifacts for logs.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/lib/onboard/messaging-channel-setup.test.ts (1)

171-198: 💤 Low value

Optionally assert token secrecy for parity with the rejected case.

The sibling "rejected" test (Lines 165-168) captures logs and asserts the raw tokens never appear in output. This indeterminate case skips that check. Since secrecy is a stated PR objective, adding a log capture + negative assertion would close the parity gap.

♻️ Optional: add log capture and secrecy assertions
     const enabled = new Set(["slack"]);
+    const logs: string[] = [];
+    vi.spyOn(console, "log").mockImplementation((message = "") => {
+      logs.push(String(message));
+    });

     await setupSelectedMessagingChannels(
       ["slack"],
       enabled,
       [{ name: "slack", ...KNOWN_CHANNELS.slack }],
     );

     expect(enabled.has("slack")).toBe(false);
     expect(saveCredential).not.toHaveBeenCalled();
     expect(process.env.SLACK_BOT_TOKEN).toBeUndefined();
     expect(process.env.SLACK_APP_TOKEN).toBeUndefined();
+    const output = logs.join("\n");
+    expect(output).not.toContain("xoxb-timeout-bot-token");
+    expect(output).not.toContain("xapp-timeout-app-token");
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/messaging-channel-setup.test.ts` around lines 171 - 198, Add
the same log-capture and negative token-secrecy assertions used in the
"rejected" sibling test to the "does not save prompted Slack credentials when
Slack API validation is indeterminate" test: capture console output (e.g., via
the test helper used elsewhere or spy on console.log/error) around the call to
setupSelectedMessagingChannels, then assert that the literal token strings
("xoxb-timeout-bot-token" and "xapp-timeout-app-token") do not appear in the
captured logs; keep the existing assertions for enabled.has("slack"),
saveCredential, and process.env unchanged and locate this change in the test
function that calls setupSelectedMessagingChannels, prompt,
validateSlackCredentials and checks saveCredential/process.env.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/onboard/messaging-channel-setup.test.ts`:
- Around line 171-198: Add the same log-capture and negative token-secrecy
assertions used in the "rejected" sibling test to the "does not save prompted
Slack credentials when Slack API validation is indeterminate" test: capture
console output (e.g., via the test helper used elsewhere or spy on
console.log/error) around the call to setupSelectedMessagingChannels, then
assert that the literal token strings ("xoxb-timeout-bot-token" and
"xapp-timeout-app-token") do not appear in the captured logs; keep the existing
assertions for enabled.has("slack"), saveCredential, and process.env unchanged
and locate this change in the test function that calls
setupSelectedMessagingChannels, prompt, validateSlackCredentials and checks
saveCredential/process.env.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 321e2fba-9b78-4647-8d9e-0ac0d6050040

📥 Commits

Reviewing files that changed from the base of the PR and between 76b58cf and b49d8c8.

📒 Files selected for processing (6)
  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/actions/sandbox/slack-channel-validation.ts
  • src/lib/onboard/messaging-channel-setup.test.ts
  • src/lib/onboard/slack-validation.test.ts
  • src/lib/onboard/slack-validation.ts
  • test/channels-add-preset.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/channels-add-preset.test.ts
  • src/lib/onboard/slack-validation.ts

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26717926316
Target ref: b49d8c83bd19cb6ae98edaaae820dc369fb93c59
Workflow ref: main
Requested jobs: messaging-providers-e2e,openclaw-slack-pairing-e2e,hermes-slack-e2e
Summary: 0 passed, 3 failed, 0 skipped

Job Result
hermes-slack-e2e ❌ failure
messaging-providers-e2e ❌ failure
openclaw-slack-pairing-e2e ❌ failure

Failed jobs: hermes-slack-e2e, messaging-providers-e2e, openclaw-slack-pairing-e2e. Check run artifacts for logs.

@ericksoa ericksoa self-assigned this May 31, 2026
@ericksoa ericksoa added bug Something fails against expected or documented behavior priority: high NV QA Bugs found by the NVIDIA QA Team integration: slack Slack integration or channel behavior UAT Issues flagged for User Acceptance Testing. v0.0.56 Release target labels May 31, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/channels-add-preset.test.ts (1)

25-34: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Clear the Slack-validation skip flag in the default child env.

runScript() inherits NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION from the parent process, so the rejected/indeterminate Slack tests can silently bypass probes and stop exercising the failure path whenever that flag is set in CI or a developer shell. Seed it to "" here and let the one opt-out test override it to "1".

🧪 Suggested fix
     env: {
       ...process.env,
+      NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION: "",
       HOME: tmpDir,
       NEMOCLAW_NON_INTERACTIVE: "1",
       TELEGRAM_BOT_TOKEN: "test-telegram-token",
       SLACK_BOT_TOKEN: "xoxb-slack-bot-token-for-test",
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/channels-add-preset.test.ts` around lines 25 - 34, The child process env
object used when calling runScript() is inheriting
NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION from the parent, allowing Slack validation
to be skipped in tests; update the env block in test/channels-add-preset.test.ts
(the env passed into runScript()/child process) to explicitly set
NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION: "" so the default tests exercise the
validation failure paths, and leave the individual opt-out test to set
NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION = "1" as needed to override it.
🧹 Nitpick comments (1)
test/e2e/test-token-rotation.sh (1)

159-164: 💤 Low value

Consider extracting shared helper to a library script.

The is_fake_slack_token helper is duplicated across 5 E2E scripts. While self-contained E2E tests have value for portability, you could extract this to test/e2e/lib/slack-helpers.sh alongside existing shared libraries like sandbox-teardown.sh if you find yourself adding more Slack-related test utilities.

This is optional—the current duplication is acceptable for E2E tests.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/test-token-rotation.sh` around lines 159 - 164, Extract the
duplicated is_fake_slack_token function into a shared shell library (e.g.,
slack-helpers.sh) and update the E2E scripts to source that library instead of
redefining the function; specifically, move the existing function body (the case
pattern matching xoxb-fake-*, xoxb-test-*, xapp-fake-*, xapp-test-*) into the
new slack-helpers.sh and in each E2E script replace the local definition with a
. ./slack-helpers.sh (or source) line so tests call is_fake_slack_token from the
shared module and avoid duplication.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/slack-validation.ts`:
- Around line 39-54: The skip-warning is misleading because
shouldSkipSlackAuthValidation (and helper isTruthyEnvFlag) accept "1", "true",
"yes", "on" as skip values but skippedSlackValidationResult always claims the
sentinel was "=1"; update the code so the message accurately reflects the actual
value used or the accepted set: either (1) restrict isTruthyEnvFlag to only
accept "1" (if you want the documented sentinel) or (2) change
skippedSlackValidationResult to read the real env value via
env[SLACK_AUTH_VALIDATION_SKIP_ENV] (or include the accepted list) and include
that in the message; modify the functions isTruthyEnvFlag,
shouldSkipSlackAuthValidation, and skippedSlackValidationResult and reference
SLACK_AUTH_VALIDATION_SKIP_ENV accordingly so the log matches the runtime
behavior.

---

Outside diff comments:
In `@test/channels-add-preset.test.ts`:
- Around line 25-34: The child process env object used when calling runScript()
is inheriting NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION from the parent, allowing
Slack validation to be skipped in tests; update the env block in
test/channels-add-preset.test.ts (the env passed into runScript()/child process)
to explicitly set NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION: "" so the default tests
exercise the validation failure paths, and leave the individual opt-out test to
set NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION = "1" as needed to override it.

---

Nitpick comments:
In `@test/e2e/test-token-rotation.sh`:
- Around line 159-164: Extract the duplicated is_fake_slack_token function into
a shared shell library (e.g., slack-helpers.sh) and update the E2E scripts to
source that library instead of redefining the function; specifically, move the
existing function body (the case pattern matching xoxb-fake-*, xoxb-test-*,
xapp-fake-*, xapp-test-*) into the new slack-helpers.sh and in each E2E script
replace the local definition with a . ./slack-helpers.sh (or source) line so
tests call is_fake_slack_token from the shared module and avoid duplication.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 27b624f9-4412-4b41-90dc-9bb32505ee5b

📥 Commits

Reviewing files that changed from the base of the PR and between b49d8c8 and 346fbb3.

📒 Files selected for processing (13)
  • docs/reference/commands.mdx
  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/actions/sandbox/slack-channel-validation.ts
  • src/lib/onboard/messaging-channel-setup.ts
  • src/lib/onboard/slack-validation.test.ts
  • src/lib/onboard/slack-validation.ts
  • test/channels-add-preset.test.ts
  • test/e2e/test-channels-stop-start.sh
  • test/e2e/test-hermes-slack-e2e.sh
  • test/e2e/test-messaging-providers.sh
  • test/e2e/test-openclaw-slack-pairing.sh
  • test/e2e/test-token-rotation.sh
  • test/onboard-messaging.test.ts
✅ Files skipped from review due to trivial changes (1)
  • docs/reference/commands.mdx
🚧 Files skipped from review as they are similar to previous changes (4)
  • src/lib/actions/sandbox/slack-channel-validation.ts
  • test/onboard-messaging.test.ts
  • src/lib/onboard/messaging-channel-setup.ts
  • src/lib/onboard/slack-validation.test.ts

Comment thread src/lib/onboard/slack-validation.ts
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26719155930
Target ref: 346fbb340f80494343e458d5c9689de89e9aeba3
Workflow ref: main
Requested jobs: messaging-providers-e2e,openclaw-slack-pairing-e2e,hermes-slack-e2e,token-rotation-e2e,channels-stop-start-e2e
Summary: 5 passed, 0 failed, 0 skipped

Job Result
channels-stop-start-e2e ✅ success
hermes-slack-e2e ✅ success
messaging-providers-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
token-rotation-e2e ✅ success

@ericksoa
Copy link
Copy Markdown
Contributor Author

Reviewer update after the Slack validation follow-ups:

  • Added NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION as an explicit opt-out for restricted networks and hermetic tests. This skips only the live Slack auth.test / apps.connections.open probes; existing Slack token format checks still apply.
  • Updated fake-token E2E harnesses to set that opt-out only when they are using xoxb-fake-* / xoxb-test-* / xapp-fake-* / xapp-test-* tokens, so real-token runs stay on the live Slack validation path.
  • Addressed CodeRabbit feedback in the latest follow-up commit 13e2fb9af:
    • changed the skip warning/docs so they match all accepted truthy flag values rather than saying only =1;
    • cleared NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION in the channel-add child-process test harness by default, so rejected/indeterminate Slack validation tests cannot silently inherit skip mode from the parent environment.
  • Left the E2E is_fake_slack_token helper duplication as-is for now. CodeRabbit marked that as low-value/optional, and the current small local helper keeps these standalone E2E scripts portable without broadening the patch.

Local verification for the pushed follow-up:

npm test -- src/lib/onboard/slack-validation.test.ts test/channels-add-preset.test.ts test/check-env-var-docs.test.ts
npm run typecheck:cli
git diff --check

Current CI/advisor state on latest head 13e2fb9afd1d76508be164f91b7fc574183e6ecf:

  • Required PR checks are green, including CodeRabbit, E2E recommendation, PR review advisor, aggregate checks, CodeQL, unit-vitest-linux, WSL/macOS, docs, ShellCheck, and self-hosted smoke jobs.
  • The E2E advisor dispatched the five Slack/channel jobs for this latest SHA in run https://github.com/NVIDIA/NemoClaw/actions/runs/26720047254: messaging-providers-e2e, hermes-slack-e2e, openclaw-slack-pairing-e2e, token-rotation-e2e, and channels-stop-start-e2e.
  • The same five-job advisor batch passed 5/5 on the immediately previous SHA in run https://github.com/NVIDIA/NemoClaw/actions/runs/26719155930; the latest run is still in progress after the tiny reviewer-feedback follow-up.

@ericksoa
Copy link
Copy Markdown
Contributor Author

Follow-up on the latest advisor items:

  • The PR Review Advisor has 0 needs attention. Its remaining two items are both risk/context notes rather than blockers.
  • For the runtime/integration proof recommendation: QA/UAT will cover the real runtime handshake and host-glue verification. This PR does not make invalid credentials easier to accept by default; it adds live Slack auth validation. The only bypass is the explicit NEMOCLAW_SKIP_SLACK_AUTH_VALIDATION override for restricted networks/hermetic tests.
  • For the skip-flag warning recommendation: the CLI now prints a visible warning whenever live Slack validation is skipped, the flag is documented as restricted-network/hermetic-test-only, and the channel-add test harness explicitly clears inherited skip state by default so failure-path coverage cannot silently bypass validation.
  • I am not adding another source change just to reword the warning again because that would reset CI and likely trigger another advisor E2E auto-dispatch. The current wording and docs already make the bypass explicit.

E2E status context:

@ericksoa ericksoa requested a review from cv May 31, 2026 18:30
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26720047254
Target ref: 13e2fb9afd1d76508be164f91b7fc574183e6ecf
Workflow ref: main
Requested jobs: messaging-providers-e2e,hermes-slack-e2e,openclaw-slack-pairing-e2e,token-rotation-e2e,channels-stop-start-e2e
Summary: 5 passed, 0 failed, 0 skipped

Job Result
channels-stop-start-e2e ✅ success
hermes-slack-e2e ✅ success
messaging-providers-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
token-rotation-e2e ✅ success

@ericksoa
Copy link
Copy Markdown
Contributor Author

Required Scenario Advisor coverage completed: ubuntu-repo-cloud-openclaw-slack passed on latest head 13e2fb9afd1d76508be164f91b7fc574183e6ecf in https://github.com/NVIDIA/NemoClaw/actions/runs/26721017816.

@cv cv added v0.0.57 Release target and removed v0.0.56 Release target labels Jun 1, 2026
@cv cv merged commit 084613e into main Jun 1, 2026
38 checks passed
@cv cv deleted the fix/1912-slack-token-validation branch June 1, 2026 18:58
miyoungc added a commit that referenced this pull request Jun 1, 2026
## Summary

- Adds the v0.0.56 release notes section with links to the deeper docs
pages for installer, status, inference, messaging, policy, and lifecycle
changes.
- Updates source docs for the remaining release-prep gaps around `uv` in
the PyPI preset, compact WhatsApp pairing guidance, and `nemoclaw
inference set` command boundaries.
- Refreshes generated `nemoclaw-user-*` skills and removes skipped
experimental command terms from generated skill surfaces.

## Source summary

- #4613 -> `docs/manage-sandboxes/lifecycle.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Documents
that public installs and `nemoclaw update` follow the maintained `lkg`
tag by default.
- #4419 -> `docs/about/release-notes.mdx`: Notes that non-interactive
Linux installs can reactivate Docker group membership and continue in
one installer run when `sg docker` is available.
- #4550 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Captures live sandbox agent-version
probing for status, connect, and upgrade checks.
- #4609 -> `docs/inference/use-local-inference.mdx`,
`docs/about/release-notes.mdx`: Captures the GPU Docker-driver
host-network local-inference reachability gate.
- #4607 -> `docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Documents
compact WhatsApp QR pairing guidance and gateway/session diagnostics.
- #4582 -> `docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Reflects
Slack credential validation before enabling the channel.
- #4554 -> `docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/troubleshooting.mdx`, `docs/about/release-notes.mdx`:
Keeps Telegram allowlist alias guidance in the generated user skills and
release notes.
- #4563 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Includes the new `nemoclaw <name> skill
remove <skill>` command in command docs and release notes.
- #4566 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Documents the `nemoclaw inference set`
redirect boundary when `--provider` or `--model` is missing.
- #4323 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Captures per-sandbox status JSON
support.
- #4506 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Captures debug command sandbox-name
validation and safer tarball writing.
- #4569 -> `docs/network-policy/integration-policy-examples.mdx`,
`docs/about/release-notes.mdx`: Documents that the `pypi` preset allows
`/usr/local/bin/uv`.
- #4579 -> `docs/network-policy/integration-policy-examples.mdx`,
`docs/about/release-notes.mdx`: Captures observable Jira preset
validation guidance.
- #4229 -> `docs/manage-sandboxes/lifecycle.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Documents
user-data preservation defaults for uninstall.
- #4399 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Captures CPU-only sandbox intent
preservation across rebuilds.
- #4058 -> `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Captures safer snapshot restore behavior
around existing destinations.
- #4155 and #4460 -> skipped by `docs/.docs-skip`: Removed skipped
experimental command terms from source docs and generated skill evals
instead of documenting those features.

## Verification

- `python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix
nemoclaw-user --doc-platform fern-mdx`
- `npm run docs` (passes; Fern reports the pre-existing light-mode
accent contrast warning)
- `rg "permissive mode|shields down|shields up|shields status|config
rotate-token|rotate-token" .agents/skills` (no matches)
- `npm run build:cli` (run to refresh local CLI artifacts for the
pre-push TypeScript hook)
- Commit hooks passed, including `NEMOCLAW_* env-var documentation
gate`, `Verify docs-to-skills output`, `markdownlint-cli2`, `gitleaks`,
and `Test (skills YAML)`.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Expanded Model Router setup with YAML examples, flow diagrams, and
credential handling; strengthened agent-config immutability and
integrity guidance; messaging channels updated (Telegram aliases,
WhatsApp pairing/diagnostics); CLI docs revised (GPU detection,
inference set behavior, uninstall/rebuild preservation); overview
rebranded to NemoClaw and added v0.0.56 release notes.

* **New Features**
* Added `nemoclaw <name> channels status` (messaging diagnostics, JSON);
added `nemoclaw <name> skill remove`; Hermes no longer marked
experimental; DGX Spark quickstart sandbox-name note.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@wscurran wscurran added area: messaging Messaging channels, bridges, manifests, or channel lifecycle bug-fix PR fixes a bug or regression feature PR adds or expands user-visible functionality and removed priority: high labels Jun 3, 2026
cv pushed a commit that referenced this pull request Jun 3, 2026
## Summary
- Add the missing `v0.0.57` release-notes section with links to the
detailed docs pages for command, inference, onboarding, messaging,
status, installer, and policy changes.
- Remove public references to docs-skip terms from source docs and
regenerate the NemoClaw user skills from the current Fern MDX docs.
- Carry forward generated references for the per-agent documentation
split, including Hermes-specific reference files.

## Source summary
- #4615 and #4653 -> `docs/about/release-notes.mdx`,
`docs/reference/commands.mdx`: Release notes now cover host-side
`sessions` and `agents` commands plus `NEMOCLAW_EXTRA_AGENTS_JSON`
secondary-agent baking.
- #4163, #4204, #4611, #4619, and #4676 ->
`docs/about/release-notes.mdx`,
`docs/inference/use-local-inference.mdx`: Release notes now cover
managed vLLM progress/readiness, DGX Spark model default changes, local
Ollama streaming usage, and inference route divergence warnings.
- #4267, #4601, #4609, #4642, #4645, and #4661 ->
`docs/about/release-notes.mdx`, `docs/reference/commands.mdx`: Release
notes now cover UFW auto-remediation, local-inference reachability
gates, gateway reuse/binding, cancel rollback, and policy selection
persistence.
- #4577, #4582, #4607, and #4660 -> `docs/about/release-notes.mdx`,
`docs/manage-sandboxes/messaging-channels.mdx`: Release notes now cover
Slack validation, atomic `channels add`, WhatsApp QR diagnostics, and
Slack placeholder normalization.
- #4388, #4600, #4646, and #4647 -> `docs/about/release-notes.mdx`,
`docs/reference/commands.mdx`: Release notes now cover status failure
layers, paused-container hints, Docker-driver doctor behavior, and
non-destructive stale-registry recovery.
- #4569, #4579, and #4678 -> `docs/about/release-notes.mdx`,
`docs/manage-sandboxes/lifecycle.mdx`,
`docs/network-policy/integration-policy-examples.mdx`: Release notes now
cover installer tag pinning, PyPI `uv` policy access, and observable
Jira validation.
- #4632 -> `.agents/skills/`: Regenerated user skills from the current
per-agent docs source, including newly generated Hermes reference files.

## Verification
- `python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix
nemoclaw-user --doc-platform fern-mdx`
- `rg "permissive mode|shields down|shields up|shields status|config
rotate-token|rotate-token" docs --glob "*.mdx"`
- `rg "permissive mode|shields down|shields up|shields status|config
rotate-token|rotate-token" .agents/skills --glob "*.md"`
- `npm run docs`
- `npm run build:cli`
- Commit hooks: markdownlint, docs-to-skills verification, gitleaks,
skills YAML, commitlint

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Restructured documentation to clearly distinguish OpenClaw and Hermes
agent variants throughout user guides.
* Enhanced security, credential storage, and deployment guidance with
clearer setup flows.
  * Added Hermes plugin installation and ecosystem documentation.
* Improved workspace, messaging, and policy management references with
variant-specific command examples.
  * Refined troubleshooting and CLI reference sections for clarity.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: messaging Messaging channels, bridges, manifests, or channel lifecycle bug Something fails against expected or documented behavior bug-fix PR fixes a bug or regression feature PR adds or expands user-visible functionality integration: slack Slack integration or channel behavior NV QA Bugs found by the NVIDIA QA Team UAT Issues flagged for User Acceptance Testing. v0.0.57 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[All Platform] Slack Bot Token is accepted and saved without validation during setup

4 participants