Skip to content

Audit and harden Maestro flows under flows/ with cross-platform guards and expanded negative coverage#6947

Draft
Copilot wants to merge 8 commits into
feat/openspec-integrationfrom
copilot/audit-update-maestro-yaml-flows
Draft

Audit and harden Maestro flows under flows/ with cross-platform guards and expanded negative coverage#6947
Copilot wants to merge 8 commits into
feat/openspec-integrationfrom
copilot/audit-update-maestro-yaml-flows

Conversation

Copy link
Copy Markdown

Copilot AI commented May 11, 2026

This PR audits every existing Maestro flow in flows/ and tightens reliability for Android+iOS execution. It also expands scenario coverage with additional negative-path journeys and documents what remains outside reliable Maestro-only automation.

  • Flow reliability hardening (all existing flows/*.yaml)

    • Replaced brittle assertions/selectors with more stable checks and conditional branches.
    • Normalized waits/navigation patterns to reduce flaky transitions.
    • Removed fragile assumptions tied to a single runtime state.
  • Reusable composition for consistency

    • Added shared helper flows to reduce duplication and centralize unstable interactions:
      • _common_unlock_if_needed.yaml
      • _common_enter_passcode_1111.yaml
      • _common_open_wallet.yaml
      • _common_open_manage_keys.yaml
      • _common_handle_runtime_permissions.yaml
    • Updated existing flows to compose via runFlow rather than repeating inline steps.
  • Expanded journey and negative-path coverage

    • Added:
      • sendInvalidInput.yaml (missing/invalid send input paths)
      • cancellationNavigation.yaml (cancel/back behavior across key screens)
      • permissionHandling.yaml (deny/allow handling for runtime permissions)
    • Expanded sanity.yaml to exercise broader end-to-end journeys already implied by existing coverage (onboarding, wallet lifecycle, receive/send, key and app settings surfaces).
  • Explicit automation boundary documentation

    • Added FLOWS_LIMITATIONS.md to capture remaining app coverage gaps and flows that cannot be made fully deterministic with Maestro alone (external hardware/app handoffs, OS dialog variance, network-dependent outcomes, etc.).

Example of the new reusable pattern:

# flows/send.yaml
appId: ${APPID}
---
- runFlow: _common_open_wallet.yaml
- tapOn:
    id: "btn_Send"
- runFlow: _common_handle_runtime_permissions.yaml
- assertVisible:
    id: "input_receive_address"
Original prompt

Create a pull request in KeeperCommunity/bitcoin-keeper against branch feat/openspec-integration that audits and updates the Maestro YAML test flows under the flows/ folder only.

Requirements:

  • Review every existing flow under flows/ and fix issues so the YAML scripts are as robust as possible for both Android and iOS.
  • Keep all changes limited to the flows/ folder.
  • Ensure the flows use valid Maestro YAML syntax and improve reliability where selectors/assertions or timing are brittle.
  • Where possible, make the flows cross-platform for Android and iOS rather than Android-only.
  • Add broader coverage using Maestro in the flows/ folder, including happy-path and negative-path scenarios for major app capabilities.
  • Cover practical user journeys already implied by the existing tests: onboarding/setup, passcode creation and validation, login, wallet creation, wallet viewing, wallet settings/details editing, receive, copy address, send, buy bitcoin, app settings, backup/export seed, version history, subscription, key management, and refresh/health-check related flows.
  • Add negative-path tests where reasonable, such as invalid passcode confirmation, invalid send input, missing/invalid address handling, validation messages, cancellation/back navigation, and permission handling.
  • Before finalizing, validate consistency across all flows and prefer reusable flow composition where appropriate.
  • Do not modify app source code outside flows/.
  • The final PR should also make clear what app coverage is still missing or what cannot be fully automated with Maestro alone.

The user specifically asked for 'perfect .yaml scripts' that run on Android and iOS, but you should be realistic: improve them significantly and document any platform limitations or app behaviors that prevent full Maestro-only automation.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: @KeeperCommunity/bitcoin-keeper/files/flows
here are my older test cases

Assistant: I’ll inspect the flows folder in KeeperCommunity/bitcoin-keeper and review the older test cases there.

Assistant: I found your older flows test cases in KeeperCommunity/bitcoin-keeper at commit 8805000cd32e029d9dd8a4865fece85b13a214e7.

Files present in flows/:

  • addNewKey.yaml
  • addwallet.yaml
  • appsettings.yaml
  • buyBTC.yaml
  • copywalletaddress.yaml
  • editwallet.yaml
  • editwalletdetails.yaml
  • exportseed.yaml
  • healthCheckKey.yaml
  • hidendeletekey.yaml
  • keySetting.yaml
  • login.yaml
  • newapp.yaml
  • receive.yaml
  • receivesats.yaml
  • refreshwallet.yaml
  • sanity.yaml
  • send.yaml
  • setpin.yaml
  • subscription.yaml
  • versionhistory.yaml
  • viewwallet.yaml
  • walletSetting.yaml

A few representative older cases I inspected:

  • sanity.yaml → bootstraps with newapp, addwallet, addNewKey
  • send.yaml → send flow with address entry, amount, fee selection, passcode, success check
  • receive.yaml → receive screen open/back flow
  • viewwallet.yaml → wallet detail screen assertions
  • appsettings.yaml → app settings assertions plus export seed/version history
  • setpin.yaml → passcode creation/confirmation flow

Example snippet:

appId: ${APPID}
---
- runFlow: newapp.yaml
- runFlow: addwallet.yaml
- runFlow: addNewKey.yaml

Code search results may be incomplete because search responses are limited to 10 results. You can view more in GitHub UI here:
https://github.com/KeeperCommunity/bitcoin-keeper/search?q=path%3A%2F%5Eflows%5C%2F.%2A%2F&type=code

If you want, I can next:

  1. summarize what each old flow is testing,
  2. compare these old Maestro flows with your new OpenSpec flows,
  3. identify brittle selectors/assertions that should be modernized.

User: just create new maestro test cases

Assistant: Please specify exactly which new Maestro test cases you want created under KeeperCommunity/bitcoin-keeper on branch feat/openspec-integration.

For example, tell me:

  • the scenarios to add, or
  • the screens/features to cover, or
  • whether you want me to convert the old flows/ cases into new Maestro tests on that branch.

If you want, you can simply say something like:

  • “Convert all old flows/*.yaml into new Maestro test cases on feat/openspec-integration
  • “Create Maestro tests for login, receive, send, and app settings”
  • “Add smoke tests only”

Once you confirm the scope, I can create the files.

User: I want you to generate maestro test cases for bitcoin keeper app.

Assistant: I’m waiting for your response to the approval dialog so I can continue creating the new Maestro test cases.

User: @copilot Accepted Confirmation: Confirm agent session

Assistant: Started — I’ve kicked off the work to generate the new Maestro test cases for KeeperCommunity/bitcoin-keeper from `feat/openspec-integrati...

This pull request was created from Copilot chat.

Copilot AI and others added 7 commits May 11, 2026 05:31
Co-authored-by: cakesoft-swati <62699947+cakesoft-swati@users.noreply.github.com>
Co-authored-by: cakesoft-swati <62699947+cakesoft-swati@users.noreply.github.com>
Co-authored-by: cakesoft-swati <62699947+cakesoft-swati@users.noreply.github.com>
Co-authored-by: cakesoft-swati <62699947+cakesoft-swati@users.noreply.github.com>
Co-authored-by: cakesoft-swati <62699947+cakesoft-swati@users.noreply.github.com>
Copilot AI changed the title [WIP] Audit and update Maestro YAML test flows in flows folder Audit and harden Maestro flows under flows/ with cross-platform guards and expanded negative coverage May 11, 2026
Copilot AI requested a review from cakesoft-swati May 11, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants