Add manual smoke test plans for MS Office, Libre Office, Web Browsers, IDEs#18419
Add manual smoke test plans for MS Office, Libre Office, Web Browsers, IDEs#18419
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds manual smoke test plans to ensure NVDA functionality across MS Office, LibreOffice, and major web browsers when migrating to 64-bit.
- Introduce a comprehensive smoke test plan for web browsers (Chrome, Edge, Firefox).
- Add smoke test plans for Microsoft Office applications (Outlook, Word, PowerPoint, Excel).
- Add smoke test plans for LibreOffice applications (Writer, Calc, Impress).
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/manual/applications/webBrowsers.md | Add smoke test plan for major web browsers |
| tests/manual/applications/msOffice.md | Add smoke test plan covering MS Office applications |
| tests/manual/applications/libreOffice.md | Add smoke test plan for LibreOffice applications |
Comments suppressed due to low confidence (2)
tests/manual/applications/webBrowsers.md:62
- The link path for the Forms example appears to reference the landmarks directory; update it to the correct patterns/form/examples path to match other ARIA sample links.
* [Forms](../../../include/w3c-aria-practices/content/patterns/landmarks/examples/form.html)
tests/manual/applications/libreOffice.md:1
- [nitpick] Consider adding a UIA Settings section (similar to the MS Office plan) to specify required UIA toggles for LibreOffice, ensuring consistency across all manual test plans.
# NVDA LibreOffice Smoke Testing Plan
See test results for failed build of commit 43168a5564 |
seanbudd
changed the title
SaschaCowley
added
the
conceptApproved
|
|
||
| ### Steps | ||
|
|
||
| 1. Use `ctrl+f` to search within the current file |
There was a problem hiding this comment.
You already had us do that in Editor Accessibility step 3
|
|
||
| 1. Use `ctrl+f` to search within the current file | ||
| 1. Use `ctrl+shift+f` to search across the project | ||
| 1. Use `ctrl+shift+n` to find any file, class, or symbol |
There was a problem hiding this comment.
You said ctrl+shift+n creates a new project in Project Operations step 1.
| 1. Use `ctrl+shift+f` to search across the project | ||
| 1. Use `ctrl+shift+n` to find any file, class, or symbol | ||
| 1. Use `ctrl+n` to find a class | ||
| 1. Use `ctrl+shift+n` to find a file |
There was a problem hiding this comment.
You already had us use ctrl+shift+n to "find any file, class, or symbol"
|
|
||
| 1. Use `ctrl+alt+s` to open Settings/Preferences | ||
| 1. Navigate through settings categories using arrow keys | ||
| 1. Use `ctrl+alt+s` to open Plugins settings |
There was a problem hiding this comment.
You already had us use ctrl+alt+s to open settings
There was a problem hiding this comment.
Not a JetBrains user, so some of these comments may be obvious to experienced JetBrains users
|
|
||
| 1. Use `alt+p` to open the Plugins menu | ||
| 1. Navigate through available plugins | ||
| 1. Use `f7` to access Settings |
There was a problem hiding this comment.
Is this a default setting? I may have removed it, but it does not work in my copy of NPP
| ### Expected Results | ||
|
|
||
| * NVDA announces plugin names and settings options | ||
| * Plugin management and settings are accessible |
There was a problem hiding this comment.
Worth noting that there are a lot of problems with mislabelled controls in NPP's settings UI
| 1. Type and navigate code in the editor using arrow keys | ||
| 1. Use `ctrl+home` and `ctrl+end` to jump to start/end of file | ||
| 1. Use `ctrl+f` to open the find dialog and search for text | ||
| 1. Use `ctrl+h` to open the replace dialog |
There was a problem hiding this comment.
Has actually using text replacement intentionally been omitted?
|
|
||
| ### Steps | ||
|
|
||
| 1. Use `ctrl+f` to search within the current file |
There was a problem hiding this comment.
Already done in Code Editor Accessibility
There was a problem hiding this comment.
Not a Visual Studio user, so some of these questions/comments may be obvious to experienced users.
|
|
||
| 1. Launch Writer | ||
| 1. Verify NVDA announces the Writer window title | ||
| 1. Use `ctrl+n` to create a new document |
There was a problem hiding this comment.
Not sure if it's just my settings, but you're already in a blank document when you open Writer
| 1. Launch Writer | ||
| 1. Verify NVDA announces the Writer window title | ||
| 1. Use `ctrl+n` to create a new document | ||
| 1. Use `tab` to move through the toolbar and document area |
There was a problem hiding this comment.
This will just insert tabs into the document
There was a problem hiding this comment.
| 1. Use `tab` to move through the toolbar and document area | |
| 1. Use `f6` and `tab` to move through the toolbar and document area |
|
|
||
| #### Steps | ||
|
|
||
| 1. Type sample text |
There was a problem hiding this comment.
Is it worth mentioning that you can get some sample text to work with by typing "DT" or "LOREM" followed by f3?
| 1. Use `f7` for spell check | ||
| 1. Use `ctrl+f` to open the find bar | ||
| 1. Use `ctrl+g` to go to a page/section | ||
| 1. Use `shift+f5` to return to last edit |
There was a problem hiding this comment.
Need to save, close and reopen a document for this to work. Also doesn't seem like NVDA is aware that the caret moves in this case.
|
|
||
| #### Steps | ||
|
|
||
| 1. Use `f7` for spell check |
|
|
||
| 1. Launch Impress | ||
| 1. Verify NVDA announces the Impress window title | ||
| 1. Use `ctrl+n` to create a new presentation |
There was a problem hiding this comment.
When I first open LO Impress, I get a new slideshow wizard. Pressing Ctrl+N in this window has no effect.
| 1. Verify NVDA announces the Impress window title | ||
| 1. Use `ctrl+n` to create a new presentation | ||
| 1. Use `tab` to move through the toolbar and slide area | ||
| 1. Use arrow keys to move between slides and within slide content |
There was a problem hiding this comment.
Page up and down moves between slides. Arrow keys moves the selected object on the slide
| #### Expected Results | ||
|
|
||
| * NVDA announces window title, toolbar, and slide area | ||
| * Navigation is smooth and consistent |
There was a problem hiding this comment.
Arguably not the case at present
|
|
||
| #### Steps | ||
|
|
||
| 1. Type sample text on a slide |
There was a problem hiding this comment.
I don't think you can type directly onto a slide; you have to type into a textbox
| 1. Use `ctrl+m` to add a new slide | ||
| 1. Use `ctrl+s` to save, `ctrl+o` to open a presentation | ||
| 1. Use `f5` to start slideshow, `esc` to exit | ||
| 1. Use arrow keys to move between slides during slideshow |
There was a problem hiding this comment.
Nothing is currently reported when presenting in LOI
|
|
||
| 1. Launch Word | ||
| 1. Verify NVDA announces the Word window title | ||
| 1. Use `ctrl+n` to create a new document |
There was a problem hiding this comment.
Behaviour of this depends on "Show the Start screen when this application starts"
- Enabled (default) - the backstage view is open when you first open Word;
ctrl+ntakes you to the "New" tab. - Disabled -
ctrl+nimmediately creates and opens a new document.
| 1. Launch Word | ||
| 1. Verify NVDA announces the Word window title | ||
| 1. Use `ctrl+n` to create a new document | ||
| 1. Use `tab` to move through the ribbon and document area |
There was a problem hiding this comment.
Pressing tab in the document will just insert tabs. Movement from section to section is done with f6
|
|
||
| #### Steps | ||
|
|
||
| 1. Type sample text |
There was a problem hiding this comment.
Possibly noting that typing =rand() on its own line then pressing enter will insert some sample text.
|
|
||
| 1. Launch PowerPoint | ||
| 1. Verify NVDA announces the PowerPoint window title | ||
| 1. Use `ctrl+n` to create a new presentation |
There was a problem hiding this comment.
Moves to new section of backstage view if open, otherwise creates and opens new presentation.
| 1. Launch PowerPoint | ||
| 1. Verify NVDA announces the PowerPoint window title | ||
| 1. Use `ctrl+n` to create a new presentation | ||
| 1. Use `tab` to move through the ribbon and slide area |
There was a problem hiding this comment.
Within a slide, tab will just cycle through the objects on the slide. f6 must be used to move between sections of the window.
| 1. Use `ctrl+m` to add a new slide | ||
| 1. Use `ctrl+s` to save, `ctrl+o` to open a presentation | ||
| 1. Use `f5` to start slideshow, `esc` to exit | ||
| 1. Use arrow keys to move between slides during slideshow |
There was a problem hiding this comment.
Arrow keys don't move between slides with NVDA on; they review the contents of the slide. pageUp and pageDown, or space and backspace, or n and p should be used instead
|
|
||
| #### Expected Results | ||
|
|
||
| * NVDA announces slide content and transitions |
There was a problem hiding this comment.
What do you mean "NVDA announces [...] transitions"?
|
|
||
| 1. Launch Excel | ||
| 1. Verify NVDA announces the Excel window title | ||
| 1. Use `ctrl+n` to create a new workbook |
There was a problem hiding this comment.
New page of backstage view is shown if same is open; creates new workbook otherwise.
| 1. Verify NVDA announces the Excel window title | ||
| 1. Use `ctrl+n` to create a new workbook | ||
| 1. Use arrow keys to move between cells | ||
| 1. Use `tab` to move across cells and ribbon |
There was a problem hiding this comment.
tab only moves horizontally (across rows). f6 should be used to move between window sections.
| ## UIA Settings | ||
|
|
||
| For Word/Outlook, and Excel toggle the relevant UIA settings in Advanced Preferences. | ||
| Note: UIA support is evolving; NVDA should work optimally with the default UIA setting. |
There was a problem hiding this comment.
I'm not sure what this section is trying to communicate?
SaschaCowley
left a comment
SaschaCowley
left a comment
There was a problem hiding this comment.
I've now reviewed all manual smoke test plans.
| ### UIA | ||
|
|
||
| For Chrome and Edge, toggle each option for use UIA in Chromium in Advanced Preferences. | ||
| Note: UIA support is experimental, NVDA should work optimally with the default UIA setting. |
There was a problem hiding this comment.
What are you getting at with this section?
|
|
||
| 1. Launch the browser | ||
| 1. Verify NVDA announces the browser window title correctly | ||
| 1. Navigate to a simple HTML page (e.g., `about:blank`) |
There was a problem hiding this comment.
Perhaps https://example.com/ would work better, since there's actually content there?
| 1. Load a W3C landmarks sample page | ||
| 1. Use arrow keys to navigate through content | ||
| 1. Use `h`, `shift+h` to navigate headings | ||
| 1. Use `l`, `shift+l` to navigate links |
There was a problem hiding this comment.
| 1. Use `l`, `shift+l` to navigate links | |
| 1. Use `k`, `shift+k` to navigate links |
| 1. Use `l`, `shift+l` to navigate links | ||
| 1. Use `d`, `shift+d` to navigate landmarks | ||
| 1. Use `f`, `shift+f` to navigate form fields | ||
| 1. Use `t`, `shift+t` to navigate tables |
There was a problem hiding this comment.
None of these pages seem to have any tables.
| #### Preconditions | ||
|
|
||
| * Browser is running with a W3C sample or custom page that demonstrates dynamic content updates (e.g., ARIA live regions, notifications, chat messages, updating counters, etc.) | ||
| * NVDA is running in browse mode and focus mode as appropriate |
There was a problem hiding this comment.
I'd probably add "Report dynamic content changes" needs to be on.
| 1. Trigger an assertive live region update | ||
| 1. Load a page with dynamic notifications (e.g., ARIA alert role) | ||
| 1. Trigger a notification (e.g., form validation error, chat message, etc.) | ||
| 1. Load a page with dynamic content insertion/removal (e.g., adding/removing list items, updating counters) |
There was a problem hiding this comment.
Would probably help to get some example pages or links for these.
|
I've moved the VS Code manual test plan into a PR for system tests and manual tests: #19208 I plan to also create:
LibreOffice and Notepad++ testing should also be developed to avoid a regression surrounding #19204 (comment) See also: #18762 Manual test for plans MS Office and JetBrains will remain in this PR without automated testing |
<!-- Please read and fill in the following template, for an explanation of the sections see: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. Please initially open PRs as a draft. When you would like a review, mark the PR as "ready for review". See https://github.com/nvaccess/nvda/blob/master/.github/CONTRIBUTING.md. --> ### Link to issue number: Part of #18762 Split from #18419 ### Summary of the issue: We should explain our system tests to test against more applications. Starting with open source applications that are easily available within GitHub actions runners, ### Description of user facing changes: None ### Description of developer facing changes: None ### Description of development approach: Perform manual tests and implement equivalent system tests where possible ### Testing strategy: See changes ### Known issues with pull request: ### Code Review Checklist: <!-- This checklist is a reminder of things commonly forgotten in a new PR. Authors, please do a self-review of this pull-request. Check items to confirm you have thought about the relevance of the item. Where items are missing (eg unit / system tests), please explain in the PR. To check an item `- [ ]` becomes `- [x]`, note spacing. You can also check the checkboxes after the PR is created. A detailed explanation of this checklist is available here: https://github.com/nvaccess/nvda/blob/master/projectDocs/dev/githubPullRequestTemplateExplanationAndExamples.md#code-review-checklist --> - [x] Documentation: - Change log entry - User Documentation - Developer / Technical Documentation - Context sensitive help for GUI changes - [x] Testing: - Unit tests - System (end to end) tests - Manual testing - [x] UX of all users considered: - Speech - Braille - Low Vision - Different web browsers - Localization in other languages / culture than English - [x] API is compatible with existing add-ons. - [x] Security precautions taken. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Sascha Cowley <16543535+SaschaCowley@users.noreply.github.com>
Part of #16304
Creates smoke test plans for MS office, Libre Office, IDEs and Web browsers to help test against regressions when migrating to 64bit