Skip to content

Fix problems with Braille output devices #1396

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dpvc merged 5 commits into from
Dec 11, 2025
Merged

Fix problems with Braille output devices #1396

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ab6a5c9
Make sure the Braille for the top-level expression is transfered to t…
dpvc Nov 17, 2025
733a5cf
Hide brailleSpeech menu option and add information about Braille outp…
dpvc Nov 19, 2025
4ef32a6
Remove commented-out code
dpvc Dec 7, 2025
d89a4c0
Add help information (again), hide the replace-speech menu (again), …
dpvc Dec 11, 2025
6663405
Add 'b' key to toggle the Braille 'Combine with Speech' option
dpvc Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions ts/a11y/explorer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ export interface ExplorerMathItem extends HTMLMATHITEM {
*/
none: string;

/**
* The string to use for when there is no Braille description;
*/
brailleNone: string;

/**
* The Explorer objects for this math item
*/
Expand Down Expand Up @@ -138,6 +143,11 @@ export function ExplorerMathItemMixin<B extends Constructor<HTMLMATHITEM>>(
*/
protected static none: string = '\u0091';

/**
* Braille decription to use when set to none
*/
protected static brailleNone: string = '\u2800';

public get ariaRole() {
return (this.constructor as typeof BaseClass).ariaRole;
}
Expand All @@ -153,6 +163,10 @@ export function ExplorerMathItemMixin<B extends Constructor<HTMLMATHITEM>>(
return (this.constructor as typeof BaseClass).none;
}

public get brailleNone() {
return (this.constructor as typeof BaseClass).brailleNone;
}

/**
* @override
*/
Expand Down Expand Up @@ -351,6 +365,8 @@ export function ExplorerMathDocumentMixin<
treeColoring: false, // tree color expression
viewBraille: false, // display Braille output as subtitles
voicing: false, // switch on speech output
brailleSpeech: false, // use aria-label for Braille
brailleCombine: false, // combine Braille with speech output
help: true, // include "press h for help" messages on focus
roleDescription: 'math', // the role description to use for math expressions
tabSelects: 'all', // 'all' for whole expression, 'last' for last explored node
Expand Down
138 changes: 110 additions & 28 deletions ts/a11y/explorer/KeyExplorer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ import { InfoDialog } from '../../ui/dialog/InfoDialog.js';

/**********************************************************************/

const isWindows = context.os === 'Windows';

const BRAILLE_PADDING = Array(40).fill('\u2800').join('');

/**
* Interface for keyboard explorers. Adds the necessary keyboard events.
*
Expand Down Expand Up @@ -125,11 +129,16 @@ export class SpeechExplorer
/**
* Creates a customized help dialog
*
* @param {string} title The title to use for the message
* @param {string} select Additional ways to select the typeset math
* @returns {string} The customized message
*/
protected static helpMessage(title: string, select: string): string {
* @param {string} title The title to use for the message
* @param {string} select Additional ways to select the typeset math
* @param {string} braille Additional Braille information
* @returns {string} The customized message
*/
protected static helpMessage(
title: string,
select: string,
braille: string
): string {
return `
<h2 role="heading" aria-level="2">Exploring expressions ${title}</h2>

Expand Down Expand Up @@ -205,6 +214,10 @@ export class SpeechExplorer
<li><kbd>&lt;</kbd> cycles through the verbosity levels
for the current rule set.</li>

<li><kbd>b</kbd> toggles whether Braille notation is combined
with speech text for tactile Braille devices, as discussed
below.

<li><kbd>h</kbd> produces this help listing.</li>
</ul>

Expand All @@ -218,6 +231,13 @@ export class SpeechExplorer
speech and Braille will disable the expression explorer, its
highlighting, and its help icon.</p>

<p>Support for tactile Braille devices varies across screen readers,
browsers, and operative systems. If you are using a Braille output
device, you may need to select the "Combine with Speech" option in the
contextual menu's Braille submenu in order to obtain Nemeth or Euro
Braille output rather than the speech text on your Braille
device. ${braille}</p>

<p>The contextual menu also provides options for viewing or copying a
MathML version of the expression or its original source format,
creating an SVG version of the expression, and viewing various other
Expand All @@ -239,12 +259,13 @@ export class SpeechExplorer
/**
* Help for the different OS versions
*/
protected static helpData: Map<string, [string, string]> = new Map([
protected static helpData: Map<string, [string, string, string]> = new Map([
[
'MacOS',
[
'on MacOS and iOS using VoiceOver',
', or the VoiceOver arrow keys to select an expression',
'',
],
],
[
Expand All @@ -258,6 +279,8 @@ export class SpeechExplorer
the NVDA or JAWS key plus the arrow keys to explore the expression
even in browse mode, and you can use NVDA+shift+arrow keys to
navigate out of an expression that has the focus in NVDA`,
`NVDA users need to select this option, while JAWS users should be able
to get Braille output without changing this setting.`,
],
],
[
Expand All @@ -267,9 +290,10 @@ export class SpeechExplorer
`, and Orca should enter focus mode automatically. If not, use the
Orca+a key to toggle focus mode on or off. Also note that you can use
Orca+arrow keys to explore expressions even in browse mode`,
'',
],
],
['unknown', ['with a Screen Reader.', '']],
['unknown', ['with a Screen Reader.', '', '']],
]);

/*
Expand Down Expand Up @@ -304,6 +328,7 @@ export class SpeechExplorer
['p', [(explorer) => explorer.prevMark(), false]],
['u', [(explorer) => explorer.clearMarks(), false]],
['s', [(explorer) => explorer.autoVoice(), false]],
['b', [(explorer) => explorer.toggleBraille(), false]],
...[...'0123456789'].map((n) => [
n,
[(explorer: SpeechExplorer) => explorer.numberKey(parseInt(n)), false],
Expand Down Expand Up @@ -348,7 +373,18 @@ export class SpeechExplorer
* @returns {string} The string to use for no description
*/
protected get none(): string {
return this.item.none;
return this.document.options.a11y.brailleSpeech
? this.item.brailleNone
: this.item.none;
}

/**
* Shorthand for the item's "brailleNone" indicator
*
* @returns {string} The string to use for no description
*/
protected get brailleNone(): string {
return this.item.brailleNone;
}

/**
Expand Down Expand Up @@ -665,6 +701,7 @@ export class SpeechExplorer
protected escapeKey(): boolean {
this.Stop();
this.focusTop();
this.setCurrent(null);
return true;
}

Expand Down Expand Up @@ -873,6 +910,15 @@ export class SpeechExplorer
this.Update();
}

protected toggleBraille() {
const value = !this.document.options.a11y.brailleCombine;
if (this.document.menu) {
this.document.menu.menu.pool.lookup('brailleCombine').setValue(value);
} else {
this.document.options.a11y.brailleCombine = value;
}
}

/**
* Get index for cell to jump to.
*
Expand Down Expand Up @@ -1018,10 +1064,10 @@ export class SpeechExplorer
return;
}
const CLASS = this.constructor as typeof SpeechExplorer;
const [title, select] = CLASS.helpData.get(context.os);
const [title, select, braille] = CLASS.helpData.get(context.os);
InfoDialog.post({
title: 'MathJax Expression Explorer Help',
message: CLASS.helpMessage(title, select),
message: CLASS.helpMessage(title, select, braille),
node: this.node,
adaptor: this.document.adaptor,
styles: {
Expand Down Expand Up @@ -1219,7 +1265,7 @@ export class SpeechExplorer
*/
protected removeSpeech() {
if (this.speech) {
this.speech.remove();
this.unspeak(this.speech);
this.speech = null;
if (this.img) {
this.node.append(this.img);
Expand All @@ -1246,28 +1292,56 @@ export class SpeechExplorer
description: string = this.none
) {
const oldspeech = this.speech;
this.speech = document.createElement('mjx-speech');
this.speech.setAttribute('role', this.role);
this.speech.setAttribute('aria-label', speech);
this.speech.setAttribute(SemAttr.SPEECH, speech);
const speechNode = (this.speech = document.createElement('mjx-speech'));
speechNode.setAttribute('role', this.role);
speechNode.setAttribute('aria-label', speech || this.none);
speechNode.setAttribute('aria-roledescription', description || this.none);
speechNode.setAttribute(SemAttr.SPEECH, speech);
if (ssml) {
this.speech.setAttribute(SemAttr.PREFIX_SSML, ssml[0] || '');
this.speech.setAttribute(SemAttr.SPEECH_SSML, ssml[1] || '');
this.speech.setAttribute(SemAttr.POSTFIX_SSML, ssml[2] || '');
speechNode.setAttribute(SemAttr.PREFIX_SSML, ssml[0] || '');
speechNode.setAttribute(SemAttr.SPEECH_SSML, ssml[1] || '');
speechNode.setAttribute(SemAttr.POSTFIX_SSML, ssml[2] || '');
}
if (braille) {
this.speech.setAttribute('aria-braillelabel', braille);
if (this.document.options.a11y.brailleSpeech) {
speechNode.setAttribute('aria-label', braille);
speechNode.setAttribute('aria-roledescription', this.brailleNone);
}
speechNode.setAttribute('aria-braillelabel', braille);
speechNode.setAttribute('aria-brailleroledescription', this.brailleNone);
if (this.document.options.a11y.brailleCombine) {
speechNode.setAttribute(
'aria-label',
braille + BRAILLE_PADDING + speech
);
}
}
speechNode.setAttribute('tabindex', '0');
if (isWindows) {
const container = document.createElement('mjx-speech-container');
container.setAttribute('role', 'application');
container.setAttribute('aria-roledescription', this.none);
container.setAttribute('aria-brailleroledescription', this.brailleNone);
container.append(speechNode);
this.node.append(container);
speechNode.setAttribute('role', 'img');
} else {
this.node.append(speechNode);
}
this.speech.setAttribute('aria-roledescription', description);
this.speech.setAttribute('tabindex', '0');
this.node.append(this.speech);
this.focusSpeech = true;
this.speech.focus();
speechNode.focus();
this.focusSpeech = false;
this.Update();
if (oldspeech) {
setTimeout(() => oldspeech.remove(), 100);
setTimeout(() => this.unspeak(oldspeech), 100);
}
}

public unspeak(node: HTMLElement) {
if (isWindows) {
node = node.parentElement;
}
node.remove();
}

/**
Expand All @@ -1292,6 +1366,18 @@ export class SpeechExplorer
role: 'img',
'aria-roledescription': item.none,
});
const braille = container.getAttribute(SemAttr.BRAILLE);
if (braille) {
if (this.document.options.a11y.brailleSpeech) {
this.img.setAttribute('aria-label', braille);
this.img.setAttribute('aria-roledescription', this.brailleNone);
}
this.img.setAttribute('aria-braillelabel', braille);
this.img.setAttribute('aria-brailleroledescription', this.brailleNone);
if (this.document.options.a11y.brailleCombine) {
this.img.setAttribute('aria-label', braille + BRAILLE_PADDING + speech);
}
}
container.appendChild(this.img);
this.adjustAnchors();
}
Expand Down Expand Up @@ -1779,10 +1865,6 @@ export class SpeechExplorer
*/
public Stop() {
if (this.active) {
const description = this.description;
if (this.node.getAttribute('aria-roledescription') !== description) {
this.node.setAttribute('aria-roledescription', description);
}
this.node.classList.remove('mjx-explorer-active');
if (this.document.options.enableExplorerHelp) {
this.document.infoIcon.remove();
Expand Down
Loading