Introduce check for special characters by jwasikpsnc · Pull Request #92 · iterorganization/IBEX

jwasikpsnc · 2026-06-22T09:05:53Z

Solution for #75

Added script to check frontend backend and docs for problematic unicode characters

FORBIDDEN_CHARS = {
    "\u2013": "EN DASH (–)",
    "\u2014": "EM DASH (—)",
    "\u2212": "UNICODE MINUS (−)",
    "\u2018": "LEFT SINGLE QUOTATION MARK (‘)",
    "\u2019": "RIGHT SINGLE QUOTATION MARK (’)",
    "\u201C": "LEFT DOUBLE QUOTATION MARK (“)",
    "\u201D": "RIGHT DOUBLE QUOTATION MARK (”)",
    "\u2026": "HORIZONTAL ELLIPSIS (…) ",
    "\u00A0": "NON-BREAKING SPACE (NBSP)",
    "\u200B": "ZERO WIDTH SPACE",
    "\u200C": "ZERO WIDTH NON-JOINER",
    "\u200D": "ZERO WIDTH JOINER",
    "\u2060": "WORD JOINER",
    "\uFEFF": "ZERO WIDTH NO-BREAK SPACE (BOM)",
    "\u00AD": "SOFT HYPHEN",
    "\u2022": "BULLET (•)",
    "\u00D7": "MULTIPLICATION SIGN (×)",
    "\u00F7": "DIVISION SIGN (÷)",
    "\u2190": "LEFT ARROW (←)",
    "\u2192": "RIGHT ARROW (→)",
    "\u21D2": "RIGHTWARDS DOUBLE ARROW (⇒)",
    "\u2260": "NOT EQUAL TO (≠)",
    "\u2264": "LESS-THAN OR EQUAL TO (≤)",
    "\u2265": "GREATER-THAN OR EQUAL TO (≥)",
}

Currently, the script fails with errors:

(venv) ~/Desktop/work/IBEX/backend/ci$ ./forbidden-chars-check.sh
Checking 280 files...
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/tests/style.rst:10:14: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/tests/style.rst:11:16: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/tests/syntax.rst:11:33: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/tests/syntax.rst:12:15: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/tests/syntax.rst:13:22: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/tests/e2e.rst:15:26: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/architecture.rst:12:37: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/architecture.rst:13:28: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/architecture.rst:14:36: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:15:36: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:17:39: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:27:16: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:29:13: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:31:15: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:33:14: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:39:15: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:45:18: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:47:15: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:49:25: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/developers_manual/frontend_development/frontend_development_introduction.rst:61:14: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/how_to_launch.rst:36:106: EM DASH (—)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/add_configuration.rst:7:5: LEFT DOUBLE QUOTATION MARK (“)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/add_configuration.rst:13:89: LEFT DOUBLE QUOTATION MARK (“)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/add_uri.rst:15:33: LEFT DOUBLE QUOTATION MARK (“)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/load_configuration.rst:7:58: LEFT DOUBLE QUOTATION MARK (“)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/load_configuration.rst:13:54: LEFT DOUBLE QUOTATION MARK (“)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/first_window.rst:13:76: LEFT DOUBLE QUOTATION MARK (“)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/add_configuration.rst:7:9: RIGHT DOUBLE QUOTATION MARK (”)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/add_configuration.rst:13:102: RIGHT DOUBLE QUOTATION MARK (”)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/add_uri.rst:15:45: RIGHT DOUBLE QUOTATION MARK (”)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/load_configuration.rst:7:63: RIGHT DOUBLE QUOTATION MARK (”)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/load_configuration.rst:13:59: RIGHT DOUBLE QUOTATION MARK (”)
/home/ITER/wasikj/Desktop/work/IBEX/docs/source/users_manual/features/first_window.rst:13:107: RIGHT DOUBLE QUOTATION MARK (”)

❌ Forbidden Unicode characters found in 3 file(s).

I made changes in .github/workflows/frontend-linting.yml , so i request your review @mfiaudrin-epsyl

@olivhoenen @SimonPinches @prasad-sawantdesai
Are you happy with the set of forbidden characters? Should we add more? Should we have less of them?

jwasikpsnc · 2026-06-22T09:06:52Z

There is script's help page:

(venv) ~/Desktop/work/IBEX/backend/ci$ ./forbidden-chars-check.sh --help
Usage: forbidden-chars-check.sh [directories...] [--ext extensions...]

Scan source files for forbidden Unicode characters.

Arguments:
  directories    One or more directories to scan (default: ../ and ../../frontend)
  --ext          File extensions to check (default: all files)

Examples:
  forbidden-chars-check.sh                                          Scan backend/ and frontend/
  forbidden-chars-check.sh backend/ --ext .py                       Scan only .py files in backend/
  forbidden-chars-check.sh frontend/ --ext .ts .tsx .js             Scan only TS/JS files in frontend/
  forbidden-chars-check.sh src/ docs/ --ext .py .md                 Scan custom dirs with custom extensions

read-the-docs-community · 2026-06-22T09:07:09Z

Documentation build overview

📚 imas-ibex | 🛠️ Build #33248432 | 📁 Comparing 41e9113 against latest (9fe753c)

🔍 Preview build

48 files changed · ± 48 modified

± Modified

olivhoenen · 2026-06-22T11:57:21Z

+        run: npm run lint
+
+      - name: Check for forbidden Unicode characters
+        run: bash backend/ci/forbidden_chars_check.sh frontend/ --ext .ts .tsx .js .jsx .json .html .css


Forgot to commit the script?

Not sure what the script is doing, but would this be captured by using RUFF preview rules like RUF001-3 (maybe in preview)

Yeah, the script was in my .gitignore....
I will check tomorrow if we can use RUF001 for /docs and /frontend (.rst, .js etc...).
For now, short answer i get from the google is "no"

Introduce check for special characters

d76dd18

jwasikpsnc requested a review from mfiaudrin-epsyl June 22, 2026 09:05

jwasikpsnc requested review from SimonPinches, olivhoenen and prasad-sawantdesai as code owners June 22, 2026 09:05

Rename script

60cafde

olivhoenen reviewed Jun 22, 2026

View reviewed changes

Add missing script

41e9113

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce check for special characters#92

Introduce check for special characters#92
jwasikpsnc wants to merge 3 commits into
iterorganization:developfrom
jwasikpsnc:feature/check_for_special_characters

jwasikpsnc commented Jun 22, 2026

Uh oh!

jwasikpsnc commented Jun 22, 2026 •

edited

Loading

Uh oh!

read-the-docs-community Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

olivhoenen Jun 22, 2026

Uh oh!

olivhoenen Jun 22, 2026

Uh oh!

jwasikpsnc Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jwasikpsnc commented Jun 22, 2026

Uh oh!

jwasikpsnc commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

read-the-docs-community Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

olivhoenen Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

olivhoenen Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

jwasikpsnc Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jwasikpsnc commented Jun 22, 2026 •

edited

Loading

read-the-docs-community Bot commented Jun 22, 2026 •

edited

Loading