feat: add acronym text expansion

[marcpfuller]

Issue

N/A

Description of the Change

This PR introduces a comprehensive acronym expansion system for the TipTap rich text editor with database-backed management. The feature enables users to automatically expand security and technology acronyms while writing reports.

Key Components:

1. Database Model & Admin Interface:

   • New Acronym model with fields: acronym, expansion, priority, override_builtin, is_active, created_by
   • Django admin interface for CRUD operations with bulk actions (activate/deactivate)
   • Database indexes for query optimization on common lookups

2. YAML Bulk Import:

   • Admin panel upload interface at /admin/reporting/acronym/upload-yaml/
   • Validates YAML structure and safely imports acronyms
   • Override mode to replace existing definitions
   • Staff/superuser authentication required

3. GraphQL API Endpoint:

   • Hasura action /api/getAcronyms with JWT authentication
   • Filtering by acronym (case-insensitive), is_active
   • Priority-based ordering with optional pagination
   • Returns JSON array of acronym definitions

4. Frontend Integration:

   • Hybrid approach: Fetches from database with fallback to bundled YAML (49 default acronyms)
   • Keyboard shortcut (Cmd/Ctrl+E) and menu button to trigger expansion
   • Auto-expands single-definition acronyms immediately
   • Highlights multi-definition acronyms with yellow background - click to see modal with all options
   • Graceful error handling with automatic fallback to offline data

5. Data Migration:

   • Migration 0066 loads 49 default security/technology acronyms from bundled YAML
   • Skipped during test runs to avoid pollution

Technical Implementation:

• TDD approach with 58 comprehensive tests (100% passing)
• ProseMirror decorations for highlighting ambiguous acronyms
• Click handlers for modal selection
• API client with error handling and fallback logic
• Code formatted with Black, isort, Prettier
• TypeScript strict mode compliance

Alternate Designs

Considered Alternatives:

1. Static YAML only (no database): Rejected - inflexible for customer customization, requires rebuild for changes

2. REST ViewSet instead of Hasura action: Rejected - inconsistent with Ghostwriter's GraphQL-first architecture

3. Always show modal for all acronyms: Rejected - poor UX for common single-definition acronyms (API, XSS, etc.)

4. Client-scoped acronyms: Deferred - adds complexity, current global scope sufficient for v1

Possible Drawbacks

1. Network dependency: Editor requires API call to fetch custom acronyms on load (mitigated by bundled YAML fallback and caching in React state)

2. Migration overhead: New databases get 49 default acronyms loaded automatically (adds ~2-3 seconds to initial migration)

3. Admin-only upload: Only staff/superusers can bulk import acronyms (intentional security choice, but limits self-service for non-admin users)

4. Performance: Scanning entire document for acronyms on Cmd+E could be slow for very large documents (not observed in testing, but theoretical concern)

5. No collaborative filtering: All users see same acronym set - no per-user or per-project customization yet

Verification Process

Testing performed:

1. Backend Tests (58 tests, 0.584s):

   docker compose -f local.yml run django coverage run manage.py test ghostwriter.reporting.tests.test_acronym_model
   docker compose -f local.yml run django coverage run manage.py test ghostwriter.reporting.tests.test_acronym_admin
   docker compose -f local.yml run django coverage run manage.py test ghostwriter.reporting.tests.test_acronym_yaml
   docker compose -f local.yml run django coverage run manage.py test ghostwriter.api.tests.test_acronym_api

   • Model: 11 tests (CRUD, ordering, soft delete, timestamps)
   • Admin: 13 tests (list display, filters, search, bulk actions, permissions)
   • YAML: 20 tests (upload, validation, override mode, error handling)
   • API: 14 tests (authentication, filtering, pagination, response validation)

2. Frontend Build:

   cd javascript
   npm run check  # TypeScript type checking - PASSED
   npm run build-frontend-prod  # Production build - SUCCESS (1.67s)

3. Manual UI Testing:

   • Created test report with acronyms: "The API uses XSS and CSRF protection. The CIA triad is fundamental."
   • Pressed Cmd+E: API/XSS/CSRF expanded immediately, CIA highlighted in yellow
   • Clicked highlighted CIA: Modal appeared with 2 options
   • Selected "Confidentiality, Integrity, and Availability": Text replaced correctly, highlight removed
   • Added custom acronym via admin: "TEST" → "Test Expansion String"
   • Refreshed editor: TEST acronym now available for expansion
   • Uploaded YAML with 10 acronyms: All imported successfully
   • Tested override mode: Replaced built-in definitions correctly
   • Disabled network: Fallback YAML loaded successfully

4. Migration Testing:

   docker compose -f local.yml run django python manage.py migrate
   docker compose -f local.yml run django python manage.py shell
   >>> from ghostwriter.reporting.models import Acronym
   >>> Acronym.objects.count()
   49

   Verified 49 default acronyms loaded from bundled YAML

5. Regression Testing:

   • Existing editor features (bold, italic, lists, tables) still work
   • Footnote insertion unaffected
   • Document saving/loading unchanged
   • No console errors in browser

Release Notes

Added database-backed acronym expansion system with auto-expand on Cmd/Ctrl+E: single-definition acronyms expand immediately, multi-definition acronyms show click-to-select modal. Includes admin interface for CRUD operations, bulk YAML import, and 49 default security/technology acronyms. API endpoint enables dynamic loading with offline fallback.

[Copilot]

Pull request overview

This PR introduces a comprehensive acronym expansion system for the TipTap rich text editor with database-backed management, enabling users to automatically expand security and technology acronyms via Cmd/Ctrl+E. The system features a Django admin interface for CRUD operations, YAML bulk import capability, and a GraphQL API endpoint with JWT authentication.

Changes:

• Added database model, admin interface, and API endpoint for managing acronyms with filtering, priority-based ordering, and override capabilities
• Implemented TipTap extension with keyboard shortcut (Cmd/Ctrl+E) that auto-expands single-definition acronyms and highlights multi-definition acronyms for manual selection via modal
• Created migration to load 49 default security/technology acronyms from bundled YAML file on initial setup

Reviewed changes

Copilot reviewed 32 out of 34 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
javascript/vite.config.frontend.ts	Added YAML plugin to support importing bundled acronym data
javascript/vite.config.collab_server.ts	Added YAML plugin for collab server build configuration
javascript/src/tiptap_gw/text_expansion.ts	Core TipTap extension implementing acronym detection, expansion logic, and ProseMirror decorations
javascript/src/tiptap_gw/index.ts	Registered TextExpansion extension with editor
javascript/src/tiptap_gw/footnote.tsx	Formatting updates (prettier/black) to existing footnote component
javascript/src/tiptap_gw/acronym_api.ts	API client for fetching acronyms from backend with fallback to bundled YAML
javascript/src/frontend/collab_forms/rich_text_editor/text_expansion_modal.tsx	React modal component for selecting from multiple acronym expansions
javascript/src/frontend/collab_forms/rich_text_editor/text_expansion_button.tsx	Toolbar button component for triggering acronym expansion
javascript/src/frontend/collab_forms/rich_text_editor/index.tsx	Integrated expansion modal and event handlers into editor
javascript/src/frontend/collab_forms/editor.scss	Styling for highlighted multi-definition acronyms
javascript/src/data/acronyms.yml	Default acronym definitions (49 security/technology terms)
javascript/package.json	Added @rollup/plugin-yaml dependency
ghostwriter/reporting/views2/report.py	Added AcronymYAMLUploadView for admin YAML import, formatting updates
ghostwriter/reporting/utils.py	YAML import logic with validation, override mode, and error handling
ghostwriter/reporting/urls.py	Added URL route for acronym YAML upload
ghostwriter/reporting/tests/test_acronym_yaml.py	Tests for YAML upload form, import functionality, and admin view
ghostwriter/reporting/tests/test_acronym_model.py	Tests for Acronym model CRUD operations, ordering, and filtering
ghostwriter/reporting/tests/test_acronym_admin.py	Tests for admin interface configuration and bulk actions
ghostwriter/reporting/templates/reporting/acronym_yaml_upload.html	User-facing YAML upload template
ghostwriter/reporting/templates/admin/reporting/acronym_upload_yaml.html	Admin YAML upload template
ghostwriter/reporting/templates/admin/reporting/acronym_changelist.html	Admin changelist template with upload button
ghostwriter/reporting/models.py	Added Acronym model with indexes for performance, formatting updates
ghostwriter/reporting/migrations/0066_load_default_acronyms.py	Data migration to load bundled acronyms on initial setup
ghostwriter/reporting/migrations/0064_acronym_reporting_a_acronym_f840a2_idx_and_more.py	Migration adding database indexes for acronym queries
ghostwriter/reporting/migrations/0063_acronym.py	Migration creating Acronym table
ghostwriter/reporting/management/commands/check_acronyms.py	Management command to display loaded acronyms
ghostwriter/reporting/forms.py	Added AcronymYAMLUploadForm with validation, formatting updates
ghostwriter/reporting/admin.py	Added AcronymAdmin with YAML upload, bulk actions, and fieldsets
ghostwriter/factories.py	Added AcronymFactory for testing, import ordering fix
ghostwriter/api/views.py	Added GraphqlGetAcronymsAction endpoint with filtering/pagination, formatting updates
ghostwriter/api/urls.py	Registered acronym API endpoint, import ordering updates
ghostwriter/api/tests/test_acronym_api.py	Tests for acronym API endpoint authentication, filtering, and responses

Files not reviewed (1)

• javascript/package-lock.json: Language not supported

Comments suppressed due to low confidence (1)

ghostwriter/reporting/forms.py:1

• Using f-strings inside gettext translation calls (_()) prevents proper string extraction for internationalization. The variable parts should be passed as format arguments instead: _('Acronym %s must have a list of expansions') % acronym

"""This contains all the forms used by the Reporting application."""

[codecov]

Codecov Report

❌ Patch coverage is 85.79017% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.24%. Comparing base (05eaa66) to head (24f1ca6).

Files with missing lines	Patch %	Lines
ghostwriter/reporting/views2/report.py	45.56%	43 Missing ⚠️
...reporting/migrations/0066_load_default_acronyms.py	22.72%	34 Missing ⚠️
ghostwriter/api/views.py	81.66%	11 Missing ⚠️
ghostwriter/reporting/utils.py	72.50%	11 Missing ⚠️
ghostwriter/reporting/forms.py	90.90%	4 Missing ⚠️
ghostwriter/reporting/models.py	95.00%	2 Missing ⚠️
ghostwriter/reporting/tests/test_acronym_admin.py	98.03%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #814      +/-   ##
==========================================
- Coverage   91.37%   91.24%   -0.14%     
==========================================
  Files         368      376       +8     
  Lines       20933    21565     +632     
==========================================
+ Hits        19127    19676     +549     
- Misses       1806     1889      +83     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chrismaddalena]

This text expansion appears to work very well with the defaults. I am able to press CMD+E to expand acronyms in my text. However, we need to gather feedback on how to manage the expansion and acronyms. The current behavior expands all instances of the acronym in the text, replacing the acronym with the expanded text.

Typically, technical writing calls for expansion of the first use of the acronym. For example:

ACL » Access Control List (ACL)

The writer is then free to use "ACL" for the rest of that section without spelling out "Access Control List." This implementation expands every use of ACL. That may work for some users, but others may want to keep uses of the acronym beyond the first use. I know we would likely prefer expansion to expand only the first use of the acronym and replace it the expanded text (<acronym>) like the above ACL example.

Users may want to choose between:

1. Replace all instances of the acronym with expanded text
2. Replace only the first instance of the acronym with the text + "()"

I also encountered an issue with the database changes. I created a new acronym, ACL, that overlapped with the built-in acronym. I deactivated ACL, changed priority, and checked the "Override Built-in" checkbox, but ACL kept being expanded to Access Control List. I still got the default expansion after setting my new acronym to have priority and override and disabled the built-in.

ACL continuing to expand even after I disabled it is a problem. I added a new acronym not already in the built-in list and that did not expand, so it looks like new acronyms and changes aren't being properly picked up from the database. I also tried restarting the server after making changes, but new expansions did not work.