Add Elasticsearch and S3FS wiki backends and Lucene full-text search with index management

[nmaguiar]

Motivation

• Add support for additional wiki backends (es for Elasticsearch and s3fs for syncing S3 to filesystem) so wikis can be stored/searched in more environments.
• Improve wiki search performance and scalability by integrating a Lucene-based index and using it when available.
• Protect internal index files from accidental modification or deletion and ensure the search index is rebuilt when content changes.

Description

• Accept new backend values es and s3fs in mcps/mcp-wiki.yaml and mini-a.js configuration handling and pass backend-specific options (esurl, esindex, esuser, espass).
• Implemented Lucene index and index metadata helpers in mini-a-wiki.js including _indexMeta, _isHiddenPath, _safeListPages, _getLuceneIndexPath, _ensureLucene, _rebuildLuceneIndex, and _rebuildSearchIndex to build and use a local search DB.
• Added an Elasticsearch channel-backed backend via _makeEsBackend(cfg) and an s3fs bootstrap backend via _makeS3FsBackend(cfg), and updated configure to select the appropriate backend implementation.
• Updated listing, write, delete, init and search flows to filter out hidden index files, trigger _rebuildSearchIndex() on mutations, and prefer Lucene-based searches when available, with fallbacks to scanning.

Testing

• No automated tests were executed during the rollout of these changes.

[Copilot]

Pull request overview

Adds new wiki storage backends (Elasticsearch/OpenSearch and S3-to-filesystem “s3fs”) and introduces a Lucene-backed full-text index to improve wiki search scalability, alongside documentation/CLI help updates.

Changes:

• Extend wiki configuration to support wikibackend=es and wikibackend=s3fs, wiring new backend options through CLI/MCP and docs.
• Implement ES and s3fs backends in mini-a-wiki.js, plus Lucene index creation and “hidden index file” filtering.
• Add a small test for ES row→path normalization and update wiki-related docs/help tables.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
USAGE.md	Documents new wiki flags/backends and ES parameter mapping.
README.md	Adds wiki configuration options to the main settings table.
CHEATSHEET.md	Expands wiki backend docs and adds ES usage examples/mapping table.
docs/WHATS-NEW.md	Updates wiki backend options and parameter descriptions.
mini-a.yaml	Exposes new wiki CLI args (es, s3fs) in help metadata.
mini-a-web.yaml	Updates web help text to include es/s3fs backend choices.
mini-a.js	Wires es/s3fs into runtime args → wiki manager configuration.
mcps/mcp-wiki.yaml	Extends MCP wiki server args parsing/config for es/s3fs.
mini-a-wiki.js	Implements ES + s3fs backends and Lucene index build/search logic.
tests/wiki.yaml	Adds an oJob entry to run the new ES rows→paths test.
tests/wiki.js	Adds testEsRowsToPathsSkipsUndefinedRows for ES list result sanitization.

Comments suppressed due to low confidence (1)

mini-a.js:8261

• For wikibackend="s3fs" this branch never sets a local filesystem root (cfg.root), so the s3fs bootstrap will default to "." (see _makeFsBackend default). That makes s3fs write/sync into the current working directory even if wikiroot is provided. Consider honoring args.wikiroot for s3fs (or introducing a dedicated local root option) so users can control where the S3 mirror is stored.

    if (args.wikibackend === "s3" || args.wikibackend === "s3fs") {
      cfg.bucket          = args.wikibucket
      cfg.prefix          = args.wikiprefix
      cfg.url             = args.wikiurl
      cfg.accessKey       = args.wikiaccesskey

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.