fix(search): guard active-index deletion and gate ES/OS delete cascade (#35640)

[fabrizzio-dotCMS]

Problem

QA on #35640 (epic #35476, TC-016–020) surfaced defects in the index delete path (DELETE /api/v1/esindex/{name} → ContentletIndexAPIImpl.delete()):

• TC-018 (High): deleting the currently active index was possible via REST/AJAX. The maintenance UI only hides the Delete option for active indices (a client-side guard), so a direct DELETE /api/v1/esindex/{name} bypassed it and could leave the site with zero indices.
• TC-016: the open design question of whether delete should cascade to the .os twin — undocumented and not configurable.
• TC-016/017: the endpoint returned a false 404 when given the full physical name (with cluster prefix), returned an empty 404 body, and had an inverted-sense helper (indexExists returned true when the index did not exist).

Changes

TC-018 — active-index guard (ContentletIndexAPIImpl.delete)

• Rejects deletion of any index reported active/building by the phase-aware getCurrentIndex() ∪ getNewIndex() — the same sources the maintenance UI uses to flag active/building — via DotStateException, mapped to HTTP 400 in ESIndexResource.
• Fail-closed if the active set can't be resolved.
• Bypass: FEATURE_FLAG_ALLOW_ACTIVE_INDEX_DELETE (default false).
• Reuses the phase-aware getters rather than VersionedIndices directly, because VersionedIndices is empty in Phase 0 (rows are only written once migration starts), which would leave the most common prod state unprotected.

TC-016 — cascade feature flag (ContentletIndexAPIImpl.delete)

• FEATURE_FLAG_INDEX_DELETE_CASCADE (default true).
• ON (default): phase-dispatched broadcast to router.writeProviders() — deletes the index and its twin (current [DEFECT] Confusing empty OS index appears in indices portlet on Phase 1 migration bootstrap #35820 behavior, no orphan).
• OFF: tag-dispatched (.os → OpenSearch, otherwise Elasticsearch) — deletes only the named engine's index, leaving the twin intact (the documented no-cascade behavior).

TC-016/017 — cleanups (ESIndexResource)

• Normalize the incoming name with removeClusterIdFromName in both deleteIndex and modIndex, so the endpoints accept both the short name and the full physical name (no more false 404).
• Readable 404 body instead of an empty response.
• Rename indexExists → indexDoesNotExist (matches its actual meaning).

Testing

• ContentletIndexAPIImplTest#delete_activeIndex_isRejected_unlessFeatureFlagOverrides — active index survives a rejected delete; FF bypass then allows it.
• ContentletIndexAPIImplMigrationIntegrationTest#test_delete_phase1_cascadeOff_removesOnlyNamedEngine — Phase 1, cascade OFF: bare name deletes ES only, .os twin survives (sibling to the existing cascade-ON test).
• Core compiles clean; integration module test-compiles clean; no openapi.yaml diff.

Out of scope (follow-ups)

• Cascade for site-search indices (separate subsystem, deferred in the migration doc).
• UI confirmation dialog on delete (requested by @jcastro-dotcms).

🤖 Generated with Claude Code

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request

[claude]

Claude finished @fabrizzio-dotCMS's task in 1m 58s —— View job

---

Rollback-Safety Analysis — Complete ✅

• Read docs/core/ROLLBACK_UNSAFE_CATEGORIES.md
• Get full PR diff (base 3f0c5681...de67363e)
• Analyze diff against every unsafe category
• Post verdict (label)

Verdict: ✅ Safe to Rollback — label AI: Safe To Rollback applied.

Scope reviewed:

• ContentletIndexAPIImpl.java — new assertIndexIsDeletable() guard, new FF_ALLOW_ACTIVE_INDEX_DELETE / FF_INDEX_DELETE_CASCADE flags, rewritten delete() cascade logic, new clearStorePointer/clearEsStorePointer/clearOsStorePointer helpers.
• ESIndexResource.java — removeClusterIdFromName normalization in deleteIndex/modIndex, structured 404/400 error bodies, indexExists → indexDoesNotExist rename (private method, not DB).
• Two test files — test-only, no runtime impact.

Why no category matches:

• No runonce DB migration task, no schema change (ADD/DROP/RENAME/ALTER COLUMN), no PK/constraint restructuring → rules out C-1, C-4, H-1, H-2, H-3, M-1.
• No ESMappingAPIImpl/ESMappingUtilHelper/ESMappingConstants touch, no putMapping() call, no reindex trigger → rules out C-2.
• No CURRENT_MODEL_VERSION / ImmutableContentlet / ToCurrentVersionConverter change → rules out C-3.
• No new ContentType field type, no storage-provider default change, no stored-procedure drop → rules out H-4, H-5, H-6.
• delete()'s new DB-pointer clearing (clearEsStorePointer/clearOsStorePointer) calls existing legacyIndiciesAPI.point() / versionedIndicesAPI.saveIndices() APIs that N-1 already understands — it removes stale pointers to now-deleted indices rather than introducing a new storage shape, so N-1 reads the same tables the same way.
• No viewtool (ViewTool) touched → rules out H-8.
• The REST changes (deleteIndex/modIndex) only add structured error bodies on 404/400 paths and fix an inverted existence check + false-404 on full physical names — these are additive/corrective, not a renamed/removed field or changed success-response shape that could break an already-deployed client → does not meet the M-3 bar ("renamed field", "removed type", "changed structure" that breaks a co-deployed consumer).
• No OSGi-exported interface signature changed (ContentletIndexAPI.delete(String) keeps its signature) → rules out M-4.

Net effect: this PR only changes in-process guard/cascade/error-handling logic in the index-delete path. Rolling back to N-1 simply restores the pre-fix behavior (TC-016/017/018 defects reappear) — it does not corrupt data, break mappings, or leave N-1 unable to start or serve requests.

[github-actions]

🤖 dotBot Review (Bedrock)

Reviewed 4 file(s); 3 candidate(s) → 2 confirmed, 0 uncertain (unverified, kept for review).

Confirmed findings

• 🟡 Medium dotCMS/src/main/java/com/dotcms/rest/api/v1/index/ESIndexResource.java:576 — modIndex endpoint lacks error handling for index operation exceptions
  The modIndex method's switch cases (lines 576-592) call index operations without try-catch blocks. Exceptions like ElasticsearchClientException from deactivateIndex/clearIndex/openIndex/etc would propagate as unhandled exceptions, resulting in generic 500 errors instead of structured error responses. Contrast with deleteIndex which handles DotStateException and others with specific status codes (400/404/500) and error entities.
• 🟡 Medium dotCMS/src/main/java/com/dotcms/rest/api/v1/index/ESIndexResource.java:576 — Race condition between index existence check and delete operation
  The code checks if the index exists via indexAPI.indexDoesNotExist() then proceeds to delete. However, there's no atomicity between check and deletion - another process could delete the index in between, leading to potential exceptions from trying to delete a non-existent index. While Elasticsearch/OpenSearch may handle non-existent deletes gracefully, the REST endpoint currently doesn't explicitly handle this case, which could surface 500 errors to callers.

---

_{us.deepseek.r1-v1:0 · Run: #28692793584 · tokens: in: 28891 · out: 10726 · total: 39617 · calls: 9 · est. ~$0.097}

[github-actions]

🟠 [High] 404 response in modIndex lacks error message body

The modIndex method throws NotFoundInDbException which returns an empty 404 response body, while deleteIndex uses ResponseUtil.mapErrorResponse() to return structured JSON errors. This inconsistency breaks client error handling expectations.

[github-actions]

🟠 [High] Cascade delete ignores secondary provider failures

In ContentletIndexAPIImpl.java line 1750, when cascade is enabled, the method deletes the index across all writeProviders but only checks the primary provider's result. If secondary providers (e.g. OpenSearch) fail to delete their indices, the overall operation still reports success based on the primary provider's success. This can leave orphaned indices in secondary systems while returning a successful response.

[github-actions]

🟠 [High] Unhandled exceptions in index deletion may result in 500 errors

The deleteIndex method in ESIndexResource.java only catches DotStateException and DotSecurityException. Other exceptions (e.g., IOException from Elasticsearch client, runtime errors) will propagate unchecked, resulting in 500 Internal Server Error responses. This violates REST error handling conventions and exposes internal details.

[github-actions]

🟠 [High] Unhandled exceptions in index deletion

The try-catch block in ESIndexResource.deleteIndex() only handles specific exception types (DotSecurityException, DotStateException, IllegalArgumentException). Other exceptions from idxApi.delete() like ES client errors or runtime exceptions would propagate as uncaught exceptions, resulting in raw 500 errors exposing internal details. This violates error handling best practices and could leak sensitive stack trace information to API consumers.

[github-actions]

🟠 [High] Cascade deletions may leave orphaned indices when secondary provider fails

The code performs fire-and-forget deletions on secondary providers after primary ES deletion. When FEATURE_FLAG_INDEX_DELETE_CASCADE=true, failure in OpenSearch deletion leaves orphaned index while API returns success. Current implementation only returns result from first provider deletion (line 1733), not verifying cascade completions.