MongoDB Atlas Search Index Replication

A demo code for replicating MongoDB Atlas Search Indexes across databases using direct REST API calls.

What This Does

Automatically replicates MongoDB Atlas Search Indexes from a source database to a destination database. Uses the official MongoDB Atlas Admin API v2 with secure HTTP Digest Authentication.

Key Features:

• ✅ Single Python script
• ✅ Direct REST API integration
• ✅ Zero intermediate files
• ✅ GitHub Actions ready
• ✅ Idempotent (safe to run multiple times)
• ✅ Detailed diff comparison before creation
• ✅ Works locally or in CI/CD

---

Quick Start

🟢 Path 1: GitHub Actions

1. Get API Credentials

1. Open MongoDB Atlas
2. Click Organization → Settings → Access Managers → API Keys
3. Click Create API Key
4. Save the Public Key and Private Key
5. Find your Project ID in Project Settings

2. Add GitHub Secrets

Go to your GitHub repository: Settings → Secrets and variables → Actions

Add these 4 secrets:

• ATLAS_PROJECT_ID = your_project_id
• ATLAS_CLUSTER_NAME = your_cluster_name
• ATLAS_CLIENT_ID = your_api_public_key
• ATLAS_CLIENT_SECRET = your_api_private_key

3. Push Code

git add .github/workflows/replicate-indexes.yml replicate_search_indexes.py
git commit -m "Add MongoDB search index replication"
git push

4. Run the Workflow

1. Go to your GitHub repository Actions tab
2. Click "Replicate MongoDB Atlas Search Indexes"
3. Click "Run workflow"
4. Fill in:
   • Source Database: dev
   • Destination Database: staging
   • Collection Filter Prefix: v1_
5. Click "Run workflow"

Done! Check the workflow logs for results.

---

🟡 Path 2: Local Testing

1. Install Python Dependency

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install requests

2. Run the Script

python3 replicate_search_indexes.py \
  --project-id YOUR_PROJECT_ID \
  --cluster-name YOUR_CLUSTER_NAME \
  --source-db dev \
  --dest-db staging \
  --api-key YOUR_API_PUBLIC_KEY \
  --api-secret YOUR_API_PRIVATE_KEY \
  --filter-prefix v1_

Example with real values:

python3 replicate_search_indexes.py \
  --project-id 0943bf111df4ff0548dc4def \
  --cluster-name Cluster0 \
  --source-db dev \
  --dest-db staging \
  --api-key abc123def456 \
  --api-secret 12345678-1234-1234-1234-123456789012 \
  --filter-prefix v1_

---

How It Works

3-Step Process

Step 1: FETCH
  GET /api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes
  ↓ Retrieves all search indexes from cluster

Step 2: FILTER
  Filters by:
  - Source database
  - Collection name prefix (default: v1_)
  ↓ Shows difference between source and destination

Step 3: CREATE
  POST /api/atlas/v2/groups/{groupId}/clusters/{clusterName}/fts/indexes
  ↓ Deletes existing index (if any)
  ↓ Creates with source definition
  ↓ Idempotent: safe to run multiple times

What Gets Replicated

✅ Included:

• MongoDB Atlas Search Indexes only (full-text search)
• All index configuration (mappings, analyzers, synonyms, etc.)
• Only collections matching filter prefix (default: v1_)
• Same collection names in destination database

❌ Excluded:

• Regular database indexes (use migration tools for those)
• System indexes (id)
• Collections not matching filter prefix
• System databases (admin, config, local)

---

Architecture

Single Python Script: replicate_search_indexes.py

├── Class: AtlasSearchIndexReplicator
│   ├── get_all_search_indexes()      → Fetch all indexes via API
│   ├── normalize_index()             → Extract mappings, analyzers, etc.
│   ├── get_destination_index()       → Check if index exists
│   ├── compare_index_definitions()   → Show differences
│   ├── delete_search_index()         → Remove old index
│   ├── create_search_index()         → Create new index (POST request)
│   ├── filter_indexes()              → Filter by source DB and prefix
│   ├── print_summary()               → Display results
│   └── run()                         → Main orchestration
│
└── Authentication: HTTP Digest Auth with API Key + Secret
    Base URL: https://cloud.mongodb.com/api/atlas/v2
    Headers: Accept: application/vnd.atlas.2024-05-30+json

---

Parameters Reference

replicate_search_indexes.py

Parameter	Required	Example	Description
--project-id	Yes	6231bf257df4ff0...	MongoDB Atlas Project ID
--cluster-name	Yes	Cluster0	Cluster name from Atlas
--source-db	Yes	dev	Source database name
--dest-db	Yes	staging	Destination database name
--api-key	Yes	abc123def456	API public key
--api-secret	Yes	12345678-1234-...	API private key
--filter-prefix	No	v1_	Collection name prefix (default: v1_)

GitHub Actions Workflow Inputs

Input	Required	Default	Description
source_db	Yes	-	Source database name
dest_db	Yes	-	Destination database name
filter_prefix	No	v1_	Collection filter prefix