Skip to content

FTRS-4045 Implement Performance scripts for dos-ingest & ETL-ODS#1034

Merged
ManithaSrinivasa merged 5 commits intomainfrom
task/FTRS-4045-performance-scripts
Mar 27, 2026
Merged

FTRS-4045 Implement Performance scripts for dos-ingest & ETL-ODS#1034
ManithaSrinivasa merged 5 commits intomainfrom
task/FTRS-4045-performance-scripts

Conversation

@ManithaSrinivasa
Copy link
Copy Markdown
Contributor

…mance Makefile targets

Description

Context


Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

  • I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

Copilot AI review requested due to automatic review settings March 16, 2026 20:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds performance testing tooling under tests/performance/ to support dos-ingest CRUD load tests and ETL-ODS pipeline load/observability workflows.

Changes:

  • Introduces a new JMeter test plan for dos-ingest CRUD Organisation GET/PUT testing (DS_Test_Plan.jmx) and updates parameter loading to support per-service APIM workspaces.
  • Adds ETL-ODS performance scripts for (a) extracting real organisation IDs from DynamoDB, (b) injecting synthetic FHIR Organisation messages into the transform SQS queue, and (c) fetching/exporting CloudWatch metrics.
  • Extends the performance Makefile and READMEs with new targets and usage guidance.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/performance/scripts/etl_ods/generate_synthetic_sqs_messages.py Generates synthetic FHIR Organisation envelopes and sends them to the ETL-ODS transform SQS queue.
tests/performance/scripts/etl_ods/extract_organisation_ids.sh Extracts Organisation IDs/ODS codes from DynamoDB into a CSV used by JMeter and injection scripts.
tests/performance/scripts/etl_ods/cloudwatch_metrics.py Fetches ETL-ODS CloudWatch metrics and optionally exports CSV/JSON reports.
tests/performance/scripts/etl_ods/cloudwatch_config.py Centralises ETL-ODS resource naming conventions from ENVIRONMENT/WORKSPACE.
tests/performance/scripts/etl_ods/README.md Documents ETL-ODS load injection and metrics workflows.
tests/performance/scripts/LoadParams.groovy Makes APIM workspace derivation configurable via a service_name JMeter property.
tests/performance/README.MD Documents DS_Test_Plan usage and links to ETL-ODS tooling.
tests/performance/Makefile Adds make targets for DS CRUD tests and ETL-ODS injection/metrics/extractor invocation.
tests/performance/DS_Test_Plan.jmx New JMeter plan for dos-ingest CRUD performance testing (backend + proxy + stress modes).

Comment thread tests/performance/Makefile
Comment thread tests/performance/scripts/etl_ods/generate_synthetic_sqs_messages.py Outdated
Comment thread tests/performance/DS_Test_Plan.jmx Outdated
Comment thread tests/performance/README.MD Outdated
Comment thread tests/performance/Makefile Outdated
Comment thread tests/performance/README.MD
Comment thread tests/performance/scripts/etl_ods/extract_organisation_ids.sh Outdated
Comment thread tests/performance/scripts/etl_ods/generate_synthetic_sqs_messages.py Outdated
Comment thread tests/performance/scripts/LoadParams.groovy
Comment thread tests/performance/README.MD Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants