Skip to content

Commit 62ff3cf

Browse files
adwitiyasushantadwitiyasushant
committed
feat(adms): add SAP ADMS module with sync/async clients and BDD tests
Adds a full-featured ADMS (Attachment Document Management Service) module to the SDK with sync and async clients, IAS X.509 token authentication, OData V4 service support (DocumentService, ConfigurationService, AdminService), and pytest-bdd integration tests with Gherkin scenarios. Also adds shared SDK building blocks consumed by ADMS: IAS token fetcher, mTLS support, async HTTP client, and the ADMS telemetry module entry.
1 parent 500b699 commit 62ff3cf

64 files changed

Lines changed: 11663 additions & 19 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/integration-tests.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,20 @@ jobs:
5656
echo "Set variable: $var_name"
5757
done
5858
59+
# Integration service URLs from secrets/variables
60+
ADMS_URL=$(echo '${{ toJSON(secrets) }}' | jq -r '.CLOUD_SDK_ADMS_INTEGRATION_URL // empty')
61+
if [ -z "$ADMS_URL" ]; then
62+
ADMS_URL=$(echo '${{ toJSON(vars) }}' | jq -r '.CLOUD_SDK_ADMS_INTEGRATION_URL // empty')
63+
fi
64+
if [ -n "$ADMS_URL" ]; then
65+
echo "CLOUD_SDK_ADMS_INTEGRATION_URL=$ADMS_URL" >> $GITHUB_ENV
66+
echo "Set: CLOUD_SDK_ADMS_INTEGRATION_URL"
67+
else
68+
# Skip ADMS integration tests when HDM service credentials are not configured
69+
echo "CLOUD_SDK_ADMS_SKIP_IF_UNAVAILABLE=true" >> $GITHUB_ENV
70+
echo "ADMS service URL not configured — ADMS integration tests will be skipped"
71+
fi
72+
5973
echo "Environment setup complete - automatically configured all CLOUD_SDK_CFG_* environment variables and secrets"
6074
6175
- name: Run integration tests

‎.gitignore‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,4 +38,12 @@ mocks/
3838

3939
# Generated files
4040
PULL_REQUEST.md
41-
RELEASE.md
41+
42+
# macOS metadata
43+
.DS_Store
44+
45+
# UCL provisioning artefacts (separate repo concern)
46+
.ucl-provision/
47+
src/sap_cloud_sdk/adms/ucl/
48+
RELEASE.md
49+
.env.adms

‎docs/INTEGRATION_TESTS_ADMS.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# DMS Integration Tests
2+
3+
End-to-end tests that verify the `sap_cloud_sdk.adms` module is correctly wired to a running **SAP Advanced Document Management (ADM / HDM)** server.
4+
5+
## Two modes
6+
7+
| Mode | When to use | What runs |
8+
|---|---|---|
9+
| **Local auto-start** | Day-to-day development | Starts `hdm/srv` via `mvn spring-boot:run` with H2 + security disabled |
10+
| **External / BTP** | CI pipelines, acceptance tests | Points to a deployed ADM instance using real IAS credentials |
11+
12+
---
13+
14+
## Prerequisites
15+
16+
### Local mode
17+
- Java 21 and Maven 3.9+ on `PATH`
18+
- The `hdm` repo checked out at the same level as `cloud-sdk-python` (i.e. `../hdm`), **or** `CLOUD_SDK_HDM_DIR` set to its path
19+
- No external services needed — H2 in-memory DB, mocked storage & virus scanner
20+
21+
### External / BTP mode
22+
- A provisioned ADM instance
23+
- IAS service binding credentials
24+
25+
---
26+
27+
## Running the tests
28+
29+
### Local mode (auto-starts HDM)
30+
31+
```bash
32+
cd /path/to/cloud-sdk-python
33+
34+
# Run all integration tests — HDM will start automatically
35+
.venv/bin/python -m pytest tests/adms/integration/ -m integration -v
36+
37+
# Skip if HDM can't start (e.g. Java not available in this env)
38+
CLOUD_SDK_ADMS_SKIP_IF_UNAVAILABLE=true \
39+
.venv/bin/python -m pytest tests/adms/integration/ -m integration -v
40+
```
41+
42+
HDM startup takes ~30–60 seconds on first run. The server is kept alive for the entire pytest session and killed at the end.
43+
44+
### External / BTP mode
45+
46+
```bash
47+
export CLOUD_SDK_ADMS_INTEGRATION_URL=https://your-adm.cfapps.eu20.hana.ondemand.com
48+
export CLOUD_SDK_CFG_ADMS_DEFAULT_SERVICE_URL=$CLOUD_SDK_ADMS_INTEGRATION_URL
49+
export CLOUD_SDK_CFG_ADMS_DEFAULT_IAS_URL=https://your-tenant.accounts.ondemand.com
50+
export CLOUD_SDK_CFG_ADMS_DEFAULT_CLIENT_ID=...
51+
export CLOUD_SDK_CFG_ADMS_DEFAULT_CLIENT_SECRET=...
52+
53+
.venv/bin/python -m pytest tests/adms/integration/ -m integration -v
54+
```
55+
56+
### Run a specific test file
57+
58+
```bash
59+
# Document lifecycle only
60+
.venv/bin/python -m pytest tests/adms/integration/test_e2e_document_flow.py -m integration -v
61+
62+
# Async client only
63+
.venv/bin/python -m pytest tests/adms/integration/test_e2e_async_flow.py -m integration -v
64+
65+
# SPII handler (no server needed — runs SpiiHandler logic directly)
66+
.venv/bin/python -m pytest tests/adms/integration/test_e2e_spii_flow.py -m integration -v
67+
```
68+
69+
### Run unit tests only (no server)
70+
71+
```bash
72+
.venv/bin/python -m pytest tests/adms/unit/ -v
73+
```
74+
75+
---
76+
77+
## Environment variables reference
78+
79+
| Variable | Default | Description |
80+
|---|---|---|
81+
| `CLOUD_SDK_ADMS_INTEGRATION_URL` | _(unset)_ | External ADM URL; if set, skips local HDM auto-start |
82+
| `CLOUD_SDK_HDM_DIR` | `../hdm` | Path to the HDM repo root (local mode) |
83+
| `CLOUD_SDK_HDM_PORT` | `18080` | Port for the locally started HDM server |
84+
| `CLOUD_SDK_ADMS_SKIP_IF_UNAVAILABLE` | `false` | Skip (not fail) if the server cannot be reached |
85+
86+
---
87+
88+
## Test files
89+
90+
| File | What it tests |
91+
|---|---|
92+
| [conftest.py](conftest.py) | Session fixtures: start HDM, `AdmsClient`, `AsyncAdmsClient`, `bo_type_id` |
93+
| [test_e2e_document_flow.py](test_e2e_document_flow.py) | Sync client: create → query → get → update → draft lifecycle → delete |
94+
| [test_e2e_async_flow.py](test_e2e_async_flow.py) | Async client: same operations + concurrent creates |
95+
| [test_e2e_spii_flow.py](test_e2e_spii_flow.py) | SPII handler: CONFIG_PENDING, READY, unassign, cert gate, validation |
96+
97+
---
98+
99+
## How the local HDM server is started
100+
101+
The `hdm_base_url` fixture in `conftest.py`:
102+
103+
1. Checks if `CLOUD_SDK_ADMS_INTEGRATION_URL` is set → use it directly
104+
2. Checks if port 18080 is already open → re-use the running server
105+
3. Otherwise runs:
106+
```
107+
mvn -pl srv spring-boot:run -q \
108+
-Dserver.port=18080 \
109+
-Dspring.security.enabled=false \
110+
-Dadm.redis.enabled=false
111+
```
112+
4. Polls `/actuator/health` every 3 seconds, up to 120 seconds
113+
5. At session teardown, sends `SIGTERM` to the process group
114+
115+
**Why `spring.security.enabled=false`**: HDM's integration tests use `MockMvc` which bypasses Spring Security. For real HTTP calls from Python, security must be disabled or mocked. In the default/H2 profile without IAS/XSUAA bindings, this is safe and consistent with the existing Java IT approach.
116+
117+
---
118+
119+
## What the tests verify
120+
121+
### `test_e2e_document_flow.py`
122+
1. `CreateDocumentWithRelation` returns a valid `DocumentRelation` with embedded `Document`
123+
2. `get_all()` with `$filter` returns the created relation
124+
3. `get()` by primary key returns correct fields
125+
4. Newly created document has `DocumentState = PENDING` (or CLEAN in fast-scan environments)
126+
5. `get_download_url()` raises `ScanNotCleanError` when state is PENDING
127+
6. `PATCH /Document(...)` updates name correctly
128+
7. Draft flow: `create_draft → validate_draft → activate_draft` produces active entities
129+
8. Draft discard: `create_draft → discard_draft` leaves no active entities
130+
9. `delete()` + subsequent `get()` raises `DocumentNotFoundError`
131+
132+
### `test_e2e_async_flow.py`
133+
- All of the above but via `AsyncAdmsClient` (httpx-based)
134+
- Plus: 3 concurrent `create()` calls via `asyncio.gather` — verifies connection pooling and async correctness
135+
136+
### `test_e2e_spii_flow.py`
137+
- `SpiiHandler` is exercised directly (no HTTP server needed)
138+
- Full CONFIG_PENDING → READY → UNASSIGN tenant lifecycle
139+
- Certificate verification gate blocks wrong CN
140+
- Validation rejects malformed notification payloads

‎docs/adms/patterns/delete_user_data_pattern.yaml‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
name: delete_user_data_pattern
2+
version: "1.0"
3+
description: >
4+
Start a DELETE_USER_DATA job in SAP ADM for GDPR erasure compliance.
5+
Replaces all audit-field references (created_by, changed_by) to the specified
6+
user across all Document and DocumentRelation records.
7+
Routes to AdminService — requires system-user (client_credentials) auth,
8+
NOT user-OBO auth. This pattern must never be triggered by end-user interaction
9+
without a confirmed deletion request workflow.
10+
11+
intent_keywords:
12+
- delete user data
13+
- gdpr erasure
14+
- right to be forgotten
15+
- anonymize user
16+
- erase personal data
17+
- delete user from documents
18+
- gdpr request
19+
20+
required_apis:
21+
- step: 1
22+
id: confirm_deletion
23+
api: "workflow_gate"
24+
description: >
25+
MANDATORY human-in-the-loop confirmation gate before starting erasure.
26+
Never auto-trigger DELETE_USER_DATA without explicit user confirmation.
27+
Log the confirmation event with timestamp and approver identity.
28+
security: CRITICAL
29+
depends_on: []
30+
31+
- step: 2
32+
id: start_delete_job
33+
api: "client.jobs.start_delete_user_data"
34+
description: >
35+
Submit a DELETE_USER_DATA job to **AdminService** (not DocumentService).
36+
Must use service-to-service credentials (client_credentials grant —
37+
do NOT use user_jwt for this call).
38+
input_schema:
39+
type: DeleteUserDataJobParameters
40+
required_fields:
41+
- user_id
42+
output: "JobOutput (job_id, job_status=RUNNING)"
43+
service_path: odata/v4/AdminService
44+
auth_note: >
45+
Use create_client("default") — NOT create_client("default", user_jwt=...).
46+
AdminService enforces system-level authorization.
47+
depends_on: [confirm_deletion]
48+
49+
- step: 3
50+
id: poll_job_status
51+
api: "client.jobs.get_status"
52+
description: >
53+
Poll using the job_id from step 2 until job_status.is_terminal() is True.
54+
poll_interval_seconds: 15
55+
max_polls: 20
56+
terminal_check: "output.job_status.is_terminal()"
57+
terminal_states:
58+
- COMPLETED
59+
- FAILED
60+
- CANCELLED
61+
depends_on: [start_delete_job]
62+
63+
- step: 4
64+
id: audit_log_completion
65+
api: "workflow_gate"
66+
description: >
67+
Write an audit log entry recording the completion (or failure) of the
68+
erasure job, including job_id, user_id, timestamp, and final status.
69+
Required for GDPR Article 17 compliance evidence.
70+
depends_on: [poll_job_status]
71+
72+
validation_rules:
73+
- rule: "user_id must be a non-empty string matching the IAS user principal"
74+
field: user_id
75+
- rule: "NEVER trigger this pattern without explicit confirmation from an authorized approver"
76+
security: CRITICAL
77+
- rule: "NEVER use user_jwt — AdminService requires system auth (client_credentials)"
78+
security: true
79+
- rule: "job completion MUST be audit-logged for GDPR Art. 17 compliance"
80+
- rule: "This pattern must only be available to GDPR officers / admins in AMS policy"
81+
82+
error_handling:
83+
- error: "job_status == FAILED"
84+
action: >
85+
Log failure with all details. Escalate to platform admin.
86+
Do not silently fail — GDPR erasure failures are compliance incidents.
87+
- error: HttpError on start
88+
action: Do not retry automatically — log and escalate.
89+
- error: "max_polls exceeded"
90+
action: >
91+
Log that the job is still running. Return job_id for manual follow-up.
92+
Do not assume the erasure has completed.
93+
94+
use_cases:
95+
- "GDPR Right to Erasure request workflow for a departing employee"
96+
- "Data subject access request — delete user from all ADM document audit fields"
97+
- "LangGraph workflow: GDPR deletion pipeline triggered by HR offboarding event"
98+
99+
compliance:
100+
regulation: GDPR Article 17 (Right to Erasure)
101+
evidence_required: true
102+
requires_human_approval: true

‎docs/adms/patterns/document_download_pattern.yaml‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: document_download_pattern
2+
version: "1.0"
3+
description: >
4+
Download a document from SAP ADM via a secure time-limited presigned URL.
5+
Enforces the virus scan gate — downloads are only permitted for CLEAN documents.
6+
The presigned URL must NOT be cached and must be consumed immediately.
7+
8+
intent_keywords:
9+
- download document
10+
- get file
11+
- retrieve attachment
12+
- export document
13+
- fetch document content
14+
- open document
15+
16+
required_apis:
17+
- step: 1
18+
id: check_scan_state
19+
api: "client.documents.get"
20+
description: >
21+
Fetch document metadata to inspect DocumentState before attempting download.
22+
Abort if state is not CLEAN (PENDING / INFECTED / SCAN_FAILED are blocked).
23+
input_schema:
24+
required_fields:
25+
- document_id
26+
optional_fields:
27+
- is_active_entity
28+
output: Document (contains DocumentState)
29+
depends_on: []
30+
31+
- step: 2
32+
id: get_download_url
33+
api: "client.documents.get_download_url"
34+
description: >
35+
Obtain a time-limited presigned download URL. This method enforces the
36+
ScanStatus.CLEAN gate internally — raises ScanNotCleanError if not ready.
37+
input_schema:
38+
required_fields:
39+
- document_relation_id
40+
- doc_content_version_id
41+
optional_fields:
42+
- is_active_entity
43+
output: "str — presigned URL (valid for a short time, do not cache)"
44+
depends_on: [check_scan_state]
45+
46+
- step: 3
47+
id: stream_to_caller
48+
api: "external_http_get"
49+
description: >
50+
Stream the file bytes from the presigned URL to the caller.
51+
Use streaming GET to avoid buffering large files in memory.
52+
The SDK does not buffer the download — use requests.get(url, stream=True)
53+
or httpx.AsyncClient.stream().
54+
depends_on: [get_download_url]
55+
56+
validation_rules:
57+
- rule: "DocumentState MUST equal CLEAN before presenting a download URL to the user"
58+
security: true
59+
- rule: "Presigned URL must NOT be stored in logs, databases, or chat history"
60+
security: true
61+
- rule: "doc_content_version_id must be a non-empty string"
62+
field: doc_content_version_id
63+
- rule: "Do not retry get_download_url — each call generates a new presigned URL; just use most recent"
64+
65+
error_handling:
66+
- error: ScanNotCleanError
67+
action: >
68+
Inform user that the document is not yet available for download —
69+
it may still be under virus scan (PENDING) or blocked (INFECTED / SCAN_FAILED).
70+
- error: DocumentNotFoundError
71+
action: Surface to user — the document was deleted or the ID is wrong.
72+
- error: HttpError
73+
action: Log and retry once; surface persistent failures to user.
74+
75+
use_cases:
76+
- "User requests to open an invoice attached to a Purchase Order"
77+
- "Batch export of all documents linked to a Contract"
78+
- "LangGraph node: retrieve document content for further AI processing"
79+
- "Streaming large CAD drawings from ADM to the browser"

0 commit comments

Comments
 (0)