Skip to content

Fails to update state file #98

@Hu6li

Description

@Hu6li

HTTP 429 during state refresh after creating many resources causes missing state + duplicate create on next apply

Description

When applying many resources (example: many CustomDetection rules) in a single terraform apply, the provider successfully creates the rules in Microsoft Graph, but then hits HTTP 429 Too Many Requests during the subsequent read/refresh while updating Terraform state.

Because the refresh fails, the provider appears to treat the resource as “not created” and does not record it in state. On the next terraform apply, Terraform attempts to create the same rule(s) again, but the API call fails because the rule already exists.

This leads to drift and a broken apply loop:

Apply creates rules successfully
Refresh/state update fails with 429
State is missing some/all created rules
Next apply tries to create again → conflicts because resources already exist

Expected behavior

After a successful create, the provider should reliably converge state even if Microsoft Graph throttles reads/refresh calls:

implement retry/backoff handling for 429 (respecting Retry-After if provided)
and/or ensure “create succeeded” isn’t discarded due to a throttled follow-up read
ideally avoid repeated create attempts for already-created resources

Actual behavior

Resources are created in the backend
Refresh/state update fails with HTTP 429
Provider does not add resources to state
Next apply tries to create again and fails because the resources already exist

Steps to reproduce

Define a configuration that creates many CustomDetection rules (or a similar resource type) in one run. (API Calls must be more than 1500 per hour to hit quota)
Run terraform apply.
Observe:
    creates succeed
    provider hits HTTP 429 during state refresh/update
    some resources are missing from state
Run terraform apply again.
Observe duplicate create attempts fail because the rule already exists.

Logs / error output

{"@level":"error","@message":"Error: Error","@module":"terraform.ui","@timestamp":"2026-02-24T15:36:42.096034+01:00","diagnostic":{"severity":"error","summary":"Error","detail":"waiting for creation of security/rules/detectionRules: retrieving resource: GET https://graph.microsoft.com/beta/security/rules/detectionRules/1521\n--------------------------------------------------------------------------------\nRESPONSE 429: 429 Too Many Requests\nERROR CODE: TooManyRequests\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"TooManyRequests\",\n    \"message\": \"API calls quota exceeded! Maximum allowed 1500 per 01:00:00 for the key Destination+TenantID+AppID. You can send requests again in 717 seconds.\",\n    \"innerError\": {\n      \"date\": \"2026-02-24T13:58:04\",\n      \"request-id\": \"1eba724d-0eba-4b76-9bb0-11364b11fa89\",\n      \"client-request-id\": \"1eba724d-0eba-4b76-9bb0-11364b11fa89\"\n    }\n  }\n}\n--------------------------------------------------------------------------------\n"


{"@level":"error","@message":"Error: Failed to create resource","@module":"terraform.ui","@timestamp":"2026-02-25T10:23:25.539787+01:00","diagnostic":{"severity":"error","summary":"Failed to create resource","detail":"POST https://graph.microsoft.com/beta/security/rules/detectionRules\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Custom detection rule with same name/title already exists.\",\n    

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions