Skip to content

LocalFileIdentifiableStore: add per-object write locking to prevent silent data loss under concurrent writes #554

@s-heppner

Description

@s-heppner

LocalFileIdentifiableStore has no concurrency control across processes. Under a multi-worker WSGI deployment, two workers handling concurrent requests that modify the same object will each call commit(), write to separate temp files, and then race on os.replace(). The last writer wins silently (no exception, no conflict signal, no log entry). The earlier write is permanently lost.

CouchDB avoids this via _rev/409: the caller is forced to detect and resolve the conflict. The local file path has no equivalent mechanism.

I suggest to take an advisory lock on a stable sibling lockfile (<hash>.lock) for the duration of the write in _write_atomic(). A sibling lockfile is preferred over locking the JSON file itself because os.replace() swaps the inode, so a lock held on the old file does not carry over to the replacement.

fcntl.flock is sufficient for single-host deployments (the local file backend implies a single host by design) and is safe under uWSGI multi-worker mode. It should be wrapped in a try/finally to guarantee the lock is released even on exception or KeyboardInterrupt.

This issue has been found by @zrgt during review of #553.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementEnhancement of an existing featuresdkSomething to do with the `sdk` package

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions