Skip to content

Add a caller-owned descriptor native-lock adapter #608

Description

@mmashwani

Severity: Enhancement (callers with hardened open policy cannot use filelock as a narrow native-lock primitive without giving filelock descriptor lifecycle ownership)

Summary

I would like filelock to provide a low-level native-lock adapter that operates on
a caller-owned descriptor or file object. The caller would open the lock file
with its own path, permission, symlink, sharing, and durability policy, then ask
filelock only to acquire and release the native OS lock on that already-open
resource.

This is intentionally separate from the opener request in issue #592 and PR
#595. An opener callback lets a caller customize descriptor creation, but the
resulting descriptor still enters filelock's high-level object lifecycle.
Callers that already own secure opening, path identity, holder diagnostics,
multi-target ordering, and cleanup policy need a narrower API boundary.

Environment

Reproduction

  1. Open a lock file with application-owned hardening, for example non-following
    path traversal, a prevalidated parent directory, private permissions, and a
    stable descriptor that the application will close itself.
  2. Try to ask filelock to perform only the native lock and unlock operations on
    that descriptor.
  3. Observe that the public API accepts a path and constructs a lock object that
    owns acquisition, polling, descriptor storage, release, close, cleanup, and,
    where applicable, fallback policy.
  4. Even with the opener shape proposed in PR ✨ feat(api): add optional lock-file opener hook #595, the caller can influence
    descriptor creation but cannot keep descriptor lifecycle and policy ownership
    outside the high-level lock object.

Observed vs expected

Observed:

Using filelock for native locking requires adopting filelock's high-level lock
object lifecycle or duplicating private backend code.

Expected:

Callers that already own descriptor creation and lifecycle can use filelock as a
narrow native lock/unlock adapter without path opening, unlinking, fallback, or
descriptor close policy being owned by filelock.

Current source behavior (verified in source, 3.29.7)

BaseFileLock.acquire()
returns an AcquireReturnProxy and documents the public acquisition surface as
the high-level lock-object context.

AcquireReturnProxy
returns the lock object, not a borrowed file object, descriptor, or
descriptor-backed native-lock lease.

FileLockContext.lock_file_fd
stores the native descriptor inside the private context once acquired.

BaseFileLock.release()
then delegates release through the same object-owned lifecycle.

PR #595 is related but not a duplicate. Its proposed FileOpener(path, flags, mode) -> fd contract controls how the descriptor is opened. It does not expose
an adapter that locks a descriptor whose lifecycle remains caller-owned.

Suggested fix

Add a low-level native adapter with a narrow contract. Possible shapes include:

  1. Functions such as lock_descriptor(fd, *, blocking=True) and
    unlock_descriptor(fd).
  2. A descriptor-backed context manager such as
    NativeDescriptorLock(fd, *, owns_descriptor=False).
  3. A platform-aware equivalent that accepts a Python file object, a POSIX file
    descriptor, or a Windows descriptor or handle according to what maintainers
    can support cleanly.

The important invariants are:

  1. The caller opens the descriptor and remains responsible for closing it unless
    an explicit owns_descriptor=True style option is provided.
  2. The adapter does not open, truncate, chmod, unlink, canonicalize, or break a
    lock path.
  3. The low-level adapter is native-only and fails closed when native locking is
    unavailable. Any soft-lock fallback belongs in a separate path-aware wrapper
    because SoftFileLock is an existence-lock protocol, not a descriptor-lock
    protocol.
  4. Native lock and unlock failures remain visible to the caller.
  5. The adapter does not participate in singleton, thread-local, lifetime, stale
    break, or recursive counter policy unless a separate wrapper explicitly adds
    those behaviors.
  6. The existing high-level FileLock API remains backward compatible.

This would let filelock serve both use cases: the current general-purpose
lock-object API, and a smaller native-lock primitive for applications that must
own file opening and lifecycle policy themselves.

Verification

  • A POSIX test opens a descriptor with os.open() and verifies that the new
    adapter locks and unlocks that descriptor without closing it.
  • A contender proves the descriptor-backed native lock blocks another native
    lock while held.
  • A release test verifies the descriptor remains open after adapter release when
    descriptor ownership remains with the caller.
  • A path-side-effect test verifies the adapter does not create, truncate,
    chmod, unlink, or otherwise mutate a path.
  • A backend-unavailable test verifies native-only fail-closed behavior. A
    soft-lock fallback is not part of the descriptor-only adapter contract.
  • Windows coverage either verifies the documented descriptor or handle contract
    or states that the adapter is POSIX-only until Windows support is added.

I did not find an existing issue or PR covering this exact low-level adapter.
The closest related work is issue #592 and PR #595, which cover caller-controlled
opening rather than caller-owned descriptor lifecycle.

Current workaround

There is no true public workaround for a caller-owned descriptor native-lock
adapter today. Approximate options are:

  1. Use a high-level FileLock and rely on private _context.lock_file_fd only
    for post-acquisition inspection, which does not provide caller-owned opening
    or lifecycle.
  2. Use an opener hook if PR ✨ feat(api): add optional lock-file opener hook #595 lands, while still accepting filelock's
    descriptor lifecycle.
  3. Copy the native backend lock/unlock code into the application.

Each option either relies on private internals, adopts the high-level filelock
lifecycle, or duplicates behavior that filelock already owns.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions