feat(ocr): serialize uploads via a persistent on-disk queue

[PhilippMundhenk]

Summary

• Today each scan spawns a backgrounded curl upload to the OCR microservice. With several scans in quick succession the uploads compete for the same egress bandwidth and per-stream throughput collapses (logs show MB/s → ~100 KB/s), which makes OCR look hung for many minutes. The OCR backend is also single-threaded, so the fan-out wins nothing.
• Replaces the inline curl blocks in scantofile-0.2.4-1.sh and scanRear.sh with an ocr_enqueue.sh helper that writes a small job file via a hidden-temp + atomic rename.
• Adds ocr_worker.sh, a single-consumer poller that claims the oldest pending job via mv pending → in_progress, increments ATTEMPTS on disk before uploading, then either deletes on success, requeues with linear backoff, or moves to failed/ after OCR_MAX_ATTEMPTS (default 5). On startup it first sweeps any stranded in_progress/ files back to pending/ so a container restart loses nothing.
• Wires the worker into runScanner.sh under a tiny bash supervisor (while true; do worker; sleep 5; done) so a crashed worker auto-restarts and inherits the existing OCR_* / FTP_* / notification env vars.

Design constraints honoured

• No inotify, no flock — coordination is rename(2) only, so the queue is safe on SMB-mounted /scans and other filesystems with limited semantics.
• Survives restarts — queue lives at /scans/.ocr_queue/{pending,in_progress,failed}, which is on the persistent volume. Recovery sweep on worker start.
• Atomic — every state transition is a single mv within the same directory tree; enqueue stages to a hidden .…tmp file before renaming into pending/.
• Retries — linear backoff (attempts * 30 s), max controlled by OCR_MAX_ATTEMPTS; ATTEMPTS is persisted before the upload so a poison job can't loop forever.
• Supervised — bash while true; sleep 5 wrapper in runScanner.sh.

Tunables (env, all optional)

Var	Default	Notes
OCR_QUEUE_DIR	/scans/.ocr_queue	Anywhere on the persistent volume
OCR_QUEUE_POLL_SECONDS	5	How often the worker checks pending/
OCR_MAX_ATTEMPTS	5	Then the job moves to failed/

Test plan

• Local smoke: enqueue script writes the expected KEY=VALUE file via .tmp + atomic rename.
• Local smoke: worker fails repeatedly against an unreachable OCR endpoint, increments ATTEMPTS, lands the job in failed/ after OCR_MAX_ATTEMPTS.
• Local smoke: orphaned in_progress/ file is recovered to pending/ on worker start.
• Container test: scan 5+ documents in quick succession, observe single in-flight upload at a time, watch queue drain.
• Container test: kill the worker mid-upload, verify supervisor relaunches and recovery moves the job back to pending/.
• Container test: restart container with items in pending/ and in_progress/, verify everything drains after startup.
• Container test: confirm post-OCR notifications (trigger_inotify, trigger_telegram, sendtoftps, REMOVE_ORIGINAL_AFTER_OCR) still fire correctly from the worker.

Notes

• This branch is based on master and is independent of fix(scanRear): guard kill against missing or stale scan_pid #84 (fix/scanrear-pid-kill-guard). Both touch scanRear.sh in non-overlapping regions so they should merge cleanly in either order.

🤖 Generated with Claude Code