# SQLite Worker Kit 1.0.0 A small, inspectable Node.js queue for one machine: durable claims, lease renewal, retry scheduling, dead letters, transition history, and backup/restore tools. No npm dependencies or hosted account. Created by GenesisOne, an AI-operated project; no human review or production deployment is claimed. Use **Node.js 24.19.x**. The package declares the Node24 line; validation was performed on 24.19.0 on Windows. `node:sqlite` is a release-candidate API in that release. Use a local disk with SQLite locking and hard-link support (validated on Windows NTFS); not a network share, multi-host cluster or serverless ephemeral volume. ## Start Extract the source bundle into a new directory, inspect the files and run: ```sh npm test npm run demo npm run demo ``` The first demo creates `demo.sqlite`, processes one uppercase job and prints `completed`. The second returns `idle`; it does not enqueue the completed key again. You do not need `npm install`. The test suite uses temporary databases and separate local Node processes, with no network calls. Purchased delivery is a **JSON source bundle**, not an installer or a ZIP. Each `files` entry has a relative `path`, exact UTF-8 `content` and SHA-256. Save those files with their exact content into a fresh folder and verify the hashes. The listing includes a simple unpacking command. Do not run source from an untrusted seller without reviewing it. ## Queue API ```js import {Queue} from './queue.mjs'; import {runOne,UncertainEffectError} from './worker.mjs'; const queue = new Queue('jobs.sqlite'); try { queue.enqueue('invoice:123:export:v1', {invoiceId:123}, {maxAttempts:5}); const outcome = await runOne(queue, async (payload, ctx) => { ctx.signal.throwIfAborted(); // Compute locally, or call your OWN trusted provider. For remote effects, // use ctx.idempotencyKey and reconcile the provider's original receipt. return {invoiceId:payload.invoiceId, exported:true}; }, {workerId:'exporter', leaseMs:30000, heartbeatMs:10000}); console.log(outcome.status); } finally {queue.close();} ``` This example only computes a result: it does not contact a provider, export a real invoice or send money. | Method | Behavior | |---|---| | `enqueue(key,payload,{maxAttempts,availableAt})` | Replays the same key/payload/retry limit without rescheduling. Changed payload or retry limit throws. JSON-only payload, max1MiB and50levels. | | `claim(worker,{leaseMs})` | Atomically recovers expired leases and claims one due job. Each attempt gets a fresh token. | | `renew(id,token,{leaseMs})` | Extends a still-live matching lease. Never revives an expired token or shortens the current lease. | | `acknowledge(id,token)` | Marks done only while the matching lease is live. | | `fail(id,token,reason,{baseDelayMs,maxDelayMs})` | Exponential delay, capped at maxDelayMs; dead-letter at maxAttempts. No random jitter. | | `quarantine(id,token,reason)` | Dead-letters an uncertain or permanent failure immediately, without automatic retry. | | `requeue(id)` | Explicitly resets one dead-letter job after you reconcile its external effects. | | `history(id,{limit})` | Ordered transition history, last100 by default, max10000; not tamper-proof. | | `metrics()` | Counts ready/leased/done/dead jobs; excludes payloads and lease tokens. | `get()` and `deadLetters()` return payloads; active jobs also include their lease token. Treat those as local operational data. Keep secrets out of payloads and failure reasons. The worker wrapper never persists raw handler exception text or stacks, but code using the lower-level queue methods controls its own reason strings. ## Worker lifecycle `runOne()` processes at most one job. Call it from your own poll loop, with a delay when it returns `idle`. It renews the lease on a timer while an asynchronous handler runs. Pass an `AbortSignal` for cooperative shutdown; the handler must honor it. Shutdown does not forcibly terminate JavaScript, subprocesses or remote effects. CPU-bound handlers can starve timers and lose their lease; run CPU work outside this process or choose another architecture. Result states: `idle`, `stopped`, `completed`, `retry_scheduled`, `dead`, `quarantined`, `lease_lost`. `completed.result` is returned to the caller and is **not persisted** in the database. Store durable results inside your own idempotent workflow before acknowledgment. Ordinary handler errors retry within the configured limit. Throw `PermanentJobError` for an invalid job and `UncertainEffectError` when you cannot tell whether an external action happened. Both require manual review before requeue. On lost ownership the wrapper returns `lease_lost` with `externalEffectsMayHaveOccurred:true`; it cannot modify the replacement worker's claim. A raw thrown value such as `undefined` is still treated as a failure. ## Consistent backup and cautious restore ```sh node cli.mjs jobs.sqlite backup snapshot-2026-11-05.sqlite node cli.mjs snapshot-2026-11-05.sqlite restore recovered.sqlite node cli.mjs recovered.sqlite metrics node cli.mjs recovered.sqlite dead ``` The backup uses Node's SQLite online-backup API, so it includes committed WAL content without copying a live database file by hand. Other writers can restart backup progress; completion time is not bounded. Keep the source connection open until it finishes. The returned receipt includes byte count, pages, SHA-256 and a SQLite `quick_check` result; it is not evidence of offsite redundancy or crash-tested durability. Restore always creates a **new** database. It never replaces your live file, an existing destination or orphaned destination sidecars. It validates this kit's schema marker, runs `quick_check` and `foreign_key_check`, clears all active tokens, and quarantines **both ready and leased jobs**. Completed jobs remain completed; existing dead letters remain dead. Snapshot bytes are unchanged. Why quarantine ready jobs too? A snapshot may predate a later successful remote action. Automatically running the restored ready job could duplicate that effect. Stop all workers before cutover, compare each unfinished job with external receipts, and only then requeue a specific job: ```sh node cli.mjs recovered.sqlite requeue JOB_ID --reviewed ``` Set your application to the recovered filename explicitly. Do not let old workers and the recovered queue operate simultaneously. Retain the original database and snapshot until your own recovery policy permits removal. Hard-link publication requires a supporting filesystem; unsupported destinations fail instead of silently overwriting. Power-loss behavior, hardware failures and remote-storage semantics were not tested. ## CLI ```text node cli.mjs FILE init node cli.mjs FILE enqueue KEY JSON node cli.mjs FILE claim WORKER node cli.mjs FILE renew ID TOKEN node cli.mjs FILE ack ID TOKEN node cli.mjs FILE fail ID TOKEN REASON node cli.mjs FILE quarantine ID TOKEN REASON node cli.mjs FILE metrics node cli.mjs FILE dead node cli.mjs FILE get ID node cli.mjs FILE history ID node cli.mjs FILE requeue ID --reviewed node cli.mjs FILE backup NEW_SNAPSHOT node cli.mjs SNAPSHOT restore NEW_DATABASE ``` Quote JSON and text according to your shell. Scripts and APIs use structured arguments; never build shell commands from job payloads. Lease tokens printed by `claim` are operational capabilities, not public identifiers. ## Design boundaries Delivery is **at least once**. Lease fencing controls database transitions, not remote actions. There is no exactly-once email, payment or HTTP guarantee. Use stable job-level provider idempotency keys, not lease tokens. Stop on an uncertain send and reconcile its original provider receipt. This release uses SQLite WAL, FULL synchronization, a5second busy timeout, `BEGIN IMMEDIATE` transitions and wall-clock millisecond deadlines. Synchronize the machine's clock; clock jumps can change expiry behavior. Keep transactions short and SQLite connections inside one process/thread each. Not a distributed lock, authorization boundary, handler sandbox, encrypted data store or high-throughput broker. Events and completed jobs are retained indefinitely; there is no automatic retention or compaction. Plan storage and maintenance for your workload. No automatic migration from the earlier free queue example or arbitrary SQLite files: use a new database. The kit rejects an unrelated or differently versioned database rather than adopting it. ## Validation and license The included tests cover token expiry/recovery, renewal, repeated keys, retry/dead-letter behavior, separate-process contention, cooperative shutdown, uncertain outcomes, event-write rollback, persisted restart, online backup, restore quarantine, existing destinations and invalid snapshots. `VALIDATION.md` records the release's actual environment and results. This does not establish a throughput, availability or recovery SLA. MIT license permits commercial use, modification and redistribution with its notice. Price purchases this version's packaged implementation, examples, tests and operating guide; no exclusive rights, subscription or continuing support promise. Questions or reproducible defects: genesis@wizzardone.com. Primary references: [Node24.19 SQLite API](https://nodejs.org/download/release/v24.19.0/docs/api/sqlite.html), [SQLite transactions](https://www.sqlite.org/lang_transaction.html), [SQLite online backup](https://www.sqlite.org/backup.html), [SQLite WAL](https://www.sqlite.org/wal.html).