Skip to content

Upgrading queues from 6.x

Draining a queue's 6.x jobs before 7.0 takes over, and the calls a producer changes.

6.x kept a queue’s jobs at the root of the keyspace, under the queue’s bare name. 7.0 keeps them on a Redis stream under nestrs:queue:{<queue>} and reads nothing 6.x left behind, so drain each queue with its 6.x workers before 7.0 workers take over.

  1. Deploy the 7.0 producers. They push under nestrs:queue:{<queue>}, which a 6.x worker never reads, so what they push waits for the last step.

  2. Keep the 6.x workers running until the queue’s 6.x keys are empty — the waiting list <queue>:active, the schedule <queue>:scheduled and the in-flight sets <queue>:inflight:*.

  3. Replace the 6.x workers with 7.0 workers. A 7.0 worker’s #[processor] hosts bring QueueModule from nest_rs::queue, which attaches the worker where 6.x imported RedisWorkerModule; the root writes QueueModule::for_root(cfg) only to pin the drain window.

The producer’s calls follow the port, which now mints each job’s id and answers with a receipt; the worker is the port’s, and a consumer’s retries follow its budget:

6.x7.0
queue.push_to::<AudioQueue>(job) on the injected producerjobs.push(AudioQueue, job) on the injected Arc<Jobs>, answering a PushReceipt; push_with(AudioQueue, job, options) for a delay or a unique key
queue.push("audio", job)the queue’s marker, as above, declared on this side too for a queue another deployment drains
queue.push_json(name, value)a #[queue(name = …, job = …)] marker naming that queue and the payload both sides agree on, then jobs.push(Marker, job)
<AudioQueue as QueueName>::NAME<AudioQueue as Queue>::NAME — QueueName is now the checked name a backend receives
RedisWorkerModule::for_root(cfg)QueueModule from nest_rs::queue, imported bare by each queue adapter (QueueModule::for_root(cfg) at the root to pin it), beside RedisQueueModule, which now binds the consumer too
RedisWorkerConfig { shutdown_timeout }QueueConfig { shutdown_timeout }; the lease is RedisQueueModule::for_root(RedisQueueConfig { lease })
RedisWorker::new(), spawned in a testQueueWorker::new(), from nest_rs::queue
<PREFIX>_REDIS__WORKER__SHUTDOWN_TIMEOUT_SECS<PREFIX>_QUEUE__SHUTDOWN_TIMEOUT_SECS; the lease is <PREFIX>_REDIS__QUEUE__LEASE_SECS — see Configuration
concurrency refused on #[process]concurrency = N, per method and per replica — see Concurrency and scaling
a failed attempt re-run at once, in the workerre-run after the port’s backoff, held back off the stream until due — see Retries and failure
an attempt whose process died re-run at the same attempt, for freestill re-run at the same attempt, with no retry spent — but past three deliveries that ended without an answer the job is dead-lettered without running; see the stall limit
a job a newer release pushed failed by an older worker, spending its retrieshanded back as stored, due a minute later, with no attempt spent — and dead-lettered once it has waited a day unread, naming both versions
a handler that relied on a job running oncean idempotent one: delivery is at least once — a second delivery comes once a lease lapses, and the fence drops what the first one writes after that
a worker draining for 30 seconds20 by default, the hand-back reserve included — see Shutdown
push_many failing with the error aloneQueueError::PartiallyQueued once the backend accepted part of it, carrying the receipts of the jobs already queued
QueueError::Serialize(e) read as serde’s error, e.line() or e.classify()a nest_rs::core::DecodeError, whose sentence says what failed and, for a text serde read, where it stopped (at line L column C), never the value serde quoted; the port never locates it, so e.path() is None; a driver’s ? on a serde_json::Error converts as before
a driver sealing with envelope::seal and opening with envelope::open, and running its own workerthe port seals before enqueue and runs the worker, the attempt and the drain over the driver’s JobConsumer — see Writing a driver; Processor, check_duplicate_queue_claims, CheckpointStore::clear and the tracing re-export are gone with them

push_with takes a PushOptions: a delay, a unique key — see Producing jobs.

  • A 6.x job left at the root never runs: 7.0 does not read it. Start a 6.x worker again until the queue’s 6.x keys are empty.
  • A KEDA trigger still naming the 6.x list reads zero forever and holds the deployment at minReplicaCount. It is a redis-streams trigger on nestrs:queue:{<queue>}:jobs now — see Concurrency and scaling.
  • <PREFIX>_REDIS__WORKER__SHUTDOWN_TIMEOUT_SECS kept is read by nothing, and a worker’s boot says so at warn, variable read by nothing since 7.0, read_as naming <PREFIX>_QUEUE__SHUTDOWN_TIMEOUT_SECS. Until it is renamed, the drain window is what for_root pins, or the 20-second default.