Upgrading queues from 6.x
Draining a queue's 6.x jobs before 7.0 takes over, and the calls a producer changes.
6.x kept a queue’s jobs at the root of the keyspace, under the queue’s bare
name. 7.0 keeps them on a Redis stream under nestrs:queue:{<queue>} and reads
nothing 6.x left behind, so drain each queue with its 6.x workers before 7.0
workers take over.
-
Deploy the 7.0 producers. They push under
nestrs:queue:{<queue>}, which a 6.x worker never reads, so what they push waits for the last step. -
Keep the 6.x workers running until the queue’s 6.x keys are empty — the waiting list
<queue>:active, the schedule<queue>:scheduledand the in-flight sets<queue>:inflight:*. -
Replace the 6.x workers with 7.0 workers. A 7.0 worker’s
#[processor]hosts bringQueueModulefromnest_rs::queue, which attaches the worker where 6.x importedRedisWorkerModule; the root writesQueueModule::for_root(cfg)only to pin the drain window.
Your code
Section titled “Your code”The producer’s calls follow the port, which now mints each job’s id and answers with a receipt; the worker is the port’s, and a consumer’s retries follow its budget:
| 6.x | 7.0 |
|---|---|
queue.push_to::<AudioQueue>(job) on the injected producer | jobs.push(AudioQueue, job) on the injected Arc<Jobs>, answering a PushReceipt; push_with(AudioQueue, job, options) for a delay or a unique key |
queue.push("audio", job) | the queue’s marker, as above, declared on this side too for a queue another deployment drains |
queue.push_json(name, value) | a #[queue(name = …, job = …)] marker naming that queue and the payload both sides agree on, then jobs.push(Marker, job) |
<AudioQueue as QueueName>::NAME | <AudioQueue as Queue>::NAME — QueueName is now the checked name a backend receives |
RedisWorkerModule::for_root(cfg) | QueueModule from nest_rs::queue, imported bare by each queue adapter (QueueModule::for_root(cfg) at the root to pin it), beside RedisQueueModule, which now binds the consumer too |
RedisWorkerConfig { shutdown_timeout } | QueueConfig { shutdown_timeout }; the lease is RedisQueueModule::for_root(RedisQueueConfig { lease }) |
RedisWorker::new(), spawned in a test | QueueWorker::new(), from nest_rs::queue |
<PREFIX>_REDIS__WORKER__SHUTDOWN_TIMEOUT_SECS | <PREFIX>_QUEUE__SHUTDOWN_TIMEOUT_SECS; the lease is <PREFIX>_REDIS__QUEUE__LEASE_SECS — see Configuration |
concurrency refused on #[process] | concurrency = N, per method and per replica — see Concurrency and scaling |
| a failed attempt re-run at once, in the worker | re-run after the port’s backoff, held back off the stream until due — see Retries and failure |
| an attempt whose process died re-run at the same attempt, for free | still re-run at the same attempt, with no retry spent — but past three deliveries that ended without an answer the job is dead-lettered without running; see the stall limit |
| a job a newer release pushed failed by an older worker, spending its retries | handed back as stored, due a minute later, with no attempt spent — and dead-lettered once it has waited a day unread, naming both versions |
| a handler that relied on a job running once | an idempotent one: delivery is at least once — a second delivery comes once a lease lapses, and the fence drops what the first one writes after that |
| a worker draining for 30 seconds | 20 by default, the hand-back reserve included — see Shutdown |
push_many failing with the error alone | QueueError::PartiallyQueued once the backend accepted part of it, carrying the receipts of the jobs already queued |
QueueError::Serialize(e) read as serde’s error, e.line() or e.classify() | a nest_rs::core::DecodeError, whose sentence says what failed and, for a text serde read, where it stopped (at line L column C), never the value serde quoted; the port never locates it, so e.path() is None; a driver’s ? on a serde_json::Error converts as before |
a driver sealing with envelope::seal and opening with envelope::open, and running its own worker | the port seals before enqueue and runs the worker, the attempt and the drain over the driver’s JobConsumer — see Writing a driver; Processor, check_duplicate_queue_claims, CheckpointStore::clear and the tracing re-export are gone with them |
push_with takes a PushOptions: a delay, a unique key — see
Producing jobs.
What fails if you get it wrong
Section titled “What fails if you get it wrong”- A 6.x job left at the root never runs: 7.0 does not read it. Start a 6.x worker again until the queue’s 6.x keys are empty.
- A KEDA trigger still naming the 6.x list reads zero forever and holds the
deployment at
minReplicaCount. It is aredis-streamstrigger onnestrs:queue:{<queue>}:jobsnow — see Concurrency and scaling. <PREFIX>_REDIS__WORKER__SHUTDOWN_TIMEOUT_SECSkept is read by nothing, and a worker’s boot says so atwarn,variable read by nothing since 7.0,read_asnaming<PREFIX>_QUEUE__SHUTDOWN_TIMEOUT_SECS. Until it is renamed, the drain window is whatfor_rootpins, or the 20-second default.
Going further
Section titled “Going further”- Delivery on Redis — where a 7.0 queue keeps what.
- Concurrency and scaling — the trigger on the 7.0 stream.