Delays, unique keys and cancels
PushOptions holds a job back or keeps one job per key, and cancel takes a waiting job back — each a capability its backend declares.
A push can hold its job back, keep one job per key, or be taken back while its job waits. Each is a capability its backend declares, and the port refuses one the backend lacks before anything is filed.
Holding a job back
Section titled “Holding a job back”push_with carries what a push declares besides its queue and its payload.
A delay holds the job for a Duration, or until a SystemTime:
self.jobs .push_with( AudioQueue, TranscodeCommand { file }, PushOptions::default().with_delay(Duration::from_secs(60)), ) .await?;A delay needs a backend declaring Capability::DelayedPush. The Redis backend
does; a backend that does not refuses the push with QueueError::Unsupported,
naming itself, before anything is filed — a delay is never dropped in silence.
On Redis the job waits off the stream, due on Redis’s clock, and a running
worker files it within a second of falling due — with none running, it waits —
see Delivery on Redis.
One job per key
Section titled “One job per key”A unique key keeps a queue from holding the same piece of work twice. While a
job pushed under a key is pending or running, a second push under that key is
refused with QueueError::UniqueKeyHeld, which names the job holding it, never
the key, which a push most often reads off its payload, and files nothing:
let pushed = self .jobs .push_with( AudioQueue, TranscodeCommand { file }, PushOptions::default().with_unique(format!("transcode-{file}")), ) .await;match pushed { Ok(receipt) => Ok(Some(receipt)), Err(QueueError::UniqueKeyHeld { holder, .. }) => { tracing::debug!(target: crate::audio::TARGET, holder = %holder, "transcode already queued"); Ok(None) } Err(error) => Err(error.into()),}The key is held from the push until the job reaches its outcome — it
completes, it dead-letters, or it is cancelled — and is free for the next push
from then on. It is scoped to the queue: the same key on two queues names two
jobs. A key is 1 to 256 bytes with no control character, and push_many_with
refuses one, since a key names one job.
At most once over pushes, never a lock. A unique key keeps a sequence of pushes from queueing one piece of work twice. It does not make two replicas or two methods take turns at anything, and a job redelivered after its replica died runs again under the key it already holds.
A unique key needs a backend declaring Capability::UniquePush. On Redis a
push is one script: it reads and claims the key in the step that files the job,
so pushes racing for one key queue one job, and a push lands whole or not at
all — a push that never learned its answer claimed the key with its job, or
claimed nothing. See Delivery on Redis.
Cancelling a job
Section titled “Cancelling a job”The receipt a push returns names the job, and cancel takes it back while it
waits:
let receipt = self.jobs.push(AudioQueue, job).await?;
if self.jobs.cancel(&receipt).await? { tracing::info!(target: crate::audio::TARGET, job_id = %receipt.id(), "transcode withdrawn");}Ok(true) is a promise: the job had not started, and now never will —
whether it waited on its queue, on the schedule its delay holds it on, or for
the next attempt a failed one left it. Cancelling is the job’s outcome, so the
unique key it held is free again and its checkpoint is gone. Ok(false) says
the job already started — it then runs to its own outcome — already finished,
or was never known to the backend. A job is never stopped mid-attempt.
A job still waiting in the outbox — its transaction committed, the relay not
yet done filing it — is cancelled there: the cancel deletes its record, waiting
a moment for a relay filing it at that instant, then asks the backend, and
either answers true.
A receipt is data: it serializes as {"queue": …, "id": …}, so a caller that
cancels later, from another request or another process, keeps it wherever it
keeps state. cancel_unique(AudioQueue, key) cancels the job holding a unique
key instead, with the same answer.
Cancelling needs a backend declaring Capability::Cancellation. On Redis a
cancel is one script: it removes a job still waiting — on the stream and not
yet handed over, or held back — with everything it held, and answers false
for one a delivery runs. See Delivery on Redis.
Reference
Section titled “Reference”crates/nest-rs-queue/src/push_options.rs—PushOptions, and the capability each option needs.crates/nest-rs-queue/src/push_receipt.rs—PushReceipt.crates/nest-rs-redis/tests/e2e/queue/producer.rs— a delay, a unique key held, raced and let go, and a cancel, against live Redis.
Going further
Section titled “Going further”- Producing jobs — the push these options ride on.
- Delivery on Redis — what the Redis backend keeps for each capability.