Skip to content

Delays, unique keys and cancels

PushOptions holds a job back or keeps one job per key, and cancel takes a waiting job back — each a capability its backend declares.

A push can hold its job back, keep one job per key, or be taken back while its job waits. Each is a capability its backend declares, and the port refuses one the backend lacks before anything is filed.

push_with carries what a push declares besides its queue and its payload. A delay holds the job for a Duration, or until a SystemTime:

crates/features/src/audio/service.rs
self.jobs
.push_with(
AudioQueue,
TranscodeCommand { file },
PushOptions::default().with_delay(Duration::from_secs(60)),
)
.await?;

A delay needs a backend declaring Capability::DelayedPush. The Redis backend does; a backend that does not refuses the push with QueueError::Unsupported, naming itself, before anything is filed — a delay is never dropped in silence. On Redis the job waits off the stream, due on Redis’s clock, and a running worker files it within a second of falling due — with none running, it waits — see Delivery on Redis.

A unique key keeps a queue from holding the same piece of work twice. While a job pushed under a key is pending or running, a second push under that key is refused with QueueError::UniqueKeyHeld, which names the job holding it, never the key, which a push most often reads off its payload, and files nothing:

crates/features/src/audio/service.rs
let pushed = self
.jobs
.push_with(
AudioQueue,
TranscodeCommand { file },
PushOptions::default().with_unique(format!("transcode-{file}")),
)
.await;
match pushed {
Ok(receipt) => Ok(Some(receipt)),
Err(QueueError::UniqueKeyHeld { holder, .. }) => {
tracing::debug!(target: crate::audio::TARGET, holder = %holder, "transcode already queued");
Ok(None)
}
Err(error) => Err(error.into()),
}

The key is held from the push until the job reaches its outcome — it completes, it dead-letters, or it is cancelled — and is free for the next push from then on. It is scoped to the queue: the same key on two queues names two jobs. A key is 1 to 256 bytes with no control character, and push_many_with refuses one, since a key names one job.

At most once over pushes, never a lock. A unique key keeps a sequence of pushes from queueing one piece of work twice. It does not make two replicas or two methods take turns at anything, and a job redelivered after its replica died runs again under the key it already holds.

A unique key needs a backend declaring Capability::UniquePush. On Redis a push is one script: it reads and claims the key in the step that files the job, so pushes racing for one key queue one job, and a push lands whole or not at all — a push that never learned its answer claimed the key with its job, or claimed nothing. See Delivery on Redis.

The receipt a push returns names the job, and cancel takes it back while it waits:

crates/features/src/audio/service.rs
let receipt = self.jobs.push(AudioQueue, job).await?;
if self.jobs.cancel(&receipt).await? {
tracing::info!(target: crate::audio::TARGET, job_id = %receipt.id(), "transcode withdrawn");
}

Ok(true) is a promise: the job had not started, and now never will — whether it waited on its queue, on the schedule its delay holds it on, or for the next attempt a failed one left it. Cancelling is the job’s outcome, so the unique key it held is free again and its checkpoint is gone. Ok(false) says the job already started — it then runs to its own outcome — already finished, or was never known to the backend. A job is never stopped mid-attempt.

A job still waiting in the outbox — its transaction committed, the relay not yet done filing it — is cancelled there: the cancel deletes its record, waiting a moment for a relay filing it at that instant, then asks the backend, and either answers true.

A receipt is data: it serializes as {"queue": …, "id": …}, so a caller that cancels later, from another request or another process, keeps it wherever it keeps state. cancel_unique(AudioQueue, key) cancels the job holding a unique key instead, with the same answer.

Cancelling needs a backend declaring Capability::Cancellation. On Redis a cancel is one script: it removes a job still waiting — on the stream and not yet handed over, or held back — with everything it held, and answers false for one a delivery runs. See Delivery on Redis.

  • crates/nest-rs-queue/src/push_options.rs — PushOptions, and the capability each option needs.
  • crates/nest-rs-queue/src/push_receipt.rs — PushReceipt.
  • crates/nest-rs-redis/tests/e2e/queue/producer.rs — a delay, a unique key held, raced and let go, and a cancel, against live Redis.