Storage

Object storage shares the rest of the framework’s shape. Storage is a regular #[injectable] provider: inject it, call a handful of async methods. It hands clients short-lived presigned URLs so bytes never transit your API, and reads or writes objects server-side for workers that transform them.

The client ships in nest-rs-storage, built on object_store — the generic object-store abstraction maintained under Apache Arrow. It is multi-driver (S3, GCS, Azure, local filesystem, in-memory) behind one trait, with presigning in a separate Signer trait the S3 driver implements. The AWS-S3 driver is wired by default and speaks to real AWS S3 as well as any S3-compatible server (MinIO, RustFS) in path- or virtual-host style.

Note

object_store is reqwest/rustls-based — no aws-runtime / aws-sdk-* in the tree — so it builds on the pinned toolchain where the official AWS SDK currently does not.

Install

cargo add nest-rs --features storage

Wire it in

StorageModule owns its StorageConfig (namespace storage, loaded from NESTRS_STORAGE__*) and registers Storage as a singleton provider. Import it where a feature needs storage:

crates/features/src/media/module.rs

use nest_rs::core::module;
use nest_rs::storage::StorageModule;

use crate::media::MediaService;

#[module(imports = [StorageModule], providers = [MediaService])]
pub struct MediaModule;

The media slice below (a post cover-image upload) is the canonical storage shape.

Configuration

Every field is settable via NESTRS_STORAGE__* env or a pinned StorageConfig — the framework-wide dual-path config rule. Pin it at the composition root; the environment still overlays it per field, so a bucket in the source does not freeze the credentials against the deployment:

apps/api/src/module.rs

StorageModule::for_root(StorageConfig {
    bucket: "acme-media".into(),
    ..Default::default()
}),

Importing the bare StorageModule — what a feature module does — leaves the whole config to the environment.

Two defaults below are profile-dependent, and both differences are security ones: outside dev/test the sentinel credentials are dropped and plain HTTP is off, so an unset key fails boot by name instead of authenticating with a public default.

Key	Default	Meaning
ENDPOINT	http://rustfs:9000	S3 endpoint; empty ⇒ real AWS S3
REGION	us-east-1	region
ACCESS_KEY / SECRET_KEY	nestrs, none outside dev/test	static credentials
BUCKET	nestrs	target bucket
FORCE_PATH_STYLE	true	true ⇒ endpoint/bucket/key; false ⇒ virtual-hosted
ALLOW_HTTP	true, false outside dev/test	reach the endpoint over plain http:// — see the caution below

So a staging or production deployment poses three keys at minimum: an https:// ENDPOINT (the default is plain HTTP and refused there), ACCESS_KEY and SECRET_KEY.

Presigned uploads

The canonical flow keeps bytes off the API. A handler injects Storage and returns a signed PUT URL; the client uploads straight to the object store; a later call reads the object’s metadata to finalize.

crates/features/src/media/service.rs

use std::sync::Arc;
use std::time::Duration;

use nest_rs::core::injectable;
use nest_rs::storage::{Result, Storage};

#[injectable]
pub struct MediaService {
    #[inject]
    storage: Arc<Storage>,
}

impl MediaService {
    /// Hand the client a short-lived URL to PUT the cover image directly.
    pub async fn request_cover_upload(&self, key: &str) -> Result<String> {
        self.storage.presign_put(key, Duration::from_secs(900)).await
    }

    /// After the client uploads, read the size back to finalize the record.
    /// `None` ⇒ the client never completed the upload.
    pub async fn confirm_cover_upload(&self, key: &str) -> Result<Option<i64>> {
        Ok(self.storage.head(key).await?.map(|info| info.byte_size))
    }
}

Caution

head reports the object’s size but not its Content-Type: object_store’s ObjectMeta does not carry it, so ObjectMetadata exposes only the byte size. Keep the mime type you were given at upload-request time rather than reading it back. put_bytes still sets the content type on write — only the head read-back path lacks it.

Server-side read and write

Workers that transform objects (e.g. generating a WebP variant) read and write bytes directly:

crates/features/src/media/service.rs

let original = self.storage.get_bytes(key).await?;   // -> bytes::Bytes
let webp = transcode(&original)?;
self.storage
    .put_bytes(&variant_key, webp, "image/webp")
    .await?;

presign_get is the counterpart to presign_put for serving private originals: a short-lived signed GET URL the client fetches directly.

API

Method	Purpose
bucket_name() -> &str	the configured bucket
presign_put(key, expires) -> String	signed PUT URL for direct client upload
presign_get(key, expires) -> String	signed GET URL for serving private originals
head(key) -> Option<ObjectMetadata>	object size; None if absent
get_bytes(key) -> Bytes	download full bytes (server-side)
get_stream(key) -> impl Stream<Item = Result<Bytes>>	download chunk by chunk, never buffering the whole object
list(prefix) -> impl Stream<Item = Result<ObjectEntry>>	the keys under a prefix, streamed page by page; "" lists the bucket
put_bytes(key, bytes, content_type)	upload bytes — takes anything Into<Bytes>, so the Bytes from get_bytes composes without a copy
put_stream(key, content_type, stream)	upload from a stream of std::io::Result<Bytes> as a multipart upload, one 5 MiB part at a time
delete(key)	remove an object; an absent key succeeds, so retention sweeps and failed-upload cleanup are idempotent

That is the whole surface.

list matches on path segments, so posts/cover returns posts/cover/a.png and never posts/cover-2.png, and each ObjectEntry carries the key, the byte size and a std::time::SystemTime. put_stream aborts its multipart upload on any failure — the store’s or your stream’s — so an interrupted upload leaves no billable parts behind.

StorageError is not a ResponseError: an infrastructure error does not get to pick its own HTTP status, so ?-ing one straight out of a handler does not compile. Wrap it in the feature’s own error, which is where the status belongs:

crates/features/src/media/error.rs

use nest_rs::http::poem::error::ResponseError;
use nest_rs::http::poem::http::StatusCode;
use nest_rs::storage::StorageError;

#[derive(Debug, thiserror::Error)]
pub enum MediaError {
    #[error("media storage operation failed")]
    Storage(#[from] StorageError),
}

impl ResponseError for MediaError {
    fn status(&self) -> StatusCode { StatusCode::INTERNAL_SERVER_ERROR }
}

The handler then returns Result<_, MediaError> and ? works. See audio/error.rs for the shape with a ProblemDetails body and the error log.

A StorageError does convert into std::io::Error, so a get_stream feeds poem::Body::from_bytes_stream directly:

crates/features/src/media/http/controller.rs

let Some(stream) = self.storage.get_stream(key).await.ok() else {
    return Ok(ProblemDetails::not_found().into_response());
};
Ok(Response::builder().body(Body::from_bytes_stream(stream)))

Caution

ALLOW_HTTP=false — the staging/production default — refuses a plain-http:// endpoint at boot, naming NESTRS_STORAGE__ENDPOINT. It covers presigned URLs too: signing is a local computation, so a plaintext endpoint would otherwise hand clients a working URL carrying the SigV4 signature. Point it at an https:// endpoint, or set NESTRS_STORAGE__ALLOW_HTTP=true to opt in deliberately.

Another backend

Because the seam is the object_store traits, pointing Storage at GCS, Azure, the local filesystem, or in-memory is a builder change inside the crate — not an API change for the features that inject it. The S3 driver is the default because it also covers every S3-compatible server.

Reference

• crates/nest-rs-storage/ — Storage, ObjectMetadata, ObjectEntry, StorageConfig, StorageModule.
• audio in the demo — presigns an upload URL, and the worker reads the stored object and writes a derived one.

Going further

• Queue — background workers that read and transform stored objects.
• Database — persist the record a presigned upload finalizes.
• Configuration — the dual-path rule behind StorageConfig.