S3-compatible storage

What this covers

Tornado talks to any storage that speaks the S3 API. Beyond AWS S3 and Cloudflare R2, which have their own tiles in the dashboard, the generic S3 path is what you use for:

MinIO

Self-hosted, on your own hostname and port

Wasabi

Hot cloud storage

Backblaze B2

Via the B2 S3-compatible API

DigitalOcean Spaces

Per-datacenter endpoints

Scaleway

Object Storage

OVHcloud

S3 Object Storage

The configuration is always the same shape: five values, of which only the endpoint distinguishes one provider from another.

Field	Required	Limit	Notes
`bucket`	yes	255 chars	Bucket name only. Never a URL, never a path
`region`	yes	64 chars	Must be non-empty even where the provider ignores it
`access_key_id`	yes	128 chars
`secret_access_key`	yes	256 chars	Stored in Azure Key Vault, never in plaintext
`endpoint`	optional for AWS, required here	512 chars	Provider S3 API host, no bucket in it
`path_style`	optional	boolean	Not exposed in the dashboard. See Addressing style

Requests are always signed with AWS Signature Version 4. This is forced, not negotiated. A provider or gateway that only supports SigV2 will not work.

Endpoint and region per provider

The examples below identify endpoint shapes; consult the linked provider documentation and console for your bucket’s current endpoint and actual region, since several providers assign it per bucket rather than per account.

Provider	Endpoint	Region	Source
MinIO	`https://your-minio-host:9000` (your own hostname and port)	The value of `MINIO_SITE_REGION` on your deployment. Send `us-east-1` if you have not set one	MinIO site settings
Wasabi	`https://s3.{region}.wasabisys.com`	The region code in the endpoint, for example `us-east-2`, `eu-central-1`, `ap-northeast-1`	Wasabi service URLs
Backblaze B2	`https://s3.{region}.backblazeb2.com`	The middle segment of your bucket’s endpoint, for example `us-west-004`	Calling the B2 S3-compatible API
DigitalOcean Spaces	`https://{region}.digitaloceanspaces.com`	The datacenter code, for example `nyc3`, `fra1`, `sgp1`	Spaces with AWS S3 SDKs
Scaleway	`https://s3.{region}.scw.cloud`	`fr-par`, `nl-ams`, `pl-waw` or `it-mil`	Scaleway Object Storage concepts
OVHcloud	`https://s3.{region}.io.cloud.ovh.net`	The region code in lowercase, for example `gra`, `rbx`, `sbg`, `de`, `uk`, `waw`, `eu-west-par`, `eu-south-mil`, `bhs`, `ca-east-tor`, `sgp`, `ap-southeast-syd`, `ap-south-mum`	OVHcloud endpoints and geoavailability

Wasabi us-east-1 is the one region that also answers on a bare https://s3.wasabisys.com. Prefer the explicit https://s3.us-east-1.wasabisys.com form so the endpoint and the region field agree.

OVHcloud: s3.{region}.io.cloud.ovh.net is the current endpoint, and OVHcloud calls it “the preferred endpoint to access OVHcloud Object Storage”. The older s3.{region}.perf.cloud.ovh.net is “maintained for backward compatibility purposes only, to allow tools that don’t support AWS’s recent Express_One_Zone storage class to continue operating”. Tornado is not one of those tools. Use the io form.

DigitalOcean: some AWS SDKs insist on us-east-1 when creating a bucket. Tornado never creates buckets, it only writes objects into one you already own, so put your datacenter code in the region field.

What Tornado does to your endpoint string

The dashboard’s S3 probe normalises its endpoint before constructing the client:

Leading and trailing whitespace is stripped.
An accidental doubled scheme is collapsed, so https://https://host becomes https://host.
If no scheme is present, https:// is prepended.
Trailing slashes are trimmed.

Two consequences worth knowing:

Use a public HTTPS endpoint. The dashboard validates the normalised URL and rejects plain HTTP, private/internal hosts and local addresses. The cleanup rules do not make an internal MinIO deployment reachable from Tornado. Save the complete, canonical HTTPS URL: the worker credential mapping does not repeat the probe’s scheme-repair steps.
The bucket must not appear in the endpoint. Only the scheme, host and port are trimmed. A path such as https://s3.fr-par.scw.cloud/my-bucket is kept verbatim and the bucket ends up in the request twice.

Filling the fields in the dashboard

The onboarding wizard has a dedicated tile for this. In Destination, pick Any S3-compatible (the wide tile at the bottom of the grid, below Google Drive), then fill the Credentials step:

Endpoint URL

The provider host from the table above, with scheme. The placeholder shows the expected shape: https://minio.internal.example.dev:9000. Despite the placeholder’s wording, your actual host must be publicly reachable over HTTPS.

Bucket name

The bucket, on its own. It must already exist; Tornado does not create it.

Region

The region value from the table above. The field cannot be left empty.

Access key

Maps to access_key_id.

Secret key

Maps to secret_access_key. Write-only in the UI, and read back masked afterwards.

Test connection

Runs a live probe against the bucket before anything is saved. Read what it proves before you trust the green tick.

The same configuration is editable later under Settings, where the S3 provider exposes Bucket, Region, Access Key ID, Secret Access Key and Endpoint (optional). The endpoint is optional only because the same form serves AWS S3, where it can be omitted. For every provider on this page it is required.

Credentials are written to Azure Key Vault. The organisation document keeps routing metadata, a masked display configuration and a vault reference. If Key Vault is unreachable the save is refused with a 503 rather than falling back to storing the secret in the database.

Addressing style: path-style vs virtual-host

An S3 client can address a bucket in two ways:

Virtual-host style: https://my-bucket.s3.example.com/key. The bucket becomes a subdomain, so the provider needs a wildcard DNS record and a matching TLS certificate.
Path-style: https://s3.example.com/my-bucket/key. The bucket is the first path segment.

Verify which addressing styles your provider and endpoint accept. DNS, certificates and bucket names can affect the SDK’s choice. MinIO is the exception. MinIO only accepts virtual-host requests when the deployment sets MINIO_DOMAIN, and its own documentation is explicit that “omitting this setting directs MinIO AIStor to only accept the default path-style requests” (core settings). A stock MinIO therefore needs path-style, and enabling virtual-host on it means adding a wildcard DNS entry and a certificate covering *.your-minio-host as well.

The current limitation

path_style is not exposed anywhere in the dashboard. Neither the onboarding wizard’s “Any S3-compatible” tile nor the Settings storage form renders a control for it. The wizard offers exactly five inputs: Endpoint URL, Bucket name, Region, Access key, Secret key. The field exists in the API model and is honoured by the backend, but a dashboard-only user cannot set it.

The worker and dashboard construct separate clients:

Operation	Component	Addressing
Uploading the finished file	Rust worker	Always path-style, regardless of `path_style`
Job-status `s3_url`	Rust worker	Always path-style
”Test connection” probe	Dashboard backend	Uses path-style when supplied; otherwise the SDK’s automatic addressing rules
Custom webhook `download_url`	Dashboard backend	Uses path-style when supplied; otherwise the SDK’s automatic addressing rules

A path-style-only endpoint can therefore behave differently in the probe and dashboard webhook signer. A missing path_style flag does not by itself prove failure: the SDK can choose path-style automatically. Test both the probe and the actual delivered link.

Supplying path-style through the API

The settings model accepts path_style, and the probe honors it when supplied. However, the current save path converts credentials into the worker’s flat format, which does not retain this flag. Sending it on save therefore does not guarantee that later dashboard webhook links use path-style. The worker already forces path-style independently.

curl -X POST "https://api-dash.tornadoapi.io/api/settings/storage" \
  -H "Authorization: Bearer $CLERK_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "s3",
    "s3": {
      "endpoint": "https://minio.internal.example.dev:9000",
      "bucket": "tornado-downloads",
      "region": "us-east-1",
      "access_key_id": "YOUR_ACCESS_KEY",
      "secret_access_key": "YOUR_SECRET_KEY",
      "path_style": true
    }
  }'

To probe without saving, send the complete same payload, including path_style: true, to POST /settings/storage/test. For example, save that JSON as storage.json and run:

curl -X POST "https://api-dash.tornadoapi.io/api/settings/storage/test" \
  -H "Authorization: Bearer $CLERK_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @storage.json

Both routes require an Admin or Owner session token, not an x-api-key. Keep credential JSON outside version control. A successful explicit path-style probe is not proof that the saved dashboard signer retains that option; verify its actual URL.

What “Test connection” proves

The probe is a real network call against your real bucket, not a credential format check. It runs five steps and reports the round-trip latency:

Step	Call	Permission it proves
1	`PutObject` to `tornado-connection-test/<random>.txt` (23 bytes)	`s3:PutObject`
2	`GetObject` on the object just written	`s3:GetObject`
3	`CreateMultipartUpload` on `<same key>.multipart`	`s3:PutObject` again, on the multipart path
4	`AbortMultipartUpload` on that upload, with no part uploaded	`s3:AbortMultipartUpload`
5	`DeleteObject` on the probe object	`s3:DeleteObject`

No part is ever uploaded, so the whole probe moves 23 bytes regardless of how many steps it runs. Each step names itself in the error string, so a failure tells you which permission is missing rather than just AccessDenied. It proves:

The endpoint resolves, the TLS handshake succeeds and the host speaks S3.
The bucket exists and is reachable at that endpoint with the region you supplied.
The credentials carry s3:PutObject, s3:GetObject, s3:AbortMultipartUpload and s3:DeleteObject on that bucket.
The read works as the identity your download links are signed with, which is the reason the read leg exists at all.
The addressing style currently in effect works for the endpoint.
The provider actually implements the multipart API. Not every S3-compatible gateway does, and a gateway that returns NotImplemented at step 3 cannot handle the multipart delivery path (used above the default 50 MiB file threshold and by streaming uploads).

It does not prove:

That a real delivery will fit. The probe moves 23 bytes and uploads no parts.
That your policy allows the delivery prefix. The probe writes to tornado-connection-test/ at the bucket root. A policy that permits that probe prefix but denies videos/* can pass the test and fail the job. A policy allowing only videos/* instead rejects the probe.
That a recipient can fetch the link. Tornado reads from its own network; a bucket policy conditioned on source IP, or a private MinIO your customers cannot reach, behaves differently for them.
That the presigned link is addressed correctly. On a stock MinIO the probe can pass while the download_url points at a hostname that does not resolve. See Addressing style.

Permissions

Minimum

Exactly the four actions the probe exercises and delivery uses, and nothing else:

Minimum

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload"
      ],
      "Resource": "arn:aws:s3:::tornado-downloads/*"
    }
  ]
}

Action	Why
`s3:PutObject`	Every upload, and the probe write. It also authorises `CreateMultipartUpload`, `UploadPart` and `CompleteMultipartUpload`, so those need nothing extra
`s3:GetObject`	The presigned download links, which are signed locally with your key and evaluated as your identity when the recipient clicks
`s3:DeleteObject`	The probe cleanup, and file deletion from the dashboard
`s3:AbortMultipartUpload`	Stopping an upload that failed part-way. Above the default 50 MiB file threshold uploads are multipart, and without the abort the uploaded parts stay in the bucket: not objects, so absent from the object listing, and billed as storage

Do not add s3:CreateMultipartUpload, s3:UploadPart or s3:CompleteMultipartUpload. They are not IAM actions. AWS’s multipart permission table maps all three to s3:PutObject, For a compatible provider, check its supported policy actions and multipart behavior rather than assuming complete AWS equivalence. AbortMultipartUpload is the only step of the lifecycle with its own action name, which is exactly why it is the one people leave out.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload"
      ],
      "Resource": "arn:aws:s3:::tornado-downloads/*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:ListBucketMultipartUploads",
      "Resource": "arn:aws:s3:::tornado-downloads"
    }
  ]
}

Added	Why it is worth adding
`s3:ListBucketMultipartUploads`	Required to call `ListMultipartUploads`, which Tornado uses only to find an upload orphaned by a `CreateMultipartUpload` timeout. Without it that sweep silently does not run and the orphan is billed until something else reaps it. Drop it if your provider does not implement `ListMultipartUploads`, or if the failure mode being billing rather than delivery makes it not worth the argument

s3:ListBucket is not needed at all. Tornado never enumerates the bucket: it writes to a key it computed itself, signs a URL for that key, and deletes by key.

Providers with their own dialect

Provider	How to express the same set
MinIO, Wasabi, Scaleway, DigitalOcean Spaces, OVHcloud	AWS-style policy documents. The JSON above works as written; substitute the resource ARN form the vendor uses
Backblaze B2	Application keys use capabilities, not actions. Grant `readFiles`, `writeFiles` and `deleteFiles` on the one bucket, which is the console’s Read and Write option. There is no separate abort capability: `writeFiles` and `deleteFiles` cover the multipart lifecycle

Scope the resource to the whole bucket, not to the delivery prefix. Delivered files land under videos/, but the connection test writes to tornado-connection-test/ at the bucket root. Narrow the policy to tornado-downloads/videos/* and the test fails with AccessDenied: s3:PutObject even though real deliveries would have worked.

Troubleshooting

The error string returned by the connection test is deliberately compact. For S3 failures it is <ProviderErrorCode>: s3:<Operation>, where the operation is whichever of the probe’s five steps failed first: PutObject, GetObject, CreateMultipartUpload, AbortMultipartUpload or DeleteObject.

Error string	What it means	Fix
`AccessDenied: s3:PutObject`	Credentials are valid, the policy does not allow writing to this bucket or prefix	Widen the policy to cover `bucket/*`, not just `bucket`
`AccessDenied: s3:GetObject`	The write succeeded, reading the same object back did not	Add `s3:GetObject`. This is the permission every presigned download link is evaluated against, so a write-only key now fails here instead of silently 403ing your customers
`AccessDenied: s3:CreateMultipartUpload`	Write and read passed, starting a multipart upload did not	The provider maps this to `s3:PutObject`, so an explicit Deny or a resource pattern that excludes `<key>.multipart` is the usual cause. Widen the resource to `bucket/*`. On a gateway that returns `NotImplemented` rather than `AccessDenied`, the multipart API is simply absent and multipart deliveries will fail (the default file threshold is 50 MiB)
`AccessDenied: s3:AbortMultipartUpload`	The multipart upload started but could not be stopped	Add `s3:AbortMultipartUpload`. It is a distinct action, not covered by `s3:PutObject`. Until it is granted, every delivery that fails part-way leaves billed parts behind that do not show up in the object listing
`AccessDenied: s3:DeleteObject`	Everything else succeeded, the cleanup did not	Add `s3:DeleteObject`. Note the probe object is now left behind in `tornado-connection-test/`
`InvalidAccessKeyId: s3:PutObject`	The access key is unknown to the provider	Check for a copied newline or a key from a different project or region
`SignatureDoesNotMatch: s3:PutObject`	Secret is wrong, or the region does not match the endpoint	Re-copy the secret. Then confirm the region is the one embedded in the endpoint host
`NoSuchBucket: s3:PutObject`	The bucket does not exist at this endpoint	Usually the right bucket at the wrong region endpoint. Check both together
`PermanentRedirect: s3:PutObject`	The bucket lives in a different region to the endpoint	Use the endpoint for the bucket’s own region
`Could not connect to the endpoint URL: ...`	DNS or routing failure, verbatim from the SDK	Typo in the host, private MinIO not reachable from the internet, firewall
`Connect timeout on endpoint URL: ...`	Host resolves but does not accept the connection within 5s	Port closed or wrong. MinIO is commonly on `:9000` and needs it written explicitly
`timeout: no response from storage endpoint after 20s`	The whole probe exceeded its budget	A black-holed endpoint, or a gateway accepting the connection then stalling
`internal error during storage test`	See below	Almost always a malformed endpoint

”internal error during storage test” means your endpoint is malformed

This is the least informative error and the most common one to hit while setting up a new provider. It says nothing about URLs, but that is usually exactly what it is. The S3 client is constructed before the probe’s own error handling begins. If the SDK rejects the endpoint string outright, for example because it contains a space, an illegal character, or an over-long hostname label left behind by a bad paste, the rejection happens during construction and is caught by the generic handler, which reports internal error during storage test with no detail. When you see it, check the endpoint before anything else:

No spaces, no stray characters

A trailing space inside the value survives the trim if it is in the middle. Retype the host rather than pasting it.

Scheme written once

https://https://... is collapsed automatically, but a mangled paste such as https://host/.provider.com/ is not repaired.

No bucket, no path

Host and optional port only.

Not the console URL

Provider consoles live on different hostnames to the S3 API. Wasabi’s console is console.wasabisys.com, which is not a storage endpoint.

Test passes, downloads return 403

The probe reads, so a policy with no s3:GetObject at all now fails the test rather than passing it. With a green probe and 403 links, check restrictions on the delivery prefix, recipient network, link expiry and credential rotation. The probe exercises its own object and network path; it cannot rule out all other access conditions.

Test passes on MinIO but webhook links do not resolve

Inspect whether the URL uses a bucket subdomain that your endpoint cannot resolve. An explicit path_style: true can validate the probe path, but the current save path does not preserve that flag for the dashboard signer. Verify the provider’s supported addressing mode and DNS/certificate configuration; do not assume re-saving the flag fixes future webhook links.

Job reports Completed but the bucket is empty

Check the destination selected for that job. A per-key configuration takes precedence over the organization default; a key with no own configuration can inherit that default. Legacy or managed storage is used only when no applicable modern destination is selected. See Custom Storage for the selection rules. A completed job in another bucket does not by itself identify a provider compatibility problem.

”missing required field(s): …”

Returned by the offline structural fallback, which only runs when the S3 SDK itself is unavailable on the server. If you see it against production, the response is telling you the round-trip never happened. Retry, and if it persists it is a Tornado-side problem rather than a credentials problem.

API Reference

Jobs

Bulk

Metadata

Batch Operations

User

S3-compatible storage

What this covers

MinIO

Wasabi

Backblaze B2

DigitalOcean Spaces

Scaleway

OVHcloud

Endpoint and region per provider

What Tornado does to your endpoint string

Filling the fields in the dashboard

Addressing style: path-style vs virtual-host

The current limitation

Supplying path-style through the API

What “Test connection” proves

Permissions

Minimum

Recommended

Providers with their own dialect

Troubleshooting

”internal error during storage test” means your endpoint is malformed

Test passes, downloads return 403

Test passes on MinIO but webhook links do not resolve

Job reports Completed but the bucket is empty

”missing required field(s): …”

​What this covers

MinIO

Wasabi

Backblaze B2

DigitalOcean Spaces

Scaleway

OVHcloud

​Endpoint and region per provider

​What Tornado does to your endpoint string

​Filling the fields in the dashboard

​Addressing style: path-style vs virtual-host

​The current limitation

​Supplying path-style through the API

​What “Test connection” proves

​Permissions

​Minimum

​Recommended

​Providers with their own dialect

​Troubleshooting

​”internal error during storage test” means your endpoint is malformed

​Test passes, downloads return 403

​Test passes on MinIO but webhook links do not resolve

​Job reports Completed but the bucket is empty

​”missing required field(s): …”

What this covers

Endpoint and region per provider

What Tornado does to your endpoint string

Filling the fields in the dashboard

Addressing style: path-style vs virtual-host

The current limitation

Supplying path-style through the API

What “Test connection” proves

Permissions

Minimum

Recommended

Providers with their own dialect

Troubleshooting

”internal error during storage test” means your endpoint is malformed

Test passes, downloads return 403

Test passes on MinIO but webhook links do not resolve

Job reports Completed but the bucket is empty

”missing required field(s): …”