> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tornadoapi.io/llms.txt
> Use this file to discover all available pages before exploring further.
# S3-compatible storage
> Configure MinIO, Wasabi, Backblaze B2, DigitalOcean Spaces, Scaleway or OVHcloud as your delivery target
## What this covers
Tornado talks to any storage that speaks the S3 API. Beyond AWS S3 and Cloudflare R2, which have
their own tiles in the dashboard, the generic S3 path is what you use for:
Self-hosted, on your own hostname and port
Hot cloud storage
Via the B2 S3-compatible API
Per-datacenter endpoints
Object Storage
S3 Object Storage
The configuration is always the same shape: five values, of which only the endpoint distinguishes
one provider from another.
| Field | Required | Limit | Notes |
| ------------------- | ----------------------------------- | --------- | -------------------------------------------------------------------------------------------------- |
| `bucket` | yes | 255 chars | Bucket name only. Never a URL, never a path |
| `region` | yes | 64 chars | Must be non-empty even where the provider ignores it |
| `access_key_id` | yes | 128 chars | |
| `secret_access_key` | yes | 256 chars | Stored in Azure Key Vault, never in plaintext |
| `endpoint` | optional for AWS, **required here** | 512 chars | Provider S3 API host, no bucket in it |
| `path_style` | optional | boolean | Not exposed in the dashboard. See [Addressing style](#addressing-style-path-style-vs-virtual-host) |
Requests are always signed with **AWS Signature Version 4**. This is forced, not negotiated.
A provider or gateway that only supports SigV2 will not work.
***
## Endpoint and region per provider
Every endpoint below was checked against the provider's own current documentation. Use the
provider's console to read your actual region, since several providers assign it per bucket
rather than per account.
| Provider | Endpoint | Region | Source |
| ------------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| MinIO | `https://your-minio-host:9000` (your own hostname and port) | The value of `MINIO_SITE_REGION` on your deployment. Send `us-east-1` if you have not set one | [MinIO site settings](https://docs.min.io/aistor/reference/aistor-server/settings/site/) |
| Wasabi | `https://s3.{region}.wasabisys.com` | The region code in the endpoint, for example `us-east-2`, `eu-central-1`, `ap-northeast-1` | [Wasabi service URLs](https://docs.wasabi.com/v1/docs/what-are-the-service-urls-for-wasabis-different-storage-regions) |
| Backblaze B2 | `https://s3.{region}.backblazeb2.com` | The middle segment of your bucket's endpoint, for example `us-west-004` | [Calling the B2 S3-compatible API](https://www.backblaze.com/docs/cloud-storage-call-the-s3-compatible-api) |
| DigitalOcean Spaces | `https://{region}.digitaloceanspaces.com` | The datacenter code, for example `nyc3`, `fra1`, `sgp1` | [Spaces with AWS S3 SDKs](https://docs.digitalocean.com/products/spaces/how-to/use-aws-sdks/) |
| Scaleway | `https://s3.{region}.scw.cloud` | `fr-par`, `nl-ams`, `pl-waw` or `it-mil` | [Scaleway Object Storage concepts](https://www.scaleway.com/en/docs/object-storage/concepts/) |
| OVHcloud | `https://s3.{region}.io.cloud.ovh.net` | The region code in lowercase, for example `gra`, `rbx`, `sbg`, `de`, `uk`, `waw`, `eu-west-par`, `eu-south-mil`, `bhs`, `ca-east-tor`, `sgp`, `ap-southeast-syd`, `ap-south-mum` | [OVHcloud endpoints and geoavailability](https://docs.ovhcloud.com/en/guides/storage-and-backup/object-storage/s3-location) |
**Wasabi us-east-1** is the one region that also answers on a bare `https://s3.wasabisys.com`.
Prefer the explicit `https://s3.us-east-1.wasabisys.com` form so the endpoint and the region
field agree.
**OVHcloud**: `s3.{region}.io.cloud.ovh.net` is the current endpoint, and OVHcloud calls it
"the preferred endpoint to access OVHcloud Object Storage". The older
`s3.{region}.perf.cloud.ovh.net` is "maintained for backward compatibility purposes only, to
allow tools that don't support AWS's recent Express\_One\_Zone storage class to continue
operating". Tornado is not one of those tools. Use the `io` form.
**DigitalOcean**: some AWS SDKs insist on `us-east-1` when creating a bucket. Tornado never
creates buckets, it only writes objects into one you already own, so put your datacenter code
in the region field.
### What Tornado does to your endpoint string
Before the S3 client is built, the endpoint is normalised:
1. Leading and trailing whitespace is stripped.
2. An accidental doubled scheme is collapsed, so `https://https://host` becomes `https://host`.
3. If no scheme is present, `https://` is prepended.
4. Trailing slashes are trimmed.
Two consequences worth knowing:
* **Plain HTTP is preserved but never assumed.** A MinIO instance on `http://` must be entered
with the scheme written out. `minio.internal:9000` is rewritten to
`https://minio.internal:9000` and the TLS handshake will fail.
* **The bucket must not appear in the endpoint.** Only the scheme, host and port are trimmed.
A path such as `https://s3.fr-par.scw.cloud/my-bucket` is kept verbatim and the bucket ends up
in the request twice.
***
## Filling the fields in the dashboard
The onboarding wizard has a dedicated tile for this. In **Destination**, pick **Any
S3-compatible** (the wide tile at the bottom of the grid, below Google Drive), then fill the
**Credentials** step:
The provider host from the table above, with scheme. The placeholder shows the expected
shape: `https://minio.internal.example.dev:9000`.
The bucket, on its own. It must already exist; Tornado does not create it.
The region value from the table above. The field cannot be left empty.
Maps to `access_key_id`.
Maps to `secret_access_key`. Write-only in the UI, and read back masked afterwards.
Runs a live probe against the bucket before anything is saved. Read
[what it proves](#what-test-connection-proves) before you trust the green tick.
The same configuration is editable later under **Settings**, where the S3 provider exposes
**Bucket**, **Region**, **Access Key ID**, **Secret Access Key** and **Endpoint (optional)**.
The endpoint is optional only because the same form serves AWS S3, where it can be omitted. For
every provider on this page it is required.
Credentials are written to Azure Key Vault. The organisation document keeps only the provider
name and a vault reference. If Key Vault is unreachable the save is refused with a 503 rather
than falling back to storing the secret in the database.
***
## Addressing style: path-style vs virtual-host
An S3 client can address a bucket in two ways:
* **Virtual-host style**: `https://my-bucket.s3.example.com/key`. The bucket becomes a
subdomain, so the provider needs a wildcard DNS record and a matching TLS certificate.
* **Path-style**: `https://s3.example.com/my-bucket/key`. The bucket is the first path segment.
The managed providers on this page (Wasabi, Backblaze B2, DigitalOcean Spaces, Scaleway,
OVHcloud) serve both styles, so the default works.
**MinIO is the exception.** MinIO only accepts virtual-host requests when the deployment sets
`MINIO_DOMAIN`, and its own documentation is explicit that "omitting this setting directs MinIO
AIStor to only accept the default path-style requests"
([core settings](https://docs.min.io/enterprise/aistor-object-store/reference/aistor-server/settings/core/)).
A stock MinIO therefore needs path-style, and enabling virtual-host on it means adding a
wildcard DNS entry and a certificate covering `*.your-minio-host` as well.
### The current limitation
**`path_style` is not exposed anywhere in the dashboard.** Neither the onboarding wizard's
"Any S3-compatible" tile nor the Settings storage form renders a control for it. The wizard
offers exactly five inputs: Endpoint URL, Bucket name, Region, Access key, Secret key. The
field exists in the API model and is honoured by the backend, but a dashboard-only user cannot
set it.
What actually breaks without it is narrower than you might expect, because two different
components sign URLs:
| Operation | Component | Addressing |
| ----------------------------- | ----------------- | --------------------------------------------- |
| Uploading the finished file | Rust worker | Always path-style, regardless of `path_style` |
| Job-status `s3_url` | Rust worker | Always path-style |
| "Test connection" probe | Dashboard backend | Follows `path_style`, virtual-host by default |
| Custom webhook `download_url` | Dashboard backend | Follows `path_style`, virtual-host by default |
So on a stock MinIO configured through the UI alone, the file uploads correctly, but
**Test connection** fails and the `download_url` in your `job.completed` webhooks points at a
hostname that does not resolve.
### Workaround: set it through the API
Configure the organisation's storage directly and include `path_style`. This is the same route
the dashboard uses, so the saved configuration is identical in every other respect.
```bash theme={null}
curl -X POST "https://api-dash.tornadoapi.io/api/settings/storage" \
-H "Authorization: Bearer $CLERK_SESSION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"provider": "s3",
"s3": {
"endpoint": "https://minio.internal.example.dev:9000",
"bucket": "tornado-downloads",
"region": "us-east-1",
"access_key_id": "YOUR_ACCESS_KEY",
"secret_access_key": "YOUR_SECRET_KEY",
"path_style": true
}
}'
```
Dry-run it first against the same payload shape, which validates without persisting:
```bash theme={null}
curl -X POST "https://api-dash.tornadoapi.io/api/settings/storage/test" \
-H "Authorization: Bearer $CLERK_SESSION_TOKEN" \
-H "Content-Type: application/json" \
-d '{ "provider": "s3", "s3": { "...": "same object as above" } }'
```
Both routes require an Admin or Owner session token, not an `x-api-key`. Re-saving from the
Settings form afterwards will drop `path_style`, because the form does not send a field it
does not render. Re-run the curl if you edit storage in the UI.
***
## What "Test connection" proves
The probe is a real network call against your real bucket, not a credential format check. It runs
five steps and reports the round-trip latency:
| Step | Call | Permission it proves |
| ---- | ---------------------------------------------------------------- | ------------------------------------------- |
| 1 | `PutObject` to `tornado-connection-test/.txt` (23 bytes) | `s3:PutObject` |
| 2 | `GetObject` on the object just written | `s3:GetObject` |
| 3 | `CreateMultipartUpload` on `.multipart` | `s3:PutObject` again, on the multipart path |
| 4 | `AbortMultipartUpload` on that upload, with no part uploaded | `s3:AbortMultipartUpload` |
| 5 | `DeleteObject` on the probe object | `s3:DeleteObject` |
No part is ever uploaded, so the whole probe moves 23 bytes regardless of how many steps it runs.
Each step names itself in the error string, so a failure tells you which permission is missing
rather than just `AccessDenied`.
It proves:
* The endpoint resolves, the TLS handshake succeeds and the host speaks S3.
* The bucket exists and is reachable at that endpoint with the region you supplied.
* The credentials carry `s3:PutObject`, `s3:GetObject`, `s3:AbortMultipartUpload` and
`s3:DeleteObject` on that bucket.
* The read works **as the identity your download links are signed with**, which is the reason the
read leg exists at all.
* The addressing style currently in effect works for the endpoint.
* The provider actually implements the multipart API. Not every S3-compatible gateway does, and a
gateway that returns `NotImplemented` at step 3 will fail every delivery above 50 MB.
It does not prove:
* That a real delivery will fit. The probe moves 23 bytes and uploads no parts.
* That your policy allows the **delivery prefix**. The probe writes to
`tornado-connection-test/` at the bucket root, so a policy scoped to `videos/*` passes the test
and fails the job.
* That a recipient can fetch the link. Tornado reads from its own network; a bucket policy
conditioned on source IP, or a private MinIO your customers cannot reach, behaves differently
for them.
* That the presigned link is addressed correctly. On a stock MinIO the probe can pass while the
`download_url` points at a hostname that does not resolve. See
[Addressing style](#addressing-style-path-style-vs-virtual-host).
***
## Permissions
### Minimum
Exactly the four actions the probe exercises and delivery uses, and nothing else:
```json Minimum theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::tornado-downloads/*"
}
]
}
```
| Action | Why |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `s3:PutObject` | Every upload, and the probe write. It also authorises `CreateMultipartUpload`, `UploadPart` and `CompleteMultipartUpload`, so those need nothing extra |
| `s3:GetObject` | The presigned download links, which are signed locally with your key and evaluated as your identity when the recipient clicks |
| `s3:DeleteObject` | The probe cleanup, and file deletion from the dashboard |
| `s3:AbortMultipartUpload` | Stopping an upload that failed part-way. Above 50 MB uploads are multipart, and without the abort the uploaded parts stay in the bucket: not objects, so absent from the object listing, and billed as storage |
**Do not add `s3:CreateMultipartUpload`, `s3:UploadPart` or
`s3:CompleteMultipartUpload`.** They are not IAM actions. AWS's multipart permission table maps
all three to `s3:PutObject`, and every S3-compatible provider that accepts AWS-style policy
documents follows the same mapping. `AbortMultipartUpload` is the only step of the lifecycle
with its own action name, which is exactly why it is the one people leave out.
### Recommended
The minimum plus one bucket-level action:
```json Recommended theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::tornado-downloads/*"
},
{
"Effect": "Allow",
"Action": "s3:ListBucketMultipartUploads",
"Resource": "arn:aws:s3:::tornado-downloads"
}
]
}
```
| Added | Why it is worth adding |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `s3:ListBucketMultipartUploads` | Required to call `ListMultipartUploads`, which Tornado uses only to find an upload orphaned by a `CreateMultipartUpload` timeout. Without it that sweep silently does not run and the orphan is billed until something else reaps it. Drop it if your provider does not implement `ListMultipartUploads`, or if the failure mode being billing rather than delivery makes it not worth the argument |
`s3:ListBucket` is not needed at all. Tornado never enumerates the bucket: it writes to a key it
computed itself, signs a URL for that key, and deletes by key.
### Providers with their own dialect
| Provider | How to express the same set |
| ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| MinIO, Wasabi, Scaleway, DigitalOcean Spaces, OVHcloud | AWS-style policy documents. The JSON above works as written; substitute the resource ARN form the vendor uses |
| Backblaze B2 | Application keys use capabilities, not actions. Grant `readFiles`, `writeFiles` and `deleteFiles` on the one bucket, which is the console's **Read and Write** option. There is no separate abort capability: `writeFiles` and `deleteFiles` cover the multipart lifecycle |
**Scope the resource to the whole bucket, not to the delivery prefix.** Delivered files land
under `videos/`, but the connection test writes to `tornado-connection-test/` at the bucket
root. Narrow the policy to `tornado-downloads/videos/*` and the test fails with
`AccessDenied: s3:PutObject` even though real deliveries would have worked.
***
## Troubleshooting
The `error` string returned by the connection test is deliberately compact. For S3 failures it is
`: s3:`, where the operation is whichever of the probe's five steps
failed first: `PutObject`, `GetObject`, `CreateMultipartUpload`, `AbortMultipartUpload` or
`DeleteObject`.
| Error string | What it means | Fix |
| ------------------------------------------------------ | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AccessDenied: s3:PutObject` | Credentials are valid, the policy does not allow writing to this bucket or prefix | Widen the policy to cover `bucket/*`, not just `bucket` |
| `AccessDenied: s3:GetObject` | The write succeeded, reading the same object back did not | Add `s3:GetObject`. This is the permission every presigned download link is evaluated against, so a write-only key now fails here instead of silently 403ing your customers |
| `AccessDenied: s3:CreateMultipartUpload` | Write and read passed, starting a multipart upload did not | The provider maps this to `s3:PutObject`, so an explicit Deny or a resource pattern that excludes `.multipart` is the usual cause. Widen the resource to `bucket/*`. On a gateway that returns `NotImplemented` rather than `AccessDenied`, the multipart API is simply absent and no delivery above 50 MB will work |
| `AccessDenied: s3:AbortMultipartUpload` | The multipart upload started but could not be stopped | Add `s3:AbortMultipartUpload`. It is a distinct action, not covered by `s3:PutObject`. Until it is granted, every delivery that fails part-way leaves billed parts behind that do not show up in the object listing |
| `AccessDenied: s3:DeleteObject` | Everything else succeeded, the cleanup did not | Add `s3:DeleteObject`. Note the probe object is now left behind in `tornado-connection-test/` |
| `InvalidAccessKeyId: s3:PutObject` | The access key is unknown to the provider | Check for a copied newline or a key from a different project or region |
| `SignatureDoesNotMatch: s3:PutObject` | Secret is wrong, or the region does not match the endpoint | Re-copy the secret. Then confirm the region is the one embedded in the endpoint host |
| `NoSuchBucket: s3:PutObject` | The bucket does not exist at this endpoint | Usually the right bucket at the wrong region endpoint. Check both together |
| `PermanentRedirect: s3:PutObject` | The bucket lives in a different region to the endpoint | Use the endpoint for the bucket's own region |
| `Could not connect to the endpoint URL: ...` | DNS or routing failure, verbatim from the SDK | Typo in the host, private MinIO not reachable from the internet, firewall |
| `Connect timeout on endpoint URL: ...` | Host resolves but does not accept the connection within 5s | Port closed or wrong. MinIO is commonly on `:9000` and needs it written explicitly |
| `timeout: no response from storage endpoint after 20s` | The whole probe exceeded its budget | A black-holed endpoint, or a gateway accepting the connection then stalling |
| `internal error during storage test` | See below | Almost always a malformed endpoint |
### "internal error during storage test" means your endpoint is malformed
This is the least informative error and the most common one to hit while setting up a new
provider. It says nothing about URLs, but that is usually exactly what it is.
The S3 client is constructed before the probe's own error handling begins. If the SDK rejects the
endpoint string outright, for example because it contains a space, an illegal character, or an
over-long hostname label left behind by a bad paste, the rejection happens during construction
and is caught by the generic handler, which reports `internal error during storage test` with no
detail.
When you see it, check the endpoint before anything else:
A trailing space inside the value survives the trim if it is in the middle. Retype the host
rather than pasting it.
`https://https://...` is collapsed automatically, but a mangled paste such as
`https://host/.provider.com/` is not repaired.
Host and optional port only.
Provider consoles live on different hostnames to the S3 API. Wasabi's console is
`console.wasabisys.com`, which is not a storage endpoint.
### Test passes, downloads return 403
The probe reads, so a policy with no `s3:GetObject` at all now fails the test rather than passing
it. A green test with 403 links means the read is allowed on `tornado-connection-test/` but not
on the delivery path: look for a policy `Resource` narrowed to something other than `bucket/*`.
Failing that, the link was signed with a key that has since been rotated, which kills a presigned
URL regardless of its expiry.
### Test passes on MinIO but webhook links do not resolve
The presigned URL is virtual-host style and your MinIO does not have `MINIO_DOMAIN` set. Either
set `path_style: true` through the API, or configure `MINIO_DOMAIN` with the matching wildcard
DNS and certificate.
### Job reports Completed but the bucket is empty
This is not an S3-compatibility problem. Storage configured per API key does not apply to other
keys, and a job sent with an unconfigured key silently lands in Tornado's managed storage while
still reporting `Completed`. See [Custom Storage](/features/custom-storage) for the per-key
model.
### "missing required field(s): ..."
Returned by the offline structural fallback, which only runs when the S3 SDK itself is
unavailable on the server. If you see it against production, the response is telling you the
round-trip never happened. Retry, and if it persists it is a Tornado-side problem rather than a
credentials problem.