AWS S3

Overview

Tornado writes every completed file into the bucket you configure, and hands your customers a presigned GET URL for it (the s3_url / download_url field in job status and in the job.completed webhook, valid 24 hours). Both halves of that sentence matter for the IAM policy. The worker needs to write, and the presigned link needs the same identity to be allowed to read. For a long time the connection test only checked the write half, so a policy with one and not the other looked healthy until a customer clicked a link. That gap is closed.

The connection test now checks the read half too. It used to write a probe object and delete it, and nothing else, so a write-only key passed with a green tick and then 403’d every download. The probe now reads the object back and exercises the multipart lifecycle as well, so a missing s3:GetObject or s3:AbortMultipartUpload fails the test by name instead of surfacing weeks later. See what it proves.Why reading matters: a presigned URL carries no permissions of its own. AWS states it plainly, “the credentials used by the presigned URL are those of the AWS user who generated the URL”, so S3 evaluates every download as the identity whose key you gave Tornado. Grant s3:GetObject or every delivered link returns 403 Forbidden while jobs keep reporting Completed.

Before you start

You need:

An AWS account with permission to create an S3 bucket and an IAM policy plus user.
An S3 bucket dedicated to Tornado deliveries (mixing it with unrelated data makes the scoped policy pointless).
An IAM identity with long-term access keys. Tornado stores a key pair and signs its own requests, so an assumed role or an IAM Identity Center session is not usable here: there is no field to hand Tornado a role ARN to assume. AWS’s standing recommendation is to prefer temporary credentials, so treat this key as a workload credential, scope it to this one bucket, and rotate it.
Admin or Owner role in your Tornado organisation. POST /settings/storage is gated on that; ordinary members can read the configuration but not change it.

1. Create the bucket

Pick the Region first

In the S3 console, choose the Region in the navigation bar before you create the bucket. AWS is explicit that a bucket’s Region cannot be changed after creation. Getting it wrong means creating a second bucket.

Create a general purpose bucket

In the left navigation pane choose General purpose buckets, then Create bucket, and give it a name. Bucket names are globally unique within the AWS partition, 3 to 63 characters, lowercase.

Leave the defaults alone

Keep Block Public Access fully enabled (all four settings are on by default) and keep ACLs disabled (Bucket owner enforced). Tornado never needs public objects. Delivery works through presigned URLs, which is exactly the mechanism that makes a private bucket sufficient.Default encryption with SSE-S3 needs no extra IAM permissions. If you switch to SSE-KMS, see the KMS note in Edge cases.

Which Region

The Region you choose sets the round trip between Tornado’s worker and your bucket for every byte of every delivery, and it is fixed for the life of the bucket.

Tornado does not currently publish a fixed Region for its download workers, and we will not invent one here. Two practical rules hold regardless:

Pick the Region closest to whoever consumes the files (your backend, your users, your CDN origin). That is the leg you control and the leg your customers feel.
Once configured, read the real number instead of guessing. “Test connection” reports the measured round trip of the whole probe (write, read back, multipart create and abort, delete), and job status exposes the upload duration separately from the download and mux phases. If the upload phase looks disproportionate, try a bucket in another Region and compare.

2. The IAM policy

Minimum

Exactly the four actions Tornado is known to call against your bucket, and nothing else. Copy this, replace my-tornado-downloads with your bucket name, and save it as a customer managed policy.

Minimum delivery policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "TornadoDelivery",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload"
      ],
      "Resource": "arn:aws:s3:::my-tornado-downloads/*"
    }
  ]
}

Action	Used by	Symptom if missing
`s3:PutObject`	The worker uploading the finished file, and the probe write	Test fails with `AccessDenied: s3:PutObject`
`s3:GetObject`	The presigned download URLs in `GET /jobs/{id}` and in `job.completed` webhooks, and the probe read-back	Test fails with `AccessDenied: s3:GetObject`. Before the probe read this was silent, and every download returned 403
`s3:DeleteObject`	The probe cleanup, and `DELETE /jobs/{id}/file`	Test fails with `AccessDenied: s3:DeleteObject`; file deletion via the API fails
`s3:AbortMultipartUpload`	Stopping an upload that failed part-way, and the probe’s multipart leg	Test fails with `AccessDenied: s3:AbortMultipartUpload`. In delivery the abort is best-effort and its failure is swallowed, so the only other symptom is a storage bill that grows with no visible objects

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "TornadoDelivery",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload"
      ],
      "Resource": "arn:aws:s3:::my-tornado-downloads/*"
    },
    {
      "Sid": "TornadoMultipartOrphanCleanup",
      "Effect": "Allow",
      "Action": "s3:ListBucketMultipartUploads",
      "Resource": "arn:aws:s3:::my-tornado-downloads"
    }
  ]
}

Added	Why it is worth adding
`s3:ListBucketMultipartUploads`	AWS requires it to run `ListMultipartUploads`, which Tornado uses only to find an upload orphaned by a `CreateMultipartUpload` timeout. Without it that sweep silently does not run and the orphan is billed until a lifecycle rule reaps it

Nothing else belongs in this policy. If the second statement is a fight with your security team, drop it: the failure mode is billing, not delivery.

Multipart needs one extra action, and only one. Above 50 MB the worker uploads in parts, and with the streaming path enabled a file of any size may do so. AWS’s multipart permission table maps CreateMultipartUpload, UploadPart and CompleteMultipartUpload all to s3:PutObject, so they need nothing added. Do not add s3:CreateMultipartUpload or s3:CompleteMultipartUpload: those are not IAM actions.Stopping an upload is the exception. AWS puts it under its own action, s3:AbortMultipartUpload, and that is the one to grant. Without it, an upload that fails part-way leaves its uploaded parts behind. AWS is explicit that you keep paying for them: “you are billed for all storage, bandwidth, and requests for this multipart upload and its associated parts” until you complete or stop the upload. Those parts are not objects, so they never appear in the console’s object list. A bucket lifecycle rule using AbortIncompleteMultipartUpload is the belt-and-braces backstop and needs no permission on Tornado’s side.

s3:ListBucket is not required. Tornado never enumerates your bucket: it writes to a key it computed itself, signs a URL for that key, and deletes by key. Leave it out. So are s3:ListMultipartUploadParts (nothing calls ListParts), s3:ListAllMyBuckets, s3:GetBucketLocation, s3:PutObjectAcl and s3:PutObjectTagging.Omitting ListBucket has one side effect worth knowing when you debug. Without it, a request for a key that does not exist comes back as 403 AccessDenied rather than 404 NoSuchKey, because S3 will not confirm the absence of an object you cannot list. So a 403 on a presigned URL means either “no GetObject” or “wrong key”. Check the object exists in the console before you go rewriting the policy.

Scope Resource to the whole bucket, not to the delivery prefix. Delivered files land under videos/, but the connection test writes its probe to tornado-connection-test/ at the bucket root. Narrow the policy to arn:aws:s3:::my-tornado-downloads/videos/* and the test fails with AccessDenied: s3:PutObject even though real deliveries would have worked.If a prefix scope is non-negotiable, list all three: my-tornado-downloads/videos/*, my-tornado-downloads/tornado-connection-test/* and my-tornado-downloads/verify_credentials.txt (the second probe key, written by the job API’s own credential check).

3. Create the user and get the keys

Create an IAM user for Tornado

In the IAM console, create a user (for example tornado-delivery) with no console access. It is a machine identity; it never needs to sign in.“Programmatic access” is no longer a checkbox during user creation. Access keys are created afterwards, from the user’s own security credentials.

Attach the policy

Attach the customer managed policy from step 2 directly to that user, or to a group containing only that user. Attach nothing else.

Create the access key

From the user’s security credentials, create an access key. You get an access key ID (AKIA…) and a secret access key.

The secret access key is shown once, at creation. AWS cannot show it to you again. If you lose it, delete the key and create a new one. A user can hold at most two access keys, which is what makes zero-downtime rotation possible: create the second, update Tornado, then delete the first.

4. Fill in the four fields

The fields Tornado accepts for S3 are exactly bucket, region, access_key_id, secret_access_key, plus two optional ones covered in Edge cases.

Onboarding wizard

On the Destination step, choose the Amazon S3 tile. The next step, Credentials (“Connect your bucket”), shows four inputs:

Label on screen	Field	Value
Bucket name	`bucket`	The bucket name only, no `s3://`, no ARN, no URL
Region	`region`	The bucket’s Region code, for example `eu-west-3`
Access key ID	`access_key_id`	`AKIA…`
Secret access key	`secret_access_key`	The secret from step 3

Then press Test connection.

Or Settings, after onboarding

Dashboard Settings, section Default storage destination, provider Amazon S3 / S3-Compatible. The same four fields appear there, labelled Bucket, Region, Access Key ID, Secret Access Key, with a fifth, Endpoint (optional), which you leave empty for AWS.

The Settings screen has a Save destination button and no “Test connection” button. Saving there does not probe your bucket. The live five-step round trip only runs in the onboarding wizard, or if you call POST /settings/storage/test yourself. After changing credentials in Settings, run the test route or a real job.

Credentials submitted through the dashboard are written to Azure Key Vault, not to the application database. Only the provider name and a vault reference are stored alongside your organisation. Reading the configuration back returns the secret masked. If Key Vault is unreachable, the save is refused with a 503 rather than falling back to plaintext storage.

This page covers the dashboard configuration, which is per organisation. The public API has a separate, older per-API-key route, POST /user/s3, and it names two of these fields differently: access_key and secret_key instead of access_key_id and secret_access_key, with endpoint required rather than optional. See Custom Storage for that surface, including its per-key gotcha.

What “Test connection” proves

Pressing Test connection calls POST /settings/storage/test, which runs a five-step probe against your real bucket and reports the measured latency. Nothing is persisted until it succeeds.

Step	Call	Permission it proves
1	`PutObject` to `tornado-connection-test/<random>.txt` (23 bytes)	`s3:PutObject`
2	`GetObject` on the object just written	`s3:GetObject`
3	`CreateMultipartUpload` on `<same key>.multipart`	`s3:PutObject` again, on the multipart path
4	`AbortMultipartUpload` on that upload, with no part uploaded	`s3:AbortMultipartUpload`
5	`DeleteObject` on the probe object	`s3:DeleteObject`

The multipart leg uploads no parts, so the whole probe still moves 23 bytes. Each step names itself in the error string, so a failure tells you which permission is missing rather than just AccessDenied. The delete runs last, and a failure at step 2, 3 or 4 still triggers a best-effort cleanup of the probe object. It proves:

The access key ID and secret are valid and active.
The bucket exists, is reachable from Tornado, and the Region you typed matches it.
The identity may write (s3:PutObject), read (s3:GetObject), stop a multipart upload (s3:AbortMultipartUpload) and delete (s3:DeleteObject).
That the read works as the identity your download links are signed with, which is the whole reason the read leg exists.
The whole round trip completes inside the 20 second budget (5 s connect, 10 s read, one attempt, no retries).

It does not prove:

That a real delivery will fit. The probe moves 23 bytes; a 2 GB video is 30-odd real UploadPart calls, all authorised by the same s3:PutObject.
That your policy allows the delivery prefix. The probe writes to tornado-connection-test/, so a Deny scoped to videos/* passes the test and fails the job.
That s3:ListBucketMultipartUploads is granted. Nothing in the probe lists uploads.
That a recipient can fetch the link. The probe reads with a signed request from Tornado’s own network; a bucket policy conditioned on aws:SourceIp, aws:SecureTransport or s3:signatureAge can still block your customers.
Anything about lifecycle rules, Object Lock, or KMS grants that only bite on larger or longer-lived objects.

On a versioning-enabled bucket the probe is not invisible. DeleteObject without a version ID writes a delete marker and leaves a noncurrent version behind, so tornado-connection-test/ accumulates one tiny noncurrent object per test. A lifecycle rule expiring noncurrent versions clears them.

Checking a real delivered link

The probe covers s3:GetObject, so this is no longer the gap it was. It is still the only way to check the delivery prefix and the path your customers actually take. Run one real job, take the s3_url from the job status or the job.completed webhook, and fetch a single byte of it:

# 200 or 206 = the link works end to end from outside Tornado.
# 403        = a prefix-scoped Deny, an expired link, or a rotated key.
curl -s -o /dev/null -w '%{http_code}\n' -r 0-0 "$S3_URL"

# Same question, asked directly with the key you gave Tornado.
AWS_ACCESS_KEY_ID=AKIA... AWS_SECRET_ACCESS_KEY=... \
  aws s3api get-object \
    --bucket my-tornado-downloads \
    --key videos/some-delivered-file.mp4 \
    --range bytes=0-0 /dev/null

Troubleshooting

The dashboard shows the backend’s own error string. Match on it exactly.

Error string	What actually happened	Fix
`AccessDenied: s3:PutObject`	The probe write was refused	Policy missing `s3:PutObject`, or the `Resource` list is missing the `arn:aws:s3:::bucket/` line (object actions need the `/` form), or a bucket policy / SCP / permissions boundary has an explicit Deny. Note that a bucket owned by another AWS account also fails this way rather than with `NoSuchBucket`
`AccessDenied: s3:GetObject`	The write succeeded, reading the same object back did not	Add `s3:GetObject`. This is the action every presigned download link depends on, so fix it before you ship: the same policy gap would have made every `s3_url` return 403. Also produced by a KMS key policy that allows `kms:GenerateDataKey` but not `kms:Decrypt` on an SSE-KMS bucket
`AccessDenied: s3:CreateMultipartUpload`	`PutObject` and `GetObject` passed, starting a multipart upload did not	AWS authorises `CreateMultipartUpload` with `s3:PutObject`, so the two failing together points at a condition rather than a missing action: a bucket policy or SCP that denies the multipart APIs specifically, or a `Resource` that covers the probe key but not `<key>.multipart`. Widen `Resource` to `bucket/*`
`AccessDenied: s3:AbortMultipartUpload`	The multipart upload started but could not be stopped	Add `s3:AbortMultipartUpload`. It is a distinct IAM action, not covered by `s3:PutObject`. Until you do, every delivery that fails part-way leaves parts you are billed for and cannot see in the object list
`AccessDenied: s3:DeleteObject`	Everything else succeeded, the cleanup did not	Add `s3:DeleteObject`. Also produced by Object Lock in compliance mode or a retention rule on the bucket, in which case the credentials are fine and the probe prefix is simply undeletable
`NoSuchBucket: s3:PutObject`	No such bucket in that Region for this account	Typo in Bucket name, or you typed a path (`my-bucket/videos`) instead of the bucket name
`InvalidAccessKeyId: s3:PutObject`	AWS does not recognise the key ID	Key deleted or deactivated, or an R2 / MinIO key pasted into the AWS form
`SignatureDoesNotMatch: s3:PutObject`	Key ID is real, secret does not match it	Almost always a truncated paste or trailing whitespace in Secret access key. Re-paste; if you no longer have it, create a new access key
`PermanentRedirect: s3:PutObject` or `AuthorizationHeaderMalformed: s3:PutObject`	The Region you typed is not the bucket’s Region	Set the bucket’s real Region. The bucket’s Region is shown in the S3 console bucket list. Leaving Region blank is the same bug in disguise: the backend falls back to `us-east-1`
`ExpiredToken: s3:PutObject`	You pasted temporary STS credentials	Tornado stores the key pair for months and has no field for a session token. Use a long-term key on a dedicated IAM user
`timeout: no response from storage endpoint after 20s`	Nothing answered within the 20 second budget	With AWS and an empty Endpoint, this points at a bucket only reachable through a VPC endpoint, or an S3 Access Point / firewall that drops traffic from outside your network. If you filled Endpoint, it is almost certainly wrong
`Could not connect to the endpoint URL: …`	DNS or TCP failure before any S3 response	Only reachable when a custom Endpoint is set. Check the host
`Parameter validation failed: Invalid length for parameter Bucket…`	A field was submitted empty or malformed, and the SDK rejected the call locally	Fill the field. No request ever left Tornado
`missing required field(s): …`	The offline structural check ran instead of the live probe	For S3 this only happens when the live probe is unavailable, which is not a production condition. Retry, then contact support
`internal error during storage test`	The probe itself raised something unexpected	Not a credentials problem. Retry once, then contact support with the timestamp
`The test write failed. Double-check the credentials.`	The backend returned a failure with no error string	Rare frontend fallback. Retry; if it repeats, call `POST /settings/storage/test` directly to see the raw response

Jobs complete, but downloads 403

The bucket contains the files, the job says Completed, and every s3_url returns 403 Forbidden / AccessDenied. The probe’s read leg makes a plain missing s3:GetObject much less likely than it used to be, so work down this list in order.

Confirm the object is really there

Open the bucket in the S3 console and find the key from the job payload. If the file is present, the upload path is fine and this is purely a read problem.

Re-run Test connection

Unlike the old two-step probe, it now reads. If it comes back AccessDenied: s3:GetObject, the policy really is missing the action and you are done diagnosing.

If the test passes, suspect the prefix or the key

A green test with 403 links means the read is allowed on tornado-connection-test/ but not on the delivery path. Look for a policy or bucket-policy Resource narrowed to something other than bucket/*, or an access key that was rotated after the link was signed. Presigned URLs die with the key that signed them, expiry or no expiry.

Confirm from outside

Fetch a delivered s3_url as shown in Checking a real delivered link. Tornado reads from its own network; a bucket policy conditioned on source IP or TLS behaves differently for your customers.

Fixing the policy repairs existing links too. A presigned URL is evaluated at request time against the current policy, so a link issued while s3:GetObject was missing starts working the moment you grant it, as long as it has not passed its 24 hour expiry.

Edge cases

Large files and multipart uploads

Uploads above 50 MB are sent as multipart, which is most videos. Do not assume small files never are: with Tornado’s streaming upload path enabled, a copy-only job of any size uses the multipart APIs.The happy path needs no extra permissions. AWS’s multipart permission table maps CreateMultipartUpload, UploadPart and CompleteMultipartUpload all to s3:PutObject. There is no s3:CreateMultipartUpload IAM action to add, and asking for one will just confuse whoever reviews the policy.Stopping a partial upload is the one separate action, s3:AbortMultipartUpload, and it is in the policy above. Without it an interrupted upload leaves its parts behind as an incomplete multipart upload: not an object, so absent from the console’s object list, and billed as storage until something removes it. A bucket lifecycle rule using AbortIncompleteMultipartUpload is a good backstop for the cases Tornado cannot clean up itself, such as a worker that died mid-upload, and it needs no permission on Tornado’s side.Tornado’s own abort is best-effort in the delivery path: if it 403s, the job still succeeds and nothing is logged to you. That is why the connection test now exercises the abort explicitly, so the gap surfaces at configuration time instead of on a storage bill.

Endpoint and path style (fields the S3 tile does not show)

The API accepts two further optional fields, endpoint and path_style.Leave endpoint empty for AWS. With no endpoint set, the SDK derives the correct regional host from Region. The field exists for S3-compatible providers (R2, MinIO, Wasabi, Backblaze B2), which is why the onboarding wizard shows it only on the “Any S3-compatible” tile, and why Settings labels it “Endpoint (optional)”.If you do fill it, the backend normalises it before use: whitespace trimmed, an accidental doubled scheme (https://https://…) collapsed, https:// prepended when absent, trailing slash removed. That defence exists because a double-wrapped endpoint used to fail every connection test with an unresolvable host.path_style forces path-style addressing (host/bucket/key). AWS does not need it and neither UI exposes it; it is reachable through the API only.Requests are always signed with AWS Signature Version 4. This is forced, not defaulted, because SigV2 presigned URLs are rejected outright by Cloudflare R2 and were never supported by Regions launched after 2014.

SSE-KMS encrypted buckets

If the bucket’s default encryption is SSE-KMS rather than SSE-S3, the IAM policy above is not enough on its own. The uploading and downloading identity additionally needs kms:GenerateDataKey and kms:Decrypt on the key, and the KMS key policy must allow that principal.We have not verified this path end to end against a Tornado delivery, so treat it as the standard AWS requirement rather than a tested Tornado configuration: test with a real job before relying on it. SSE-S3 needs no KMS permissions and is the simpler choice.

Rotating the access key

An IAM user can hold two access keys at once, which is the whole rotation mechanism:

Create the second key

On the same user, create a second access key.

Update Tornado

Paste the new pair into Settings and save. In-flight jobs finish with whichever credentials they already resolved.

Watch, then delete

Confirm new jobs deliver and new links download, then delete the old key in IAM. Deactivating it first is the reversible version of the same step.

Presigned URLs signed with the old key stop working the moment that key is deleted or deactivated, even if their 24 hour window has not elapsed. If your customers hold live links, deactivate rather than delete, and wait out the window.

Where files land inside the bucket

By default objects go under a videos/ top-level folder, with any folder_prefix nested inside it and the per-job folder below that. Both are configurable. See Custom Storage for the full path layout, including base_folder.

Custom Storage

All providers, the per-API-key configuration route, folder layout, and inline per-request credentials

Webhooks

The job.completed payload, including the presigned s3_url this policy has to allow

Get Started

Features

Storage guides

Integrations

Overview

Before you start

1. Create the bucket

Which Region

2. The IAM policy

Minimum

Recommended

3. Create the user and get the keys

4. Fill in the four fields

What “Test connection” proves

Checking a real delivered link

Troubleshooting

Jobs complete, but downloads 403

Edge cases

Custom Storage

Webhooks

​Overview

​Before you start

​1. Create the bucket

​Which Region

​2. The IAM policy

​Minimum

​Recommended

​3. Create the user and get the keys

​4. Fill in the four fields

​What “Test connection” proves

​Checking a real delivered link

​Troubleshooting

​Jobs complete, but downloads 403

​Edge cases

​Related

Custom Storage

Webhooks

Overview

Before you start

1. Create the bucket

Which Region

2. The IAM policy

Minimum

Recommended

3. Create the user and get the keys

4. Fill in the four fields

What “Test connection” proves

Checking a real delivered link

Troubleshooting

Jobs complete, but downloads 403

Edge cases

Related