Overview
Tornado can upload files to your own storage or to managed storage. The destination depends on the request, the API key’s saved configuration and any organization default. Custom S3-compatible and OSS endpoints must use HTTPS and a public host. Supply credentials in the dedicated fields; credentials embedded in the endpoint URL are rejected.Tornado has backends for S3-compatible storage, Azure Blob Storage, Google Cloud Storage, Alibaba OSS and Google Drive. Verify your provider’s endpoint, permissions and supported operations with a delivery to your destination.
Supported Providers
AWS S3 / S3-Compatible
AWS S3, Cloudflare R2, MinIO, DigitalOcean Spaces, Backblaze B2, Wasabi, OVH
Azure Blob Storage
Azure Storage Accounts with Blob containers
Google Cloud Storage
GCS buckets with service account authentication
Alibaba OSS
Alibaba Cloud Object Storage Service
Google Drive
Service-account delivery to a Drive folder (see Configure Google Drive)
Among these public per-key endpoints, only S3 has a
GET endpoint to read
back its saved configuration. Dashboard storage has separate read-back
routes and a different contract. For any provider, confirm the destination
with a test job and inspect the object in your storage.storage_provider of s3, blob, gcs or oss instead. It must exist on
the key or organization, and cannot be combined with inline storage. See
Create Job.
S3 / S3-Compatible Storage
Uses the S3 API for AWS S3 and compatible providers. Compatibility depends on the provider’s support for the operations used by the upload path.Supported S3 Providers
Configure S3 Storage
Cloudflare R2 Setup
Cloudflare R2 Setup
1
Create R2 Bucket
In Cloudflare dashboard, go to R2 and create a new bucket.
2
Create API Token
Create an R2 API token with Object Read & Write permissions.
3
Get Account ID
Find your Account ID in the Cloudflare dashboard URL or overview page.
4
Configure Tornado
Required S3 Permissions
s3:ListBucket is not required for these object operations. Uploads,
signing and deletion address known object keys. Multipart cleanup can use
the separate s3:ListBucketMultipartUploads permission.File uploads use multipart above the configured threshold (50 MiB by default);
streaming uploads also use multipart. Without s3:AbortMultipartUpload, an
upload that fails part-way can leave its parts in the bucket. They are not
objects, so they do not appear in the normal object listing. AWS charges for
incomplete parts until they are removed. The three multipart write operations
use s3:PutObject; encryption policies can require additional permissions.
See AWS multipart permissions.For the full minimum and recommended sets, including the optional
s3:ListBucketMultipartUploads on the bucket ARN, see the
AWS S3 and S3-compatible guides.Azure Blob Storage
Use Azure Storage Accounts with Blob containers.Configure Azure Blob
Azure Setup Guide
Azure Setup Guide
1
Create Storage Account
In Azure Portal, go to Storage Accounts > Create.
- Choose Standard performance
- Select Hot access tier
- Keep anonymous blob access disabled; delivery URLs use SAS authorization
2
Create Container
In your Storage Account, go to Containers > + Container.Name it (e.g.,
tornado-downloads).3
Get Access Key
Go to Access keys in your Storage Account.Copy key1 or key2.
4
Configure Tornado
Alternative: SAS Token
POST /user/blob accepts a SAS token instead of the account key.
Azure delivery URLs are supported. With an account key, the service signs a blob SAS URL. With a supplied SAS token, the delivery URL reuses that token. The token must allow reading the blob for a recipient to download it, and its own expiry limits the URL lifetime. A successful upload alone does not prove read access.
account_key or sas_token. If both are supplied, the account key takes precedence.
Required Azure Permissions
For an account SAS, use signed servicesss=b (blob), signed resource
types srt=o (object), and permissions sp=rwd. A container-scoped
service SAS uses sr=c instead of the account-SAS service/resource fields.
The token must cover the generated object names and the probe object.
See account SAS
and service SAS.
- Read (r) - Reading a delivered blob.
- Write (w) - Uploading. Azure defines
was “Create or write content, properties, metadata, or block list”, which covers Put Blob, Put Block and Put Block List. Create (c) on its own is not enough, because a retry overwrites an existing blob. - Delete (d) - Cleanup after the credential probe.
l) is not needed for these uploads, reads and probe cleanup. The
delivery path does not enumerate the container or require append, tag or
immutability-policy permissions. SAS permission names depend on the resource
type; use the provider reference when creating the token.
With the account key instead of a SAS there is nothing to size: the key grants full access to
every container, blob, queue, table and file share in the account, and cannot be narrowed. Give
Tornado a storage account used only for deliveries.
Google Cloud Storage
Use GCS buckets with service account authentication.Configure GCS
GCS Setup Guide
GCS Setup Guide
1
Create GCS Bucket
In Google Cloud Console, go to Cloud Storage > Create Bucket.
- Choose a unique name
- Select your preferred region
- Choose Standard storage class
2
Create Service Account
Go to IAM & Admin > Service Accounts > Create Service Account.Name it (e.g.,
tornado-storage).3
Grant Permissions
Assign the Storage Object Admin role to the service account for your bucket:
4
Download JSON Key
In the service account details, go to Keys > Add Key > Create new key > JSON.Download and save the JSON file.
5
Configure Tornado
Use the JSON key content (minified):
Required GCS Permissions
For the standard JSON upload path, grant these object permissions on the bucket:storage.objects.create- every upload, and the credential probe.storage.objects.get- reading signed downloads and the source parts used by composite uploads. The worker can select composite uploads from 100 MiB, depending on its configuration and concurrency.storage.objects.delete- temp-part cleanup, probe cleanup, and overwriting on retry.
roles/storage.objectUser covers these operations; roles/storage.objectAdmin
also works. roles/storage.objectCreator is insufficient for the required
reads and deletes. If XML multipart upload is enabled by the service, additional
storage.multipartUploads.* permissions are needed; Object User includes them.
See Google’s role definitions
and compose permissions.
See the Google Cloud Storage guide for the full breakdown.
Alibaba Cloud OSS
Alibaba OSS uses its own endpoint and credential format.Configure Alibaba OSS
OSS Endpoint Regions
Folder Prefix
The provider configuration endpoints accept an optionalfolder_prefix:
The folder prefix is placed inside the base folder (
videos/ by default) and combined with any folder parameter you specify in individual job requests.
Google Drive uses the combined path as the file name inside the destination
folder; it does not create the nested folders shown above. Legacy S3 settings
do not retain these optional folder fields.Base Folder
The provider configuration endpoints accept an optionalbase_folder to change
the first segment of the generated object key. It defaults to videos. The
diagrams below describe object storage; Drive keeps these segments in a flat file name.
Examples
Default behavior (nobase_folder specified):
base_folder:
folder_prefix:
If you don’t specify
base_folder, it defaults to videos for backward compatibility. The base_folder is always the top-level folder, with folder_prefix nested inside it.Presigned URLs
When a delivery URL is available, the job statuss3_url field contains it.
It can be null when no usable delivery link is available:
Legacy Endpoint (S3 Only)
The/user/bucket endpoint still works for S3-compatible storage only:
Reset to Default Storage
Use the DELETE endpoint to remove a saved per-key provider entry:Removing one entry leaves other provider entries and any organization default
in place. Those can become the next destination. Legacy S3 configuration is
removed separately with
DELETE /user/bucket; DELETE /user/s3 only removes
the modern S3 entry. Managed storage is used only when no applicable custom
destination remains and no inline destination is provided. These configuration
deletions do not delete existing delivered files.Troubleshooting
Common Errors
Job completed, but the file isn’t in my storage
The job reportsCompleted but the file is nowhere in your bucket/container.
Check which destination was selected: inline credentials, a per-key entry, an
organization default, legacy S3 settings or managed storage. Also check the
generated prefix and filename. Completion alone does not identify the bucket.
1
Confirm which key you used
Check the
x-api-key header on the request that created the job. Storage is
resolved for that key, together with request options and any applicable
organization default.2
Verify that key has storage configured
For S3, read it back with
GET /user/s3. The
other public per-key provider endpoints have no read-back route. Check any
organization default separately, then confirm with a test job.3
Configure storage on the correct key
Re-run the configuration endpoint using the key you actually send jobs with,
then re-run the job.
Testing Your Configuration
After configuring storage, create a test job to verify everything works:Delivery URLs use the configured destination. S3 uses a presigned URL, GCS a V4 signature, Azure a locally signed read SAS or the SAS token supplied by the customer, and OSS its provider signature. A supplied SAS keeps its existing permissions and expiry. Signing a URL does not verify that the object still exists or that a recipient can read it.When the original credentials are unavailable, the API may return a previously recorded URL while it considers it fresh; once that delivery record expires,
s3_url may become null instead of being regenerated against another destination. The single-job status response does not expose a key field. GET /jobs can expose the stored s3_key, and multi-clip responses include each clip’s s3_key. Retain those object identifiers if you need to retrieve files directly from your storage after a URL expires.Inline Storage (Per-Request)
For marketplace users or one-off configurations, you can provide storage credentials directly in the job request:Inline storage credentials:
- Take priority over pre-configured storage
- Are validated before admission, with successful live probes cached for five minutes
- Travel with the queued job; production requires credential encryption to accept this path
- Do not become a reusable per-key configuration for direct users. For marketplace users, the service attempts to save them through Key Vault or encrypted storage for later reuse; a persistence failure can leave the current request accepted without saving the configuration
- Support 4 providers (S3, Azure Blob, GCS, OSS) — Google Drive is available via
/user/gdriveonly, not inline - Support
folder_prefixandbase_folderparameters
Marketplace users (RapidAPI, Apify, Zyla) can use inline credentials, previously
saved credentials, or managed storage when
MARKETPLACE_DEFAULT_STORAGE permits
it (enabled by default). Credentials are not required on every request. If the
managed fallback is disabled and no usable saved configuration exists, provide
inline storage.
See the Marketplace Integration guide for details.Security Best Practices
Use Least Privilege
Use Least Privilege
Create dedicated credentials with only the permissions needed:
- S3: Custom IAM policy with
s3:PutObject,s3:GetObject,s3:DeleteObjectands3:AbortMultipartUploadon one bucket - Azure: The account key cannot be scoped, so narrow the account instead: use a storage account dedicated to Tornado deliveries. A SAS can narrow permissions and scope; include read access and an expiry covering the intended download window. Delivery URLs reuse that SAS.
- GCS: Service account with
roles/storage.objectUseron the specific bucket, granted on the bucket and not the project
Rotate Credentials Regularly
Rotate Credentials Regularly
Update the saved configuration when replacing credentials and verify delivery:
- AWS: Replace the access key, update Tornado, verify a delivery, then retire the old key
- Azure: Replace an expiring SAS before it expires, or swap account keys before retiring the old one
- GCS: Rotate service account keys
Enable Bucket Logging
Enable Bucket Logging
Monitor access to your storage:
- S3: Enable Server Access Logging
- Azure: Enable Storage Analytics
- GCS: Enable Cloud Audit Logs
Storage fallback
For named storage destinations, configure Fallback in the dashboard’s Storage destinations settings. Choose a second destination and a transfer deadline between 30 and 300 seconds (default: 120). Enabling this option authorizes this order: your primary storage, your fallback storage, then TornadoAPI storage. Final storage confirmation can take additional time. Quota failures and cancellations do not trigger fallback; an uncertain final write stops delivery. The selected destinations must belong to your organization and already exist in the storage catalog. New jobs capture the policy; existing jobs keep their original configuration. Without this option, the current storage behavior is unchanged. You can override the policy for one API key usingGET /user/storage/failover
with the x-api-key header, followed by PUT /user/storage/failover:
failover to null disables fallback for that API key.
Reassigning its primary destination in the dashboard resets the key override.
Completed job details and webhooks include delivery_storage. Its failover_tier
is 0 for the primary, 1 for your backup and 2 for TornadoAPI. Always use the
returned download URL because the file may have been delivered to a fallback.
Fallback supports S3-compatible storage, Azure Blob, Google Cloud Storage,
Alibaba OSS and Google Drive, including combinations of different providers.
Each dashboard option displays its provider. The GET response identifies it with
provider on each destination (s3, blob, gcs, oss, gdrive) and includes
primary_provider. Configure the destination’s credentials once; the fallback
policy references its ID. TornadoAPI remains the final destination.