> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tornadoapi.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Custom Storage
> Use your own cloud storage for downloads - S3, Azure Blob, GCS, and more
## Overview
By default, Tornado uploads files to our managed storage. You can configure your own cloud storage to receive downloads directly.
Tornado supports all major S3-compatible providers, Azure Blob Storage, Google Cloud Storage, Alibaba OSS, and Google Drive delivery — giving you flexibility to use the storage solution that best fits your infrastructure.
**Storage is configured per API key, not per account.** Each API key must be
configured with its own storage — a configuration set on one key does **not**
apply to your other keys.
If you submit a job with a key that has **no** storage configured, the file is
uploaded to Tornado's managed storage (**not** your bucket) and the job still
reports as `Completed`. This is the most common cause of "my file isn't in my
storage": the storage was configured on one key, but the job was sent with a
different key.
When you create a new API key, run the storage configuration (`POST /user/s3`,
`/user/blob`, `/user/gcs`, `/user/gdrive`, or `/user/oss`) with **that key's**
`x-api-key` before sending jobs. Confirm with a test job (see
[Testing Your Configuration](#testing-your-configuration)).
## Supported Providers
AWS S3, Cloudflare R2, MinIO, DigitalOcean Spaces, Backblaze B2, Wasabi, OVH
Azure Storage Accounts with Blob containers
GCS buckets with service account authentication
Alibaba Cloud Object Storage Service
Service-account delivery to a Drive folder (see [Configure Google Drive](/api-reference/user/configure-gdrive))
**Google Drive: the destination must be a Shared Drive.** Google documents that "service
accounts don't have storage quota and can't own any files. Instead, they must upload files and
folders into shared drives, or use OAuth 2.0 to upload items on behalf of a human user."
A service account has no small quota of its own to fall back on, so a My Drive folder shared
with the service account fails at upload with `403 storageQuotaExceeded`, however much free
space the folder's owner has. Tornado does not use domain-wide delegation, so the OAuth
alternative in Google's sentence is not available either. See the
[Google Drive guide](/storage/google-drive).
Each provider has its own dedicated API endpoint:
| Provider | Configure | Check | Remove |
| -------------------- | ----------------------------------------------------------- | -------------------------------------------- | ---------------------------------------------------------- |
| S3 / S3-Compatible | [`POST /user/s3`](/api-reference/user/configure-s3) | [`GET /user/s3`](/api-reference/user/get-s3) | [`DELETE /user/s3`](/api-reference/user/delete-s3) |
| Azure Blob | [`POST /user/blob`](/api-reference/user/configure-blob) | — | [`DELETE /user/blob`](/api-reference/user/delete-blob) |
| Google Cloud Storage | [`POST /user/gcs`](/api-reference/user/configure-gcs) | — | [`DELETE /user/gcs`](/api-reference/user/delete-gcs) |
| Google Drive | [`POST /user/gdrive`](/api-reference/user/configure-gdrive) | — | [`DELETE /user/gdrive`](/api-reference/user/delete-gdrive) |
| Alibaba OSS | [`POST /user/oss`](/api-reference/user/configure-oss) | — | [`DELETE /user/oss`](/api-reference/user/delete-oss) |
Only S3 currently has a "Check" (`GET`) endpoint to read back the saved configuration. The other providers don't have one yet — if you need to confirm what's configured for Blob/GCS/Drive/OSS today, the only way is to run a test job and inspect where the file lands.
***
## S3 / S3-Compatible Storage
Works with AWS S3 and any S3-compatible provider.
### Supported S3 Providers
| Provider | Endpoint Format |
| ------------------- | ----------------------------------------------- |
| AWS S3 | `https://s3.{region}.amazonaws.com` |
| Cloudflare R2 | `https://{account_id}.r2.cloudflarestorage.com` |
| DigitalOcean Spaces | `https://{region}.digitaloceanspaces.com` |
| Backblaze B2 | `https://s3.{region}.backblazeb2.com` |
| Wasabi | `https://s3.{region}.wasabisys.com` |
| MinIO | `https://your-minio-server.com` |
| OVH Object Storage | `https://s3.{region}.io.cloud.ovh.net` |
### Configure S3 Storage
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/s3" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"endpoint": "https://s3.us-east-1.amazonaws.com",
"bucket": "my-tornado-downloads",
"region": "us-east-1",
"access_key": "AKIAIOSFODNN7EXAMPLE",
"secret_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"folder_prefix": "downloads/"
}'
```
In Cloudflare dashboard, go to **R2** and create a new bucket.
Create an R2 API token with **Object Read & Write** permissions.
Find your Account ID in the Cloudflare dashboard URL or overview page.
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/s3" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"endpoint": "https://YOUR_ACCOUNT_ID.r2.cloudflarestorage.com",
"bucket": "my-downloads",
"region": "auto",
"access_key": "YOUR_R2_ACCESS_KEY_ID",
"secret_key": "YOUR_R2_SECRET_ACCESS_KEY"
}'
```
### Required S3 Permissions
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::my-tornado-downloads/*"
}
]
}
```
**`s3:ListBucket` is not required and used to be listed here in error.** Nothing in Tornado
enumerates your bucket: it writes to a key it computed itself, signs a URL for that key, and
deletes by key.
`s3:AbortMultipartUpload` replaces it. Above 50 MB uploads are multipart, and without the
abort permission an upload that fails part-way leaves its parts in the bucket. They are not
objects, so they never appear in the object listing, and AWS bills them as storage until
something removes them. `CreateMultipartUpload`, `UploadPart` and `CompleteMultipartUpload`
need nothing added: AWS maps all three to `s3:PutObject`.
For the full minimum and recommended sets, including the optional
`s3:ListBucketMultipartUploads` on the bucket ARN, see the
[AWS S3](/storage/aws-s3) and [S3-compatible](/storage/s3-compatible) guides.
***
## Azure Blob Storage
Use Azure Storage Accounts with Blob containers.
### Configure Azure Blob
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/blob" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"account_name": "mystorageaccount",
"container": "tornado-downloads",
"account_key": "your-storage-account-key-base64...",
"folder_prefix": "videos/"
}'
```
In Azure Portal, go to **Storage Accounts** > **Create**.
* Choose **Standard** performance
* Select **Hot** access tier
* Enable **Blob public access** if needed for direct URLs
In your Storage Account, go to **Containers** > **+ Container**.
Name it (e.g., `tornado-downloads`).
Go to **Access keys** in your Storage Account.
Copy **key1** or **key2**.
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/blob" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"account_name": "mystorageaccount",
"container": "tornado-downloads",
"account_key": "xxxxxxxxxxxxxxxxxxx=="
}'
```
### Alternative: SAS Token
`POST /user/blob` accepts a SAS token instead of the account key.
**A SAS-only configuration delivers files that nobody can be handed a link to.** Uploads work.
Download URLs do not exist: minting the 24-hour download SAS requires the account key, and the
job API's own Azure presign path is not implemented. `job.completed` webhooks arrive with no
`s3_url` / `download_url` field at all.
This is not a "more granular permissions" alternative to the account key. It is a trade of all
your download links for a narrower credential. Use the account key unless you have somewhere
else to get links from.
It is also **not accepted on the dashboard route**. `POST /settings/storage` takes
`account_name`, `container_name` and `account_key`, and has no `sas_token` field. See the
[Azure Blob guide](/storage/azure-blob).
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/blob" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"account_name": "mystorageaccount",
"container": "tornado-downloads",
"sas_token": "sv=2022-11-02&ss=b&srt=o&sp=rwd&se=2026-01-01..."
}'
```
Provide either `account_key` OR `sas_token`, not both.
### Required Azure Permissions
Sized for what Tornado actually calls, a SAS needs signed services `b` (blob), signed resource
types `o` (object), and the permission string **`rwd`**:
* **Read** (r) - Reading a delivered blob.
* **Write** (w) - Uploading. Azure defines `w` as "Create or write content, properties, metadata,
or block list", which covers Put Blob, Put Block and Put Block List. Create (`c`) on its own is
not enough, because a retry overwrites an existing blob.
* **Delete** (d) - Cleanup after the credential probe.
**List (`l`) is not needed.** Nothing in the delivery path enumerates the container, and `l` is
not even a valid permission on a blob-scoped SAS. Neither are `a` (add), `t` (tags), `i` or `p`.
With the account key instead of a SAS there is nothing to size: the key grants full access to
every container, blob, queue, table and file share in the account, and cannot be narrowed. Give
Tornado a storage account used only for deliveries.
***
## Google Cloud Storage
Use GCS buckets with service account authentication.
### Configure GCS
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/gcs" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"project_id": "my-gcp-project",
"bucket": "tornado-downloads",
"service_account_json": "{\"type\":\"service_account\",\"project_id\":\"my-gcp-project\",...}",
"folder_prefix": "videos/"
}'
```
In Google Cloud Console, go to **Cloud Storage** > **Create Bucket**.
* Choose a unique name
* Select your preferred region
* Choose **Standard** storage class
Go to **IAM & Admin** > **Service Accounts** > **Create Service Account**.
Name it (e.g., `tornado-storage`).
Assign the **Storage Object Admin** role to the service account for your bucket:
```bash theme={null}
gsutil iam ch \
serviceAccount:tornado-storage@PROJECT.iam.gserviceaccount.com:objectAdmin \
gs://tornado-downloads
```
In the service account details, go to **Keys** > **Add Key** > **Create new key** > **JSON**.
Download and save the JSON file.
Use the JSON key content (minified):
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/gcs" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"project_id": "my-gcp-project",
"bucket": "tornado-downloads",
"service_account_json": "{\"type\":\"service_account\",\"project_id\":\"my-gcp-project\",\"private_key_id\":\"...\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\n...\\n-----END PRIVATE KEY-----\\n\",\"client_email\":\"tornado-storage@my-gcp-project.iam.gserviceaccount.com\",\"client_id\":\"...\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\"}"
}'
```
**Do not base64-encode the `service_account_json` field.** Earlier versions of this page said
you could. You cannot: the value is handed straight to a JSON parser, so a base64 blob fails
with *"Invalid JSON in service account credentials"*.
Pass the JSON as an escaped string. If you are building the payload programmatically, let your
JSON serialiser do the escaping rather than escaping it by hand, which is the other common way
the `private_key` newlines get mangled.
### Required GCS Permissions
Three object permissions, granted **on the bucket** and not on the project:
* `storage.objects.create` - every upload, and the credential probe.
* `storage.objects.get` - signed download URLs, and `objects.compose` reading its own source
parts on files of 100 MB or more.
* `storage.objects.delete` - temp-part cleanup, probe cleanup, and overwriting on retry.
`roles/storage.objectUser` is the smallest predefined role that covers them;
`roles/storage.objectAdmin` also works. **`roles/storage.objectCreator` does not**: it carries
`create` and nothing else, so files of 100 MB or more fail and every download link returns 403.
See the [Google Cloud Storage guide](/storage/google-cloud-storage) for the full breakdown.
***
## Alibaba Cloud OSS
Alibaba OSS uses its own endpoint and credential format.
### Configure Alibaba OSS
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/user/oss" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"endpoint": "https://oss-cn-hangzhou.aliyuncs.com",
"bucket": "tornado-downloads",
"access_key_id": "your-oss-access-key-id",
"access_key_secret": "your-oss-access-key-secret",
"folder_prefix": "videos/"
}'
```
### OSS Endpoint Regions
| Region | Endpoint |
| ---------------- | ----------------------------------------- |
| China (Hangzhou) | `https://oss-cn-hangzhou.aliyuncs.com` |
| China (Shanghai) | `https://oss-cn-shanghai.aliyuncs.com` |
| China (Beijing) | `https://oss-cn-beijing.aliyuncs.com` |
| Singapore | `https://oss-ap-southeast-1.aliyuncs.com` |
| US West | `https://oss-us-west-1.aliyuncs.com` |
| Germany | `https://oss-eu-central-1.aliyuncs.com` |
***
## Folder Prefix
All providers support an optional `folder_prefix` to organize your downloads:
```json theme={null}
{
"folder_prefix": "tornado/downloads/2024/"
}
```
Files will be uploaded to:
```
your-bucket/
└── videos/
└── tornado/
└── downloads/
└── 2024/
├── video-title-1.mp4
└── video-title-2.mp4
```
The folder prefix is placed inside the base folder (`videos/` by default) and combined with any `folder` parameter you specify in individual job requests.
***
## Base Folder
All providers support an optional `base_folder` parameter to change the top-level folder where files are organized. By default, files are placed inside a `videos/` folder.
```json theme={null}
{
"base_folder": "media"
}
```
The full path structure is:
```
your-bucket/
└── [base_folder]/ ← Configurable (default: "videos")
└── [folder_prefix]/
└── [job folder]/
└── filename.ext
```
### Examples
**Default behavior** (no `base_folder` specified):
```
your-bucket/videos/my-video.mp4
your-bucket/videos/my-folder/my-video.mp4
```
**Custom `base_folder`**:
```json theme={null}
{
"base_folder": "downloads"
}
```
```
your-bucket/downloads/my-video.mp4
your-bucket/downloads/my-folder/my-video.mp4
```
**Combined with `folder_prefix`**:
```json theme={null}
{
"folder_prefix": "alex/test/",
"base_folder": "premier"
}
```
```
your-bucket/premier/alex/test/my-video.mp4
your-bucket/premier/alex/test/my-folder/my-video.mp4
```
If you don't specify `base_folder`, it defaults to `videos` for backward compatibility. The `base_folder` is always the top-level folder, with `folder_prefix` nested inside it.
***
## Presigned URLs
When you poll job status, the `s3_url` field contains a presigned/signed URL for your bucket:
```json theme={null}
{
"status": "Completed",
"s3_url": "https://my-bucket.s3.amazonaws.com/video.mp4?X-Amz-Signature=..."
}
```
| Provider | URL Format | Validity |
| ----------- | ---------------- | -------- |
| S3/R2 | AWS Signature V4 | 24 hours |
| Azure Blob | SAS URL | 24 hours |
| GCS | Signed URL V4 | 24 hours |
| Alibaba OSS | OSS Signature | 24 hours |
***
## Legacy Endpoint (S3 Only)
The `/user/bucket` endpoint still works for S3-compatible storage only:
```bash theme={null}
# Legacy endpoint (S3 only)
curl -X POST "https://api.tornadoapi.io/user/bucket" \
-H "x-api-key: sk_your_api_key" \
-d '{"endpoint": "...", "bucket": "...", "region": "...", "access_key": "...", "secret_key": "..."}'
```
The `/user/bucket` endpoint is **deprecated**. Use [`/user/s3`](/api-reference/user/configure-s3) for all new S3 integrations, or the provider-specific endpoints for other cloud providers.
***
## Reset to Default Storage
To switch back to Tornado's managed storage, use the DELETE endpoint for your provider:
```bash theme={null}
# Remove S3 storage
curl -X DELETE "https://api.tornadoapi.io/user/s3" \
-H "x-api-key: sk_your_api_key"
# Remove Azure Blob storage
curl -X DELETE "https://api.tornadoapi.io/user/blob" \
-H "x-api-key: sk_your_api_key"
# Remove GCS storage
curl -X DELETE "https://api.tornadoapi.io/user/gcs" \
-H "x-api-key: sk_your_api_key"
# Remove Google Drive delivery
curl -X DELETE "https://api.tornadoapi.io/user/gdrive" \
-H "x-api-key: sk_your_api_key"
# Remove Alibaba OSS storage
curl -X DELETE "https://api.tornadoapi.io/user/oss" \
-H "x-api-key: sk_your_api_key"
```
After removing, all new downloads will use Tornado's managed storage. Existing files in your custom storage remain untouched.
***
## Troubleshooting
### Common Errors
| Error | Cause | Solution |
| --------------------------------------------- | ------------------------------ | ----------------------------------- |
| `Credential validation failed: Access Denied` | Invalid credentials | Verify access key/secret/token |
| `Credential validation failed: NoSuchBucket` | Bucket/container doesn't exist | Create the bucket first |
| `Credential validation failed: timeout` | Endpoint unreachable | Check endpoint URL and network |
| `Invalid service account JSON` | Malformed GCS credentials | Validate JSON format |
| `Account key must be valid Base64` | Azure key format error | Copy the full key from Azure Portal |
### Job completed, but the file isn't in my storage
The job reports `Completed` but the file is nowhere in your bucket/container.
Almost always, the job was submitted with an API key that has **no storage
configured**, so it was uploaded to Tornado's managed storage instead of yours.
Remember that **storage is configured per API key, not per account** — a config
on one key does not apply to another.
Check the `x-api-key` header on the request that created the job. Storage is
resolved from *that* key only.
For S3, read it back with [`GET /user/s3`](/api-reference/user/get-s3). The
other providers have no read-back endpoint yet, so confirm with a test job.
Re-run the configuration endpoint using the key you actually send jobs with,
then re-run the job.
### Testing Your Configuration
After configuring storage, create a test job to verify everything works:
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/jobs" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"max_resolution": "360"
}'
```
If the job completes successfully with a valid `s3_url`, your storage is configured correctly.
***
## Inline Storage (Per-Request)
For marketplace users or one-off configurations, you can provide storage credentials directly in the job request:
```bash theme={null}
curl -X POST "https://api.tornadoapi.io/jobs" \
-H "x-api-key: sk_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"format": "mp4",
"storage": {
"provider": "s3",
"endpoint": "https://s3.amazonaws.com",
"bucket": "my-videos",
"region": "us-east-1",
"access_key": "AKIAIOSFODNN7EXAMPLE",
"secret_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
}'
```
Inline storage credentials:
* Take priority over pre-configured storage
* Are validated before the job is accepted
* Are **never logged**. For direct API users they are not persisted; for marketplace users they are stored **encrypted** and reused automatically on subsequent requests
* Support 4 providers (S3, Azure Blob, GCS, OSS) — Google Drive is available via [`/user/gdrive`](/api-reference/user/configure-gdrive) only, not inline
* Support `folder_prefix` and `base_folder` parameters
For API marketplace users (RapidAPI, Apify, Zyla), inline storage credentials are **required** for every request.
See the [Marketplace Integration guide](/integrations/marketplace) for details.
***
## Security Best Practices
Create dedicated credentials with only the permissions needed:
* **S3**: Custom IAM policy with `s3:PutObject`, `s3:GetObject`, `s3:DeleteObject` and `s3:AbortMultipartUpload` on one bucket
* **Azure**: The account key cannot be scoped, so narrow the account instead: use a storage account dedicated to Tornado deliveries. A SAS is narrower but costs you every download link
* **GCS**: Service account with `roles/storage.objectUser` on the specific bucket, granted on the bucket and not the project
Set up credential rotation:
* AWS: Use IAM Access Analyzer
* Azure: Set SAS token expiration
* GCS: Rotate service account keys
Monitor access to your storage:
* S3: Enable Server Access Logging
* Azure: Enable Storage Analytics
* GCS: Enable Cloud Audit Logs