Appearance
File Upload API — Customer Integration Guide
Version: 1.0.0 — Initial release
Overview
The Opera ADs File Upload API provides resumable, large-file uploads.
It implements the TUS resumable upload protocol so uploads can survive network failures and continue from where they left off.
Base URL: https://<service-host>/upload
Set this shell variable once before running any of the curl examples below:
bash
export UPLOAD_HOST="https://<host>" # replace with the host provided by Opera teamAuthentication
Every request must be authenticated, except OPTIONS /upload/files which is public. Two methods are supported — choose one.
Option A: API Key
Include the API key in every request header:
X-API-Key: adx_<your_key>API keys are static credentials issued by the Opera team. Contact Opera support to rotate them if compromised.
Option B: HMAC Signature
Per-request signatures prevent replay attacks and do not require a long-lived secret to travel with every request.
Required headers:
| Header | Description |
|---|---|
X-HMAC-Key-Id | Your HMAC key ID, provided by Opera team (format: ADX_HMAC_...) |
X-HMAC-Timestamp | Current Unix timestamp (seconds). Request is rejected if ` |
X-HMAC-Signature | HMAC-SHA256(secret, message) encoded as lowercase hex |
Signature message format:
{METHOD}\n{PATH}\n{TIMESTAMP}Example for PATCH /upload/files/abc123 at timestamp 1700000000:
PATCH\n/upload/files/abc123\n1700000000Python example:
python
import hmac, hashlib, time, requests
HOST = "https://<host>"
KEY_ID = "ADX_HMAC_K7MN2PQR4XVBXXXXXXXXXXXX" # provided by Opera team
SECRET = "your_hmac_secret" # provided by Opera team
def sign(method: str, path: str) -> dict:
ts = str(int(time.time()))
message = f"{method}\n{path}\n{ts}"
sig = hmac.new(SECRET.encode(), message.encode(), hashlib.sha256).hexdigest()
return {"X-HMAC-Key-Id": KEY_ID, "X-HMAC-Timestamp": ts, "X-HMAC-Signature": sig}
headers = sign("POST", "/upload/files")
resp = requests.post(f"{HOST}/upload/files", headers={
**headers,
"Upload-Length": str(file_size),
"Upload-Metadata": f"key {b64(object_key)},filename {b64(filename)}",
"Tus-Resumable": "1.0.0",
"Content-Length": "0",
})Go example:
go
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"fmt"
"time"
)
const (
KEY_ID = "ADX_HMAC_K7MN2PQR4XVBXXXXXXXXXXXX" // provided by Opera team
SECRET = "your_hmac_secret" // provided by Opera team
)
func signRequest(method, path string) (keyID, timestamp, sig string) {
timestamp = fmt.Sprintf("%d", time.Now().Unix())
message := method + "\n" + path + "\n" + timestamp
mac := hmac.New(sha256.New, []byte(SECRET))
mac.Write([]byte(message))
return KEY_ID, timestamp, hex.EncodeToString(mac.Sum(nil))
}JavaScript example:
js
const crypto = require("crypto");
const KEY_ID = "ADX_HMAC_K7MN2PQR4XVBXXXXXXXXXXXX"; // provided by Opera team
const SECRET = "your_hmac_secret"; // provided by Opera team
function sign(method, path) {
const ts = String(Math.floor(Date.now() / 1000));
const message = `${method}\n${path}\n${ts}`;
const sig = crypto.createHmac("sha256", SECRET).update(message).digest("hex");
return { "X-HMAC-Key-Id": KEY_ID, "X-HMAC-Timestamp": ts, "X-HMAC-Signature": sig };
}Uploading Files (TUS Protocol)
The TUS protocol uses three requests: Create → Upload chunks → (optional) Resume.
Step 0 — Capability discovery (optional)
OPTIONS /upload/files
OPTIONS /upload/files/:idOptional — use this to confirm the server's supported protocol version and maximum file size before starting an upload. No authentication required.
bash
curl -i -X OPTIONS "$UPLOAD_HOST/upload/files"Response 200 OK:
Tus-Version: 1.0.0
Tus-Resumable: 1.0.0
Tus-Extension: creation,creation-with-upload,termination,concatenation,creation-defer-length
Tus-Max-Size: 4294967296| Response header | Meaning |
|---|---|
Tus-Version | TUS protocol versions supported by the server |
Tus-Resumable | Version to use in all subsequent requests |
Tus-Extension | TUS extensions enabled on this server |
Tus-Max-Size | Maximum upload size in bytes (4 GB) |
Step 1 — Create an upload session
POST /upload/filesRequired headers:
| Header | Value |
|---|---|
Tus-Resumable | 1.0.0 |
Upload-Length | Total file size in bytes |
Upload-Metadata | Comma-separated name base64value pairs (see below) |
Content-Length | 0 |
Upload-Metadata fields:
| Field | Required | Description |
|---|---|---|
key | Yes | Destination path for the file, relative to your allocated prefix, e.g. 20250106/file.csv.gz |
filename | No | Display name; defaults to basename of key |
The key field determines where the file is stored under your allocated storage prefix.
Rules:
- Leading
/is stripped automatically - Path traversal (
..) is rejected with400 - The value must be non-empty
Example request:
bash
KEY=$(echo -n "20250106/file.csv.gz" | base64 -w 0)
FILENAME=$(echo -n "file.csv.gz" | base64 -w 0)
curl -i "$UPLOAD_HOST/upload/files" \
-X POST \
-H "X-API-Key: adx_yourkey" \
-H "Tus-Resumable: 1.0.0" \
-H "Upload-Length: 104857600" \
-H "Upload-Metadata: key $KEY,filename $FILENAME" \
-H "Content-Length: 0"Successful response 201 Created:
Location: https://<host>/upload/files/a1b2c3d4e5f6...
Tus-Resumable: 1.0.0
Upload-Offset: 0| Response header | Meaning |
|---|---|
Location | Upload session URL — save this; all subsequent PATCH / HEAD / DELETE requests use it |
Tus-Resumable | TUS protocol version confirmed by the server |
Upload-Offset | Always 0 on creation |
Error responses:
| HTTP Status | Cause |
|---|---|
400 | Missing or invalid Upload-Metadata (missing key, path traversal, empty key) |
401 | Missing or invalid authentication |
413 | Upload-Length exceeds the server limit (4 GB) |
Step 2 — Upload chunks
PATCH {Location}Required headers:
| Header | Value |
|---|---|
Tus-Resumable | 1.0.0 |
Content-Type | application/offset+octet-stream |
Upload-Offset | Byte offset where this chunk starts (0 for first chunk) |
Content-Length | Size of this chunk in bytes |
Example — upload entire file in one request:
bash
UPLOAD_URL="$UPLOAD_HOST/upload/files/a1b2c3d4e5f6..."
curl -i "$UPLOAD_URL" \
-X PATCH \
-H "X-API-Key: adx_yourkey" \
-H "Tus-Resumable: 1.0.0" \
-H "Content-Type: application/offset+octet-stream" \
-H "Upload-Offset: 0" \
-H "Content-Length: 104857600" \
--data-binary @file.csv.gzResponse 204 No Content:
Upload-Offset: 104857600| Response header | Meaning |
|---|---|
Upload-Offset | Total bytes the server has received so far; use this as the next chunk's Upload-Offset |
When Upload-Offset equals Upload-Length, the upload is complete.
Error responses:
| Status | Condition |
|---|---|
401 Unauthorized | Missing or invalid credentials |
403 Forbidden | Upload ID exists but belongs to a different customer |
404 Not Found | Upload ID not found |
409 Conflict | Upload already complete, or Upload-Offset does not match the server's current offset |
Step 3 — Resume after failure (optional)
Check the current offset:
HEAD {Location}bash
curl -I "$UPLOAD_URL" \
-H "X-API-Key: adx_yourkey" \
-H "Tus-Resumable: 1.0.0"Response 200 OK:
Upload-Length: 104857600
Upload-Offset: 52428800
Upload-Metadata: customer-name <base64>,filename <base64>,key <base64>| Response header | Meaning |
|---|---|
Upload-Offset | Bytes the server has durably received — start the next PATCH from this offset |
Upload-Length | Total file size declared at session creation |
Upload-Metadata | Metadata supplied at session creation, base64-encoded per field; customer-name is injected by the server |
If the upload is already complete, the server returns 200 OK with Upload-Offset equal to Upload-Length — no further PATCH is needed.
Then send a PATCH starting from Upload-Offset.
Error responses:
| Status | Condition |
|---|---|
401 Unauthorized | Missing or invalid credentials |
403 Forbidden | Upload ID exists but belongs to a different customer |
404 Not Found | Upload ID not found |
Note — HEAD may be slow after an interrupted upload
If the previous PATCH was cut short by a network failure, the server needs time to finalize, synchronize, and clean up resources before it can report a consistent offset. The HEAD request may block for up to 1–3 minutes in this case — this is expected. Do not treat a slow HEAD response as an error; wait for it to complete before resuming.
Cancel an upload
DELETE {Location}bash
curl -i "$UPLOAD_URL" \
-X DELETE \
-H "X-API-Key: adx_yourkey" \
-H "Tus-Resumable: 1.0.0"Response 204 No Content
The operation is idempotent: if the upload is already complete or cancelled the server returns 204 without taking any further action.
Error responses
| Status | Condition |
|---|---|
401 Unauthorized | Missing or invalid X-API-Key / HMAC credentials |
403 Forbidden | Upload ID exists but belongs to a different customer |
404 Not Found | Upload ID not found |
Chunked Upload with Retry and Resume
Algorithm (pseudocode)
The TUS protocol guarantees that the server's Upload-Offset is always the authoritative resume point. The pattern below works for both a fresh upload and resuming after any interruption:
function upload_file(path, key):
file_size = stat(path)
upload_url = POST /upload/files (Upload-Length: file_size, Upload-Metadata: key)
offset = 0
while offset < file_size:
chunk = read(path, offset, min(CHUNK_SIZE, file_size - offset))
for attempt = 1 .. MAX_RETRIES:
try:
offset = PATCH upload_url (Upload-Offset: offset, body: chunk)
break # chunk accepted — move to next
except NetworkError | HTTP 5xx:
if attempt == MAX_RETRIES: raise
sleep(min(2^(attempt-1), 32)) # 1 s → 2 s → 4 s → … → 32 s
# HEAD asks the server for the confirmed offset.
# After a mid-chunk interruption the server may take 1–3 minutes
# to finalize and clean up before responding — this is expected;
# keep retrying HEAD with a generous timeout until it succeeds.
offset = HEAD upload_url (timeout=240, retry until success)
chunk = read(path, offset, min(CHUNK_SIZE, file_size - offset))
return upload_url # save this URL; pass it to resume_upload() if process is killed
function resume_upload(upload_url, path):
file_size = stat(path)
# The server may need time to finalize resources after an interruption.
# Use a generous timeout and retry HEAD until the server responds.
offset = HEAD upload_url (timeout=240, retry until success)
# continue with the same retry loop starting from offset
...Full Python Example
The script below is production-ready. It supports both HMAC and API key authentication (HMAC preferred), uploads files in 50 MB chunks, and handles network failures with exponential backoff and automatic resume. It can also be used to query the status of any upload by its session URL.
Save it as upload.py and invoke it from the command line — no external dependencies beyond requests.
python
#!/usr/bin/env python3
"""Opera ADs File Upload CLI — upload, resume, or query status.
Usage:
# Upload with HMAC (preferred):
python upload.py --hmac-key-id ADX_HMAC_... --hmac-secret <secret> \\
--file /data/file.csv.gz --key 20260401/file.csv.gz
# Upload with API key:
python upload.py --api-key adx_... --file /data/file.csv.gz --key 20260401/file.csv.gz
# Resume an interrupted upload (use the URL printed during the original upload):
python upload.py --hmac-key-id ADX_HMAC_... --hmac-secret <secret> \\
--file /data/file.csv.gz --resume https://<host>/upload/files/<id>
# Query upload status:
python upload.py --hmac-key-id ADX_HMAC_... --hmac-secret <secret> \\
--query https://<host>/upload/files/<id>
"""
import argparse, base64, hashlib, hmac as hmac_mod, os, sys, time
import requests
from requests.exceptions import RequestException
from urllib.parse import urlparse
DEFAULT_HOST = "https://<host>"
DEFAULT_MAX_RETRIES = 10
CHUNK_SIZE = 50 * 1024 * 1024 # 50 MB
STATUS_LABELS = {0: "uploading", 1: "complete", 2: "failed", 3: "cancelled"}
# ── helpers ───────────────────────────────────────────────────────────────────
def _make_auth(api_key: str = "", hmac_key_id: str = "", hmac_secret: str = ""):
"""Return a callable auth(method, path) -> dict that produces request auth headers.
HMAC is used when hmac_key_id and hmac_secret are both provided; falls back to API key.
"""
if hmac_key_id and hmac_secret:
def _sign(method: str, path: str) -> dict:
ts = str(int(time.time()))
msg = f"{method}\n{path}\n{ts}"
sig = hmac_mod.new(hmac_secret.encode(), msg.encode(), hashlib.sha256).hexdigest()
return {"X-HMAC-Key-Id": hmac_key_id, "X-HMAC-Timestamp": ts, "X-HMAC-Signature": sig}
return _sign
def _key(method: str, path: str) -> dict:
return {"X-API-Key": api_key}
return _key
def _urlpath(url: str) -> str:
return urlparse(url).path
def _b64(s: str) -> str:
return base64.b64encode(s.encode()).decode()
def _create_upload(host: str, auth, key: str, file_size: int) -> str:
resp = requests.post(
f"{host}/upload/files",
headers={
**auth("POST", "/upload/files"),
"Tus-Resumable": "1.0.0",
"Upload-Length": str(file_size),
"Upload-Metadata": f"key {_b64(key)},filename {_b64(os.path.basename(key))}",
"Content-Length": "0",
},
)
resp.raise_for_status()
url = resp.headers["Location"]
print(f"\n\033[1;33m>>> Upload session URL: {url}\033[0m")
print("\033[1;33m>>> SAVE THIS URL — you will need it to resume if the upload is interrupted\033[0m\n")
return url
def _get_offset(auth, upload_url: str) -> int:
"""HEAD with retry — server may take 1–3 min to finalize after an interrupted chunk."""
for attempt in range(1, 10):
try:
resp = requests.head(
upload_url,
headers={**auth("HEAD", _urlpath(upload_url)), "Tus-Resumable": "1.0.0"},
timeout=240,
)
resp.raise_for_status()
return int(resp.headers["Upload-Offset"])
except KeyboardInterrupt:
raise
except RequestException as exc:
# Retry on any network error (timeout, DNS failure, connection reset, etc.)
wait = min(30 * attempt, 120)
print(f" HEAD failed (attempt {attempt}: {exc}) — retry in {wait}s…")
time.sleep(wait)
raise RuntimeError("Server did not respond to HEAD after multiple attempts")
def _patch_chunk(auth, upload_url: str, offset: int, data: bytes) -> int:
resp = requests.patch(
upload_url,
headers={
**auth("PATCH", _urlpath(upload_url)),
"Tus-Resumable": "1.0.0",
"Content-Type": "application/offset+octet-stream",
"Upload-Offset": str(offset),
"Content-Length": str(len(data)),
},
data=data,
# Single value sets both connect and read timeout. Do NOT use a tuple here:
# (connect_s, read_s) also applies connect_s to the write phase in some urllib3
# versions, which causes spurious write timeouts on large chunks over slow links.
timeout=300,
)
resp.raise_for_status()
return int(resp.headers["Upload-Offset"])
def _upload_from_offset(auth, local_path: str, upload_url: str,
file_size: int, offset: int, max_retries: int) -> None:
total_chunks = (file_size + CHUNK_SIZE - 1) // CHUNK_SIZE
with open(local_path, "rb") as f:
while offset < file_size:
chunk_size = min(CHUNK_SIZE, file_size - offset)
f.seek(offset)
chunk = f.read(chunk_size)
for attempt in range(1, max_retries + 1):
try:
prev_offset = offset
t0 = time.monotonic()
offset = _patch_chunk(auth, upload_url, offset, chunk)
elapsed = max(time.monotonic() - t0, 0.001)
speed_mb = len(chunk) / elapsed / 1024 / 1024
chunk_idx = prev_offset // CHUNK_SIZE + 1
print(f" chunk {chunk_idx}/{total_chunks} — "
f"{offset}/{file_size} bytes ({100 * offset // file_size}%) — "
f"{speed_mb:.1f} MB/s")
break
except KeyboardInterrupt:
raise
except (RequestException, KeyError) as exc:
if attempt == max_retries:
raise RuntimeError(f"Chunk failed after {max_retries} attempts") from exc
backoff = min(2 ** (attempt - 1), 32)
print(f" Attempt {attempt} failed ({exc}). Retry in {backoff}s…")
time.sleep(backoff)
offset = _get_offset(auth, upload_url)
chunk_size = min(CHUNK_SIZE, file_size - offset)
f.seek(offset)
chunk = f.read(chunk_size)
# ── public API ────────────────────────────────────────────────────────────────
def upload_file(host: str, auth, local_path: str, key: str, max_retries: int) -> str:
"""Upload a file. Returns the upload URL — save it for resume_upload() if interrupted."""
file_size = os.path.getsize(local_path)
upload_url = _create_upload(host, auth, key, file_size)
_upload_from_offset(auth, local_path, upload_url, file_size, 0, max_retries)
print(f"Upload complete: {key}")
return upload_url
def resume_upload(auth, local_path: str, upload_url: str, max_retries: int) -> None:
"""Resume an interrupted upload using the URL printed when the session was created."""
file_size = os.path.getsize(local_path)
offset = _get_offset(auth, upload_url)
print(f"Resuming from {offset}/{file_size} bytes ({100 * offset // file_size}%)")
_upload_from_offset(auth, local_path, upload_url, file_size, offset, max_retries)
print("Upload complete")
def query_upload(auth, upload_url: str) -> None:
"""Print the current status of an upload."""
resp = requests.get(upload_url, headers=auth("GET", _urlpath(upload_url)))
resp.raise_for_status()
r = resp.json()
status = r.get("status", -1)
print(f"upload_id : {r.get('upload_id')}")
print(f"key : {r.get('key')}")
print(f"filename : {r.get('filename')}")
print(f"size : {r.get('size')}")
print(f"offset : {r.get('offset')}")
print(f"status : {status} ({STATUS_LABELS.get(status, 'unknown')})")
print(f"created : {r.get('create_time')}")
print(f"updated : {r.get('update_time')}")
# ── CLI ───────────────────────────────────────────────────────────────────────
def main() -> None:
parser = argparse.ArgumentParser(
description="Opera ADs File Upload CLI",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("--host", default=DEFAULT_HOST,
help=f"Service base URL (default: {DEFAULT_HOST})")
parser.add_argument("--hmac-key-id", default=os.environ.get("ADS_HMAC_KEY_ID", ""),
help="HMAC key ID (or set ADS_HMAC_KEY_ID env var; preferred)")
parser.add_argument("--hmac-secret", default=os.environ.get("ADS_HMAC_SECRET", ""),
help="HMAC secret (or set ADS_HMAC_SECRET env var; preferred)")
parser.add_argument("--api-key", default=os.environ.get("ADS_API_KEY", ""),
help="API key (or set ADS_API_KEY env var)")
parser.add_argument("--file", help="Local file path")
parser.add_argument("--key", help="Upload destination key, e.g. 20260401/file.csv.gz")
parser.add_argument("--max-retries", type=int, default=DEFAULT_MAX_RETRIES,
help=f"Per-chunk retry limit (default: {DEFAULT_MAX_RETRIES})")
parser.add_argument("--resume", metavar="UPLOAD_URL",
help="Resume interrupted upload using this URL")
parser.add_argument("--query", metavar="UPLOAD_URL",
help="Query status of an upload by its URL")
args = parser.parse_args()
has_hmac = bool(args.hmac_key_id and args.hmac_secret)
has_apikey = bool(args.api_key)
if args.hmac_key_id and not args.hmac_secret:
parser.error("--hmac-secret is required when --hmac-key-id is set")
if args.hmac_secret and not args.hmac_key_id:
parser.error("--hmac-key-id is required when --hmac-secret is set")
if not has_hmac and not has_apikey:
parser.error("provide --hmac-key-id + --hmac-secret (preferred) or --api-key")
auth = _make_auth(
api_key =args.api_key,
hmac_key_id=args.hmac_key_id,
hmac_secret=args.hmac_secret,
)
if args.query:
query_upload(auth, args.query)
elif args.resume:
if not args.file:
parser.error("--file is required for --resume")
resume_upload(auth, args.file, args.resume, args.max_retries)
else:
if not args.file:
parser.error("--file is required")
if not args.key:
parser.error("--key is required")
upload_file(args.host, auth, args.file, args.key, args.max_retries)
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("\nInterrupted.", file=sys.stderr)
sys.exit(1)File Management API
List uploads
GET /upload/filesQuery parameters:
| Parameter | Default | Description |
|---|---|---|
page | 1 | Page number (1-based) |
page_size | 20 | Records per page (max 100) |
status | (all) | Filter by status: 0=uploading, 1=complete, 2=failed, 3=cancelled |
Example request:
bash
# All uploads (page 1)
curl -i "$UPLOAD_HOST/upload/files" \
-H "X-API-Key: adx_yourkey"
# Completed uploads only
curl -i "$UPLOAD_HOST/upload/files?status=1&page=1&page_size=50" \
-H "X-API-Key: adx_yourkey"Response 200 OK:
json
{
"uploads": [
{
"upload_id": "a1b2c3...",
"filename": "file.csv.gz",
"size": 104857600,
"offset": 104857600,
"status": 1,
"key": "20250106/file.csv.gz",
"create_time": "2025-01-06T10:00:00Z",
"update_time": "2025-01-06T10:05:00Z"
}
],
"page": 1,
"page_size": 20
}Get upload details
GET /upload/files/:idReturns the full record for a single upload. You can only access your own uploads.
Note —
offsetfor in-progress uploadsFor uploads with
status=0(uploading),offsetreflects the last completed chunk and is updated once per PATCH request. It is not suitable as a resume point — use the TUSHEADmethod to obtain the authoritative resume offset before sending the next chunk.
Example request:
bash
curl -i "$UPLOAD_HOST/upload/files/a1b2c3d4e5f6..." \
-H "X-API-Key: adx_yourkey"Response 200 OK:
json
{
"upload_id": "a1b2c3d4e5f6...",
"filename": "file.csv.gz",
"size": 104857600,
"offset": 104857600,
"status": 1,
"key": "20250106/file.csv.gz",
"create_time": "2025-01-06T10:00:00Z",
"update_time": "2025-01-06T10:05:00Z"
}Error responses:
| HTTP Status | Cause |
|---|---|
401 | Missing or invalid authentication |
403 | Upload ID exists but belongs to a different customer |
404 | Upload ID not found |
Upload Status Codes
| Status | Value | Meaning |
|---|---|---|
| Uploading | 0 | Upload in progress |
| Complete | 1 | File fully uploaded and available |
| Failed | 2 | Upload failed; contact Opera support |
| Cancelled | 3 | Cancelled by client |
Error Responses
All errors return JSON:
json
{"error": "human-readable description"}| HTTP Status | Cause |
|---|---|
400 | Missing or invalid Upload-Metadata (missing key, path traversal, empty key) |
401 | Missing or invalid authentication credentials |
403 | Attempting to access another customer's upload |
404 | Upload ID not found |
500 | Internal server error |
TUS Protocol Headers Reference
| Header | Sent by | Description |
|---|---|---|
Tus-Resumable | Both | Must be 1.0.0 |
Upload-Length | Client (POST) | Total file size in bytes |
Upload-Metadata | Client (POST) | Comma-separated name base64val pairs |
Upload-Offset | Both | Current byte offset |
Content-Type | Client (PATCH) | Must be application/offset+octet-stream |
Location | Server (POST) | Full URL of the created upload resource |
Quotas and Limits
- Total storage per customer: 500 GB
- Maximum single file size: 4 GB — uploads larger than this are rejected at creation with
413; we recommend keeping individual files under 2 GB to minimise retry cost on failure - Maximum chunk size per request: 256 MB (gateway hard limit — requests larger than this are rejected with
413) - Minimum chunk size: 5 MB (except for the final chunk, which may be any size ≥ 1 byte)
- Upload session expiration: 7 days — incomplete uploads older than 7 days are automatically cancelled and their data deleted; resume is no longer possible after expiration
- Request timestamp window: ±300 seconds (HMAC auth only)
Chunk Size Guidance
| Network condition | Recommended chunk size | Rationale |
|---|---|---|
| General / unknown | 50 MB | Good balance of throughput and retry cost; well within the gateway limit |
| Stable high-bandwidth (≥ 100 Mbps) | 100–200 MB | Fewer round trips; lower per-chunk overhead |
| Unstable / mobile | 10–25 MB | Less data to re-send when a chunk fails |
Sending a chunk larger than 256 MB returns HTTP 413 immediately.
Sending a chunk smaller than 5 MB (except the final one) returns HTTP 400.
Best Practices
Use date-based directories
Organize files by date so each day's batch is easy to identify and reprocess if needed. Use YYYYMMDD/ as the directory prefix in your key:
20260401/file_0000.csv.gz
20260401/file_0001.csv.gz
20260401/file_0002.csv.gzSignal completion with a _SUCCESS marker
Upload all data files first. Once every data file has been confirmed complete (status 1), upload a zero-byte _SUCCESS file in the same directory. The Opera pipeline watches for this marker before processing the batch — without it, a partial batch may be picked up mid-transfer.
20260401/_SUCCESSPython example — upload a batch and signal completion:
Note: This example is intentionally simplified — each file is uploaded in a single PATCH request with no chunked transfer, retry, or resume logic. For large files or unreliable networks, use the Full Python Example above instead.
python
import os, requests, base64
HOST = "https://<host>"
API_KEY = "adx_yourkey"
DATE = "20260401"
FILES = [
"/data/20260401/file_0000.csv.gz",
"/data/20260401/file_0001.csv.gz",
"/data/20260401/file_0002.csv.gz",
]
def b64(s: str) -> str:
return base64.b64encode(s.encode()).decode()
def upload_file(local_path: str, key: str, data: bytes = None):
size = len(data) if data is not None else os.path.getsize(local_path)
metadata = f"key {b64(key)},filename {b64(os.path.basename(key))}"
resp = requests.post(
f"{HOST}/upload/files",
headers={
"X-API-Key": API_KEY,
"Tus-Resumable": "1.0.0",
"Upload-Length": str(size),
"Upload-Metadata": metadata,
"Content-Length": "0",
},
)
resp.raise_for_status()
upload_url = resp.headers["Location"]
body = data if data is not None else open(local_path, "rb").read()
patch = requests.patch(
upload_url,
headers={
"X-API-Key": API_KEY,
"Tus-Resumable": "1.0.0",
"Content-Type": "application/offset+octet-stream",
"Upload-Offset": "0",
"Content-Length": str(size),
},
data=body,
)
patch.raise_for_status()
print(f"Uploaded: {key}")
# 1. Upload all data files
for path in FILES:
key = f"{DATE}/{os.path.basename(path)}"
upload_file(path, key)
# 2. Signal that the batch is complete
upload_file("", f"{DATE}/_SUCCESS", data=b"")
print(f"Batch {DATE} complete.")bash example:
Note: Same simplification as above — single-request upload, no chunked transfer or resume.
bash
DATE="20260401"
API_KEY="adx_yourkey"
upload() {
local key="$1"
local file="$2"
local size="${3:-$(wc -c < "$file" | tr -d ' ')}"
url=$(curl -si -X POST "$UPLOAD_HOST/upload/files" \
-H "X-API-Key: $API_KEY" \
-H "Tus-Resumable: 1.0.0" \
-H "Upload-Length: $size" \
-H "Upload-Metadata: key $(echo -n "$key" | base64),filename $(echo -n "$(basename "$key")" | base64)" \
-H "Content-Length: 0" \
| grep -i location | awk '{print $2}' | tr -d '\r')
curl -s -X PATCH "$url" \
-H "X-API-Key: $API_KEY" \
-H "Tus-Resumable: 1.0.0" \
-H "Content-Type: application/offset+octet-stream" \
-H "Upload-Offset: 0" \
-H "Content-Length: $size" \
--data-binary @"$file"
echo "Uploaded: $key"
}
# Upload data files
for i in $(seq -f "%04.0f" 0 2); do
upload "$DATE/file_${i}.csv.gz" "/data/$DATE/file_${i}.csv.gz"
done
# Upload _SUCCESS marker (zero bytes)
echo -n "" > /tmp/_SUCCESS
upload "$DATE/_SUCCESS" /tmp/_SUCCESS 0
echo "Batch $DATE complete."Getting Started
- Contact the Opera ADs team to obtain credentials (API key or HMAC key pair).
- Your credentials are scoped to your own storage prefix — you cannot read or write another customer's files.
- Use any TUS client library or the raw HTTP examples above.
- Monitor upload status via the
GET /upload/filesendpoint.
Recommended TUS client libraries:
| Language | Library |
|---|---|
| Python | tus-py-client |
| JavaScript | tus-js-client |
| Java | tus-java-client |
| Go | tus |
