Skip to content

File Upload API — Customer Integration Guide

Version: 1.0.0 — Initial release

Overview

The Opera ADs File Upload API provides resumable, large-file uploads.
It implements the TUS resumable upload protocol so uploads can survive network failures and continue from where they left off.

Base URL: https://<service-host>/upload

Set this shell variable once before running any of the curl examples below:

bash
export UPLOAD_HOST="https://<host>"   # replace with the host provided by Opera team

Authentication

Every request must be authenticated, except OPTIONS /upload/files which is public. Two methods are supported — choose one.

Option A: API Key

Include the API key in every request header:

X-API-Key: adx_<your_key>

API keys are static credentials issued by the Opera team. Contact Opera support to rotate them if compromised.

Option B: HMAC Signature

Per-request signatures prevent replay attacks and do not require a long-lived secret to travel with every request.

Required headers:

HeaderDescription
X-HMAC-Key-IdYour HMAC key ID, provided by Opera team (format: ADX_HMAC_...)
X-HMAC-TimestampCurrent Unix timestamp (seconds). Request is rejected if `
X-HMAC-SignatureHMAC-SHA256(secret, message) encoded as lowercase hex

Signature message format:

{METHOD}\n{PATH}\n{TIMESTAMP}

Example for PATCH /upload/files/abc123 at timestamp 1700000000:

PATCH\n/upload/files/abc123\n1700000000

Python example:

python
import hmac, hashlib, time, requests

HOST   = "https://<host>"
KEY_ID = "ADX_HMAC_K7MN2PQR4XVBXXXXXXXXXXXX"  # provided by Opera team
SECRET = "your_hmac_secret"                      # provided by Opera team

def sign(method: str, path: str) -> dict:
    ts = str(int(time.time()))
    message = f"{method}\n{path}\n{ts}"
    sig = hmac.new(SECRET.encode(), message.encode(), hashlib.sha256).hexdigest()
    return {"X-HMAC-Key-Id": KEY_ID, "X-HMAC-Timestamp": ts, "X-HMAC-Signature": sig}

headers = sign("POST", "/upload/files")
resp = requests.post(f"{HOST}/upload/files", headers={
    **headers,
    "Upload-Length": str(file_size),
    "Upload-Metadata": f"key {b64(object_key)},filename {b64(filename)}",
    "Tus-Resumable": "1.0.0",
    "Content-Length": "0",
})

Go example:

go
import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "time"
)

const (
    KEY_ID = "ADX_HMAC_K7MN2PQR4XVBXXXXXXXXXXXX" // provided by Opera team
    SECRET = "your_hmac_secret"                     // provided by Opera team
)

func signRequest(method, path string) (keyID, timestamp, sig string) {
    timestamp = fmt.Sprintf("%d", time.Now().Unix())
    message := method + "\n" + path + "\n" + timestamp
    mac := hmac.New(sha256.New, []byte(SECRET))
    mac.Write([]byte(message))
    return KEY_ID, timestamp, hex.EncodeToString(mac.Sum(nil))
}

JavaScript example:

js
const crypto = require("crypto");

const KEY_ID = "ADX_HMAC_K7MN2PQR4XVBXXXXXXXXXXXX"; // provided by Opera team
const SECRET = "your_hmac_secret";                     // provided by Opera team

function sign(method, path) {
  const ts = String(Math.floor(Date.now() / 1000));
  const message = `${method}\n${path}\n${ts}`;
  const sig = crypto.createHmac("sha256", SECRET).update(message).digest("hex");
  return { "X-HMAC-Key-Id": KEY_ID, "X-HMAC-Timestamp": ts, "X-HMAC-Signature": sig };
}

Uploading Files (TUS Protocol)

The TUS protocol uses three requests: CreateUpload chunks → (optional) Resume.

Step 0 — Capability discovery (optional)

OPTIONS /upload/files
OPTIONS /upload/files/:id

Optional — use this to confirm the server's supported protocol version and maximum file size before starting an upload. No authentication required.

bash
curl -i -X OPTIONS "$UPLOAD_HOST/upload/files"

Response 200 OK:

Tus-Version:   1.0.0
Tus-Resumable: 1.0.0
Tus-Extension: creation,creation-with-upload,termination,concatenation,creation-defer-length
Tus-Max-Size:  4294967296
Response headerMeaning
Tus-VersionTUS protocol versions supported by the server
Tus-ResumableVersion to use in all subsequent requests
Tus-ExtensionTUS extensions enabled on this server
Tus-Max-SizeMaximum upload size in bytes (4 GB)

Step 1 — Create an upload session

POST /upload/files

Required headers:

HeaderValue
Tus-Resumable1.0.0
Upload-LengthTotal file size in bytes
Upload-MetadataComma-separated name base64value pairs (see below)
Content-Length0

Upload-Metadata fields:

FieldRequiredDescription
keyYesDestination path for the file, relative to your allocated prefix, e.g. 20250106/file.csv.gz
filenameNoDisplay name; defaults to basename of key

The key field determines where the file is stored under your allocated storage prefix.

Rules:

  • Leading / is stripped automatically
  • Path traversal (..) is rejected with 400
  • The value must be non-empty

Example request:

bash
KEY=$(echo -n "20250106/file.csv.gz" | base64 -w 0)
FILENAME=$(echo -n "file.csv.gz" | base64 -w 0)

curl -i "$UPLOAD_HOST/upload/files" \
  -X POST \
  -H "X-API-Key: adx_yourkey" \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Length: 104857600" \
  -H "Upload-Metadata: key $KEY,filename $FILENAME" \
  -H "Content-Length: 0"

Successful response 201 Created:

Location: https://<host>/upload/files/a1b2c3d4e5f6...
Tus-Resumable: 1.0.0
Upload-Offset: 0
Response headerMeaning
LocationUpload session URL — save this; all subsequent PATCH / HEAD / DELETE requests use it
Tus-ResumableTUS protocol version confirmed by the server
Upload-OffsetAlways 0 on creation

Error responses:

HTTP StatusCause
400Missing or invalid Upload-Metadata (missing key, path traversal, empty key)
401Missing or invalid authentication
413Upload-Length exceeds the server limit (4 GB)

Step 2 — Upload chunks

PATCH {Location}

Required headers:

HeaderValue
Tus-Resumable1.0.0
Content-Typeapplication/offset+octet-stream
Upload-OffsetByte offset where this chunk starts (0 for first chunk)
Content-LengthSize of this chunk in bytes

Example — upload entire file in one request:

bash
UPLOAD_URL="$UPLOAD_HOST/upload/files/a1b2c3d4e5f6..."

curl -i "$UPLOAD_URL" \
  -X PATCH \
  -H "X-API-Key: adx_yourkey" \
  -H "Tus-Resumable: 1.0.0" \
  -H "Content-Type: application/offset+octet-stream" \
  -H "Upload-Offset: 0" \
  -H "Content-Length: 104857600" \
  --data-binary @file.csv.gz

Response 204 No Content:

Upload-Offset: 104857600
Response headerMeaning
Upload-OffsetTotal bytes the server has received so far; use this as the next chunk's Upload-Offset

When Upload-Offset equals Upload-Length, the upload is complete.

Error responses:

StatusCondition
401 UnauthorizedMissing or invalid credentials
403 ForbiddenUpload ID exists but belongs to a different customer
404 Not FoundUpload ID not found
409 ConflictUpload already complete, or Upload-Offset does not match the server's current offset

Step 3 — Resume after failure (optional)

Check the current offset:

HEAD {Location}
bash
curl -I "$UPLOAD_URL" \
  -H "X-API-Key: adx_yourkey" \
  -H "Tus-Resumable: 1.0.0"

Response 200 OK:

Upload-Length: 104857600
Upload-Offset: 52428800
Upload-Metadata: customer-name <base64>,filename <base64>,key <base64>
Response headerMeaning
Upload-OffsetBytes the server has durably received — start the next PATCH from this offset
Upload-LengthTotal file size declared at session creation
Upload-MetadataMetadata supplied at session creation, base64-encoded per field; customer-name is injected by the server

If the upload is already complete, the server returns 200 OK with Upload-Offset equal to Upload-Length — no further PATCH is needed.

Then send a PATCH starting from Upload-Offset.

Error responses:

StatusCondition
401 UnauthorizedMissing or invalid credentials
403 ForbiddenUpload ID exists but belongs to a different customer
404 Not FoundUpload ID not found

Note — HEAD may be slow after an interrupted upload

If the previous PATCH was cut short by a network failure, the server needs time to finalize, synchronize, and clean up resources before it can report a consistent offset. The HEAD request may block for up to 1–3 minutes in this case — this is expected. Do not treat a slow HEAD response as an error; wait for it to complete before resuming.


Cancel an upload

DELETE {Location}
bash
curl -i "$UPLOAD_URL" \
  -X DELETE \
  -H "X-API-Key: adx_yourkey" \
  -H "Tus-Resumable: 1.0.0"

Response 204 No Content

The operation is idempotent: if the upload is already complete or cancelled the server returns 204 without taking any further action.

Error responses

StatusCondition
401 UnauthorizedMissing or invalid X-API-Key / HMAC credentials
403 ForbiddenUpload ID exists but belongs to a different customer
404 Not FoundUpload ID not found

Chunked Upload with Retry and Resume

Algorithm (pseudocode)

The TUS protocol guarantees that the server's Upload-Offset is always the authoritative resume point. The pattern below works for both a fresh upload and resuming after any interruption:

function upload_file(path, key):
    file_size  = stat(path)
    upload_url = POST /upload/files  (Upload-Length: file_size, Upload-Metadata: key)

    offset = 0
    while offset < file_size:
        chunk = read(path, offset, min(CHUNK_SIZE, file_size - offset))

        for attempt = 1 .. MAX_RETRIES:
            try:
                offset = PATCH upload_url  (Upload-Offset: offset, body: chunk)
                break                          # chunk accepted — move to next
            except NetworkError | HTTP 5xx:
                if attempt == MAX_RETRIES: raise
                sleep(min(2^(attempt-1), 32))  # 1 s → 2 s → 4 s → … → 32 s
                # HEAD asks the server for the confirmed offset.
                # After a mid-chunk interruption the server may take 1–3 minutes
                # to finalize and clean up before responding — this is expected;
                # keep retrying HEAD with a generous timeout until it succeeds.
                offset = HEAD upload_url (timeout=240, retry until success)
                chunk  = read(path, offset, min(CHUNK_SIZE, file_size - offset))

    return upload_url   # save this URL; pass it to resume_upload() if process is killed


function resume_upload(upload_url, path):
    file_size = stat(path)
    # The server may need time to finalize resources after an interruption.
    # Use a generous timeout and retry HEAD until the server responds.
    offset    = HEAD upload_url (timeout=240, retry until success)
    # continue with the same retry loop starting from offset
    ...

Full Python Example

The script below is production-ready. It supports both HMAC and API key authentication (HMAC preferred), uploads files in 50 MB chunks, and handles network failures with exponential backoff and automatic resume. It can also be used to query the status of any upload by its session URL.

Save it as upload.py and invoke it from the command line — no external dependencies beyond requests.

python
#!/usr/bin/env python3
"""Opera ADs File Upload CLI — upload, resume, or query status.

Usage:
  # Upload with HMAC (preferred):
  python upload.py --hmac-key-id ADX_HMAC_... --hmac-secret <secret> \\
      --file /data/file.csv.gz --key 20260401/file.csv.gz

  # Upload with API key:
  python upload.py --api-key adx_... --file /data/file.csv.gz --key 20260401/file.csv.gz

  # Resume an interrupted upload (use the URL printed during the original upload):
  python upload.py --hmac-key-id ADX_HMAC_... --hmac-secret <secret> \\
      --file /data/file.csv.gz --resume https://<host>/upload/files/<id>

  # Query upload status:
  python upload.py --hmac-key-id ADX_HMAC_... --hmac-secret <secret> \\
      --query https://<host>/upload/files/<id>
"""
import argparse, base64, hashlib, hmac as hmac_mod, os, sys, time
import requests
from requests.exceptions import RequestException
from urllib.parse import urlparse

DEFAULT_HOST        = "https://<host>"
DEFAULT_MAX_RETRIES = 10
CHUNK_SIZE          = 50 * 1024 * 1024  # 50 MB

STATUS_LABELS = {0: "uploading", 1: "complete", 2: "failed", 3: "cancelled"}


# ── helpers ───────────────────────────────────────────────────────────────────

def _make_auth(api_key: str = "", hmac_key_id: str = "", hmac_secret: str = ""):
    """Return a callable auth(method, path) -> dict that produces request auth headers.

    HMAC is used when hmac_key_id and hmac_secret are both provided; falls back to API key.
    """
    if hmac_key_id and hmac_secret:
        def _sign(method: str, path: str) -> dict:
            ts  = str(int(time.time()))
            msg = f"{method}\n{path}\n{ts}"
            sig = hmac_mod.new(hmac_secret.encode(), msg.encode(), hashlib.sha256).hexdigest()
            return {"X-HMAC-Key-Id": hmac_key_id, "X-HMAC-Timestamp": ts, "X-HMAC-Signature": sig}
        return _sign
    def _key(method: str, path: str) -> dict:
        return {"X-API-Key": api_key}
    return _key


def _urlpath(url: str) -> str:
    return urlparse(url).path


def _b64(s: str) -> str:
    return base64.b64encode(s.encode()).decode()


def _create_upload(host: str, auth, key: str, file_size: int) -> str:
    resp = requests.post(
        f"{host}/upload/files",
        headers={
            **auth("POST", "/upload/files"),
            "Tus-Resumable":   "1.0.0",
            "Upload-Length":   str(file_size),
            "Upload-Metadata": f"key {_b64(key)},filename {_b64(os.path.basename(key))}",
            "Content-Length":  "0",
        },
    )
    resp.raise_for_status()
    url = resp.headers["Location"]
    print(f"\n\033[1;33m>>> Upload session URL: {url}\033[0m")
    print("\033[1;33m>>> SAVE THIS URL — you will need it to resume if the upload is interrupted\033[0m\n")
    return url


def _get_offset(auth, upload_url: str) -> int:
    """HEAD with retry — server may take 1–3 min to finalize after an interrupted chunk."""
    for attempt in range(1, 10):
        try:
            resp = requests.head(
                upload_url,
                headers={**auth("HEAD", _urlpath(upload_url)), "Tus-Resumable": "1.0.0"},
                timeout=240,
            )
            resp.raise_for_status()
            return int(resp.headers["Upload-Offset"])
        except KeyboardInterrupt:
            raise
        except RequestException as exc:
            # Retry on any network error (timeout, DNS failure, connection reset, etc.)
            wait = min(30 * attempt, 120)
            print(f"  HEAD failed (attempt {attempt}: {exc}) — retry in {wait}s…")
            time.sleep(wait)
    raise RuntimeError("Server did not respond to HEAD after multiple attempts")


def _patch_chunk(auth, upload_url: str, offset: int, data: bytes) -> int:
    resp = requests.patch(
        upload_url,
        headers={
            **auth("PATCH", _urlpath(upload_url)),
            "Tus-Resumable":  "1.0.0",
            "Content-Type":   "application/offset+octet-stream",
            "Upload-Offset":  str(offset),
            "Content-Length": str(len(data)),
        },
        data=data,
        # Single value sets both connect and read timeout. Do NOT use a tuple here:
        # (connect_s, read_s) also applies connect_s to the write phase in some urllib3
        # versions, which causes spurious write timeouts on large chunks over slow links.
        timeout=300,
    )
    resp.raise_for_status()
    return int(resp.headers["Upload-Offset"])


def _upload_from_offset(auth, local_path: str, upload_url: str,
                        file_size: int, offset: int, max_retries: int) -> None:
    total_chunks = (file_size + CHUNK_SIZE - 1) // CHUNK_SIZE
    with open(local_path, "rb") as f:
        while offset < file_size:
            chunk_size = min(CHUNK_SIZE, file_size - offset)
            f.seek(offset)
            chunk = f.read(chunk_size)

            for attempt in range(1, max_retries + 1):
                try:
                    prev_offset = offset
                    t0          = time.monotonic()
                    offset      = _patch_chunk(auth, upload_url, offset, chunk)
                    elapsed     = max(time.monotonic() - t0, 0.001)
                    speed_mb    = len(chunk) / elapsed / 1024 / 1024
                    chunk_idx   = prev_offset // CHUNK_SIZE + 1
                    print(f"  chunk {chunk_idx}/{total_chunks} — "
                          f"{offset}/{file_size} bytes ({100 * offset // file_size}%) — "
                          f"{speed_mb:.1f} MB/s")
                    break
                except KeyboardInterrupt:
                    raise
                except (RequestException, KeyError) as exc:
                    if attempt == max_retries:
                        raise RuntimeError(f"Chunk failed after {max_retries} attempts") from exc
                    backoff = min(2 ** (attempt - 1), 32)
                    print(f"  Attempt {attempt} failed ({exc}). Retry in {backoff}s…")
                    time.sleep(backoff)
                    offset = _get_offset(auth, upload_url)
                    chunk_size = min(CHUNK_SIZE, file_size - offset)
                    f.seek(offset)
                    chunk = f.read(chunk_size)


# ── public API ────────────────────────────────────────────────────────────────

def upload_file(host: str, auth, local_path: str, key: str, max_retries: int) -> str:
    """Upload a file. Returns the upload URL — save it for resume_upload() if interrupted."""
    file_size  = os.path.getsize(local_path)
    upload_url = _create_upload(host, auth, key, file_size)
    _upload_from_offset(auth, local_path, upload_url, file_size, 0, max_retries)
    print(f"Upload complete: {key}")
    return upload_url


def resume_upload(auth, local_path: str, upload_url: str, max_retries: int) -> None:
    """Resume an interrupted upload using the URL printed when the session was created."""
    file_size = os.path.getsize(local_path)
    offset    = _get_offset(auth, upload_url)
    print(f"Resuming from {offset}/{file_size} bytes ({100 * offset // file_size}%)")
    _upload_from_offset(auth, local_path, upload_url, file_size, offset, max_retries)
    print("Upload complete")


def query_upload(auth, upload_url: str) -> None:
    """Print the current status of an upload."""
    resp = requests.get(upload_url, headers=auth("GET", _urlpath(upload_url)))
    resp.raise_for_status()
    r      = resp.json()
    status = r.get("status", -1)
    print(f"upload_id : {r.get('upload_id')}")
    print(f"key       : {r.get('key')}")
    print(f"filename  : {r.get('filename')}")
    print(f"size      : {r.get('size')}")
    print(f"offset    : {r.get('offset')}")
    print(f"status    : {status} ({STATUS_LABELS.get(status, 'unknown')})")
    print(f"created   : {r.get('create_time')}")
    print(f"updated   : {r.get('update_time')}")


# ── CLI ───────────────────────────────────────────────────────────────────────

def main() -> None:
    parser = argparse.ArgumentParser(
        description="Opera ADs File Upload CLI",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog=__doc__,
    )
    parser.add_argument("--host",         default=DEFAULT_HOST,
                        help=f"Service base URL (default: {DEFAULT_HOST})")
    parser.add_argument("--hmac-key-id",  default=os.environ.get("ADS_HMAC_KEY_ID", ""),
                        help="HMAC key ID (or set ADS_HMAC_KEY_ID env var; preferred)")
    parser.add_argument("--hmac-secret",  default=os.environ.get("ADS_HMAC_SECRET", ""),
                        help="HMAC secret (or set ADS_HMAC_SECRET env var; preferred)")
    parser.add_argument("--api-key",      default=os.environ.get("ADS_API_KEY", ""),
                        help="API key (or set ADS_API_KEY env var)")
    parser.add_argument("--file",         help="Local file path")
    parser.add_argument("--key",          help="Upload destination key, e.g. 20260401/file.csv.gz")
    parser.add_argument("--max-retries",  type=int, default=DEFAULT_MAX_RETRIES,
                        help=f"Per-chunk retry limit (default: {DEFAULT_MAX_RETRIES})")
    parser.add_argument("--resume",       metavar="UPLOAD_URL",
                        help="Resume interrupted upload using this URL")
    parser.add_argument("--query",        metavar="UPLOAD_URL",
                        help="Query status of an upload by its URL")

    args = parser.parse_args()

    has_hmac   = bool(args.hmac_key_id and args.hmac_secret)
    has_apikey = bool(args.api_key)
    if args.hmac_key_id and not args.hmac_secret:
        parser.error("--hmac-secret is required when --hmac-key-id is set")
    if args.hmac_secret and not args.hmac_key_id:
        parser.error("--hmac-key-id is required when --hmac-secret is set")
    if not has_hmac and not has_apikey:
        parser.error("provide --hmac-key-id + --hmac-secret (preferred) or --api-key")

    auth = _make_auth(
        api_key    =args.api_key,
        hmac_key_id=args.hmac_key_id,
        hmac_secret=args.hmac_secret,
    )

    if args.query:
        query_upload(auth, args.query)
    elif args.resume:
        if not args.file:
            parser.error("--file is required for --resume")
        resume_upload(auth, args.file, args.resume, args.max_retries)
    else:
        if not args.file:
            parser.error("--file is required")
        if not args.key:
            parser.error("--key is required")
        upload_file(args.host, auth, args.file, args.key, args.max_retries)


if __name__ == "__main__":
    try:
        main()
    except KeyboardInterrupt:
        print("\nInterrupted.", file=sys.stderr)
        sys.exit(1)

File Management API

List uploads

GET /upload/files

Query parameters:

ParameterDefaultDescription
page1Page number (1-based)
page_size20Records per page (max 100)
status(all)Filter by status: 0=uploading, 1=complete, 2=failed, 3=cancelled

Example request:

bash
# All uploads (page 1)
curl -i "$UPLOAD_HOST/upload/files" \
  -H "X-API-Key: adx_yourkey"

# Completed uploads only
curl -i "$UPLOAD_HOST/upload/files?status=1&page=1&page_size=50" \
  -H "X-API-Key: adx_yourkey"

Response 200 OK:

json
{
  "uploads": [
    {
      "upload_id": "a1b2c3...",
      "filename": "file.csv.gz",
      "size": 104857600,
      "offset": 104857600,
      "status": 1,
      "key": "20250106/file.csv.gz",
      "create_time": "2025-01-06T10:00:00Z",
      "update_time": "2025-01-06T10:05:00Z"
    }
  ],
  "page": 1,
  "page_size": 20
}

Get upload details

GET /upload/files/:id

Returns the full record for a single upload. You can only access your own uploads.

Note — offset for in-progress uploads

For uploads with status=0 (uploading), offset reflects the last completed chunk and is updated once per PATCH request. It is not suitable as a resume point — use the TUS HEAD method to obtain the authoritative resume offset before sending the next chunk.

Example request:

bash
curl -i "$UPLOAD_HOST/upload/files/a1b2c3d4e5f6..." \
  -H "X-API-Key: adx_yourkey"

Response 200 OK:

json
{
  "upload_id": "a1b2c3d4e5f6...",
  "filename": "file.csv.gz",
  "size": 104857600,
  "offset": 104857600,
  "status": 1,
  "key": "20250106/file.csv.gz",
  "create_time": "2025-01-06T10:00:00Z",
  "update_time": "2025-01-06T10:05:00Z"
}

Error responses:

HTTP StatusCause
401Missing or invalid authentication
403Upload ID exists but belongs to a different customer
404Upload ID not found

Upload Status Codes

StatusValueMeaning
Uploading0Upload in progress
Complete1File fully uploaded and available
Failed2Upload failed; contact Opera support
Cancelled3Cancelled by client

Error Responses

All errors return JSON:

json
{"error": "human-readable description"}
HTTP StatusCause
400Missing or invalid Upload-Metadata (missing key, path traversal, empty key)
401Missing or invalid authentication credentials
403Attempting to access another customer's upload
404Upload ID not found
500Internal server error

TUS Protocol Headers Reference

HeaderSent byDescription
Tus-ResumableBothMust be 1.0.0
Upload-LengthClient (POST)Total file size in bytes
Upload-MetadataClient (POST)Comma-separated name base64val pairs
Upload-OffsetBothCurrent byte offset
Content-TypeClient (PATCH)Must be application/offset+octet-stream
LocationServer (POST)Full URL of the created upload resource

Quotas and Limits

  • Total storage per customer: 500 GB
  • Maximum single file size: 4 GB — uploads larger than this are rejected at creation with 413; we recommend keeping individual files under 2 GB to minimise retry cost on failure
  • Maximum chunk size per request: 256 MB (gateway hard limit — requests larger than this are rejected with 413)
  • Minimum chunk size: 5 MB (except for the final chunk, which may be any size ≥ 1 byte)
  • Upload session expiration: 7 days — incomplete uploads older than 7 days are automatically cancelled and their data deleted; resume is no longer possible after expiration
  • Request timestamp window: ±300 seconds (HMAC auth only)

Chunk Size Guidance

Network conditionRecommended chunk sizeRationale
General / unknown50 MBGood balance of throughput and retry cost; well within the gateway limit
Stable high-bandwidth (≥ 100 Mbps)100–200 MBFewer round trips; lower per-chunk overhead
Unstable / mobile10–25 MBLess data to re-send when a chunk fails

Sending a chunk larger than 256 MB returns HTTP 413 immediately.
Sending a chunk smaller than 5 MB (except the final one) returns HTTP 400.


Best Practices

Use date-based directories

Organize files by date so each day's batch is easy to identify and reprocess if needed. Use YYYYMMDD/ as the directory prefix in your key:

20260401/file_0000.csv.gz
20260401/file_0001.csv.gz
20260401/file_0002.csv.gz

Signal completion with a _SUCCESS marker

Upload all data files first. Once every data file has been confirmed complete (status 1), upload a zero-byte _SUCCESS file in the same directory. The Opera pipeline watches for this marker before processing the batch — without it, a partial batch may be picked up mid-transfer.

20260401/_SUCCESS

Python example — upload a batch and signal completion:

Note: This example is intentionally simplified — each file is uploaded in a single PATCH request with no chunked transfer, retry, or resume logic. For large files or unreliable networks, use the Full Python Example above instead.

python
import os, requests, base64

HOST = "https://<host>"
API_KEY = "adx_yourkey"
DATE = "20260401"

FILES = [
    "/data/20260401/file_0000.csv.gz",
    "/data/20260401/file_0001.csv.gz",
    "/data/20260401/file_0002.csv.gz",
]


def b64(s: str) -> str:
    return base64.b64encode(s.encode()).decode()


def upload_file(local_path: str, key: str, data: bytes = None):
    size = len(data) if data is not None else os.path.getsize(local_path)
    metadata = f"key {b64(key)},filename {b64(os.path.basename(key))}"

    resp = requests.post(
        f"{HOST}/upload/files",
        headers={
            "X-API-Key": API_KEY,
            "Tus-Resumable": "1.0.0",
            "Upload-Length": str(size),
            "Upload-Metadata": metadata,
            "Content-Length": "0",
        },
    )
    resp.raise_for_status()
    upload_url = resp.headers["Location"]

    body = data if data is not None else open(local_path, "rb").read()
    patch = requests.patch(
        upload_url,
        headers={
            "X-API-Key": API_KEY,
            "Tus-Resumable": "1.0.0",
            "Content-Type": "application/offset+octet-stream",
            "Upload-Offset": "0",
            "Content-Length": str(size),
        },
        data=body,
    )
    patch.raise_for_status()
    print(f"Uploaded: {key}")


# 1. Upload all data files
for path in FILES:
    key = f"{DATE}/{os.path.basename(path)}"
    upload_file(path, key)

# 2. Signal that the batch is complete
upload_file("", f"{DATE}/_SUCCESS", data=b"")
print(f"Batch {DATE} complete.")

bash example:

Note: Same simplification as above — single-request upload, no chunked transfer or resume.

bash
DATE="20260401"
API_KEY="adx_yourkey"

upload() {
  local key="$1"
  local file="$2"
  local size="${3:-$(wc -c < "$file" | tr -d ' ')}"

  url=$(curl -si -X POST "$UPLOAD_HOST/upload/files" \
    -H "X-API-Key: $API_KEY" \
    -H "Tus-Resumable: 1.0.0" \
    -H "Upload-Length: $size" \
    -H "Upload-Metadata: key $(echo -n "$key" | base64),filename $(echo -n "$(basename "$key")" | base64)" \
    -H "Content-Length: 0" \
    | grep -i location | awk '{print $2}' | tr -d '\r')

  curl -s -X PATCH "$url" \
    -H "X-API-Key: $API_KEY" \
    -H "Tus-Resumable: 1.0.0" \
    -H "Content-Type: application/offset+octet-stream" \
    -H "Upload-Offset: 0" \
    -H "Content-Length: $size" \
    --data-binary @"$file"
  echo "Uploaded: $key"
}

# Upload data files
for i in $(seq -f "%04.0f" 0 2); do
  upload "$DATE/file_${i}.csv.gz" "/data/$DATE/file_${i}.csv.gz"
done

# Upload _SUCCESS marker (zero bytes)
echo -n "" > /tmp/_SUCCESS
upload "$DATE/_SUCCESS" /tmp/_SUCCESS 0
echo "Batch $DATE complete."

Getting Started

  1. Contact the Opera ADs team to obtain credentials (API key or HMAC key pair).
  2. Your credentials are scoped to your own storage prefix — you cannot read or write another customer's files.
  3. Use any TUS client library or the raw HTTP examples above.
  4. Monitor upload status via the GET /upload/files endpoint.

Recommended TUS client libraries:

LanguageLibrary
Pythontus-py-client
JavaScripttus-js-client
Javatus-java-client
Gotus