Tasks & Long-Running Requests

Some requests take longer than an HTTP connection should stay open. On the endpoints below, every request runs as a task. If it finishes within 60 seconds, you get the result exactly as before. If it does not, you get 202 Accepted with a task_id, and you collect the result from GET /v1/tasks/{task_id} when it is ready. The work is not lost and you are not charged twice.

Which endpoints run as tasks

Every other endpoint, including Backlinks, Google Images and Technical SEO, still answers synchronously, as before. More endpoints will follow.

How it works

1. 1

   Send the request as usual

   Same URL, same headers, same body. We store the task before any work starts, then hold the connection open for up to wait seconds (60 by default).

2. 2

   Finished in time: you get the result

   The status code and body are the same as they have always been. The only addition is an X-Task-Id header that holds the task id.

3. 3

   Still running: you get 202

   The response has a Location header pointing to the task and a Retry-After header saying how many seconds to wait before asking again.

4. 4

   Collect the result

   Call GET /v1/tasks/{task_id} after Retry-After seconds, and repeat until the status is no longer 202. The final response is exactly what the POST would have returned.

Integrations written before tasks keep working for every request that finishes within a minute. Every client should still handle 202, because any request can take longer than wait, and during a deploy a waiting request can receive its 202 early. The helper functions in Examples handle it for you.

Where to find the task id

Every task has an id, a UUID like 4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39. It comes with every answer, so you can always go back to a result:

task_id, request_id, X-Task-Id and X-Request-Id are always the same value. X-Request-Id stays for clients written before tasks existed. Log the task id with every request: it is how you fetch a result again, and how our support traces a request end to end.

The 202 response

When a task is still running after wait seconds, the POST returns 202 Accepted:

HTTP/1.1 202 Accepted
Location: /v1/tasks/4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39
Retry-After: 5
X-Task-Id: 4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39
X-Request-Id: 4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39
X-Task-Status: pending
Content-Type: application/json

{
  "success": true,
  "status": "pending",
  "task_id": "4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39",
  "request_id": "4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39",
  "service": "serp",
  "poll_url": "https://data.semust.com/v1/tasks/4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39",
  "retry_after": 5,
  "created_at": "2026-11-06T13:15:26Z",
  "elapsed_seconds": 60,
  "deadline_at": "2026-11-06T13:25:26Z",
  "deadline_in_seconds": 540
}

Polling a task

GET /v1/tasks/{task_id} with the same SEMUST-API-USER and SEMUST-API-PASSWORD headers you used for the POST.

A finished task answers with the same status and the same body bytes the POST would have returned, so the response parser you already have handles it unchanged. Only the X-Task-* headers tell the two apart:

• X-Task-Status: completed means the task succeeded and was charged. failed means it was not charged.
• X-Task-Expires-At is the time (RFC 3339) after which the result is deleted, and X-Task-Expires-In the seconds left until then.

These headers come with every finished answer, including a result returned directly on the POST, so you always know whether you were charged and how long you can fetch the result again:

HTTP/1.1 200 OK
X-Task-Id: 4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39
X-Task-Status: completed
X-Task-Expires-At: 2026-11-09T13:16:41Z
X-Task-Expires-In: 259200

Other rules:

• Fetch as often as you like. A finished result can be fetched any number of times until it expires. Fetching is free.
• Only the creator can read it. A task can be read only by the API user that created it. Any other API user, including another API user on the same account, gets 404 TASK_NOT_FOUND.
• Synchronous answers are tasks too. Every request that does real work is stored as a task, including the ones answered within wait. Any response with an X-Task-Id header can be fetched again by that id within the retention window.
• Polling does not wait. GET /v1/tasks/{task_id} answers immediately. Honour Retry-After between calls: 3 seconds during a task's first 30 seconds, 5 seconds until 2 minutes, then 10 seconds.

The wait parameter

?wait= sets how many seconds the POST holds the connection open before answering 202.

The maximum is 80 because the edge in front of the API cuts connections at 100 seconds. Set your HTTP client's read timeout above wait, for example wait + 30 seconds.

wait=0 is fire-and-collect: submit a batch of requests without holding a connection open for each, then collect the results by polling or by listing. A request that finishes instantly, such as a cache hit, is still answered with its result straight away.

POST https://data.semust.com/v1/domain-research?wait=0

Listing tasks

GET /v1/tasks lists your recent tasks whose results can still be fetched, newest first. It uses the same headers and is free.

An invalid status, fetched, page or limit returns 400 INVALID_PARAMETER, and field names the parameter.

curl "https://data.semust.com/v1/tasks?status=completed&service=domain_research&limit=20" \
  -H "SEMUST-API-USER: your_username" \
  -H "SEMUST-API-PASSWORD: your_password"

{
  "success": true,
  "total": 2,
  "page": 1,
  "limit": 50,
  "items": [
    {
      "task_id": "9d3a6c1e-27b4-4f0a-8e5d-3c1b7a9f2e60",
      "service": "serp",
      "status": "pending",
      "http_status": null,
      "cost": null,
      "fetched": false,
      "poll_url": "https://data.semust.com/v1/tasks/9d3a6c1e-27b4-4f0a-8e5d-3c1b7a9f2e60",
      "created_at": "2026-11-06T13:17:02Z",
      "completed_at": null,
      "expires_at": null,
      "expires_in_seconds": null,
      "deadline_at": "2026-11-06T13:27:02Z"
    },
    {
      "task_id": "4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39",
      "service": "domain_research",
      "status": "completed",
      "http_status": 200,
      "cost": 0.0264,
      "fetched": false,
      "poll_url": "https://data.semust.com/v1/tasks/4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39",
      "created_at": "2026-11-06T13:15:26Z",
      "completed_at": "2026-11-06T13:16:41Z",
      "expires_at": "2026-11-09T13:16:41Z",
      "expires_in_seconds": 259080,
      "deadline_at": null
    }
  ]
}

total is the number of matching tasks; page and limit are the paging applied.

Recovering lost work

If your process crashed, lost a 202, or never waited for one, GET /v1/tasks?fetched=false lists the finished results that have not reached you yet. Fetch each poll_url and the task is marked as fetched.

A result that was returned to you directly on the POST already counts as fetched, so fetched=false works as an inbox: it holds only results nobody has received.

Idempotency

Send an Idempotency-Key header to make a POST safe to repeat. Generate one key, such as a UUID, for each logical request, and send the same key again when you retry after a network error or a client timeout.

• Up to 128 characters. A longer key returns 400 INVALID_PARAMETER with "field": "Idempotency-Key".
• Works on every task endpoint. Domain Research, YouTube and Trends also keep their idempotency_key body field. If you send both, the body field wins.
• The same key from the same API user to the same endpoint always refers to the same task:

The task is charged once, however many times you send the key. The key stays reserved as long as the result is stored, which is 72 hours after the task finishes. After that, the same key starts a new task.

The body of a repeated request is not compared with the original. A request with different parameters needs a new key.

curl -X POST https://data.semust.com/v1/serp \
  -H "SEMUST-API-USER: your_username" \
  -H "SEMUST-API-PASSWORD: your_password" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 0c5d2e8a-61f4-4b39-a7de-58c90f2b4e17" \
  -d '{"keyword": "seo tools", "language": "en", "country": "US"}'

Retention

Store anything you need longer on your side. Eventually the task record itself is removed, and its id then returns 404 TASK_NOT_FOUND.

Billing

• A task is charged when it completes successfully, whether or not you ever poll it.
• A failed task is refunded in full, including one that passed its deadline.
• Polling and listing are free.
• Cache hits work as before. They are answered immediately and billed as each endpoint's page describes.
• Each endpoint's pricing is unchanged. The balance is checked and held when you POST, so 402 INSUFFICIENT_CREDITS comes back on the POST, never on a poll. Whatever the task does not use is refunded when it finishes.

Limits

Polling does not count against the per-minute or daily limits of your POST requests, so collecting results never stops you from submitting new ones.

With many tasks in flight, call GET /v1/tasks?fetched=false every few seconds and fetch only the finished ones, rather than polling each task separately.

Reliability

You can count on the following:

• The task is stored before work starts. If you get a task_id back, the task exists.
• Results travel through a durable queue. A deploy or restart of our API does not lose a task. It finishes, and you can fetch it.
• Your connection is not the task's lifeline. If you disconnect, time out or crash while waiting, the task carries on. Fetch it by id, or find it with GET /v1/tasks?fetched=false.
• Every task has a 10-minute deadline. If no result has arrived by then, the task is marked failed with 504 TIMEOUT and refunded, within a few minutes of the deadline. Keep polling until you get a final answer rather than giving up at exactly 10 minutes.
• Money moves exactly once per task. One hold, then one charge or one refund, however many times you poll or repeat the same Idempotency-Key.

Errors

Errors have the shape {"error", "code", "hint", "retryable", ...}. Send Accept-Language: tr to get error and hint in Turkish; code never changes.

TIMEOUT no longer means a slow HTTP request. A slow request becomes a 202, and 504 TIMEOUT is the final answer of a task that failed to finish. See Error Codes for the full list.

{
  "error": "Task not found",
  "code": "TASK_NOT_FOUND",
  "hint": "A task can only be read with the API user that created it",
  "retryable": false
}

{
  "error": "This task's result is no longer stored",
  "code": "TASK_EXPIRED",
  "hint": "Results are kept for 72 hours after the task finishes",
  "retryable": false
}

{
  "error": "Too many tasks are still running for this API user",
  "code": "TOO_MANY_PENDING_TASKS",
  "hint": "At most 100 tasks can be pending at once; wait for some to finish",
  "retryable": true
}

{
  "error": "'wait' must be an integer from 0 to 80 (seconds)",
  "code": "INVALID_PARAMETER",
  "field": "wait",
  "retryable": false
}

Examples

A call(path, body) helper that sends the POST, polls while it gets 202 (honouring Retry-After), and returns the final response. Every request gets an Idempotency-Key, so a POST that fails with a network error is sent again safely. You handle the returned response exactly as you did before tasks existed.

import time
import uuid

import requests

BASE_URL = "https://data.semust.com"
AUTH = {
    "SEMUST-API-USER": "your_username",
    "SEMUST-API-PASSWORD": "your_password",
}


def call(path, body, wait=60, give_up_after=900):
    """POST to a Semust Data API endpoint and return the final response."""
    headers = {**AUTH, "Idempotency-Key": str(uuid.uuid4())}

    # A retry with the same Idempotency-Key joins the same task and is charged once.
    for attempt in range(3):
        try:
            response = requests.post(
                BASE_URL + path,
                params={"wait": wait},
                headers=headers,
                json=body,
                timeout=wait + 30,
            )
            break
        except (requests.ConnectionError, requests.Timeout):
            if attempt == 2:
                raise
            time.sleep(2 ** attempt)

    deadline = time.monotonic() + give_up_after
    while response.status_code == 202:
        task = response.json()
        if time.monotonic() > deadline:
            raise TimeoutError(f"task {task['task_id']} is still running")
        time.sleep(int(response.headers.get("Retry-After", task["retry_after"])))
        response = requests.get(task["poll_url"], headers=AUTH, timeout=30)

    return response


response = call(
    "/v1/domain-research",
    {"type": "keywords", "domain": "example.com.tr", "limit": 100},
)
data = response.json()
if response.status_code != 200:
    raise SystemExit(f"{data['code']}: {data['error']} (task {response.headers.get('X-Task-Id')})")

print(f"{data['count']} keywords, cost ${data['cost']:.6f}")

import { randomUUID } from "node:crypto";

const BASE_URL = "https://data.semust.com";
const AUTH = {
  "SEMUST-API-USER": "your_username",
  "SEMUST-API-PASSWORD": "your_password",
};
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

// POST to a Semust Data API endpoint and return the final response.
async function call(path, body, { wait = 60, giveUpAfter = 900 } = {}) {
  const headers = {
    ...AUTH,
    "Content-Type": "application/json",
    "Idempotency-Key": randomUUID(),
  };

  // A retry with the same Idempotency-Key joins the same task and is charged once.
  let response;
  for (let attempt = 0; ; attempt++) {
    try {
      response = await fetch(`${BASE_URL}${path}?wait=${wait}`, {
        method: "POST",
        headers,
        body: JSON.stringify(body),
        signal: AbortSignal.timeout((wait + 30) * 1000),
      });
      break;
    } catch (err) {
      if (attempt === 2) throw err;
      await sleep(1000 * 2 ** attempt);
    }
  }

  const deadline = Date.now() + giveUpAfter * 1000;
  while (response.status === 202) {
    const task = await response.json();
    if (Date.now() > deadline) throw new Error(`task ${task.task_id} is still running`);
    const retryAfter = Number(response.headers.get("Retry-After")) || task.retry_after;
    await sleep(retryAfter * 1000);
    response = await fetch(task.poll_url, { headers: AUTH });
  }
  return response;
}

const response = await call("/v1/domain-research", {
  type: "keywords",
  domain: "example.com.tr",
  limit: 100,
});
const data = await response.json();
if (!response.ok) {
  throw new Error(`${data.code}: ${data.error} (task ${response.headers.get("X-Task-Id")})`);
}
console.log(`${data.count} keywords, cost $${data.cost}`);

# 1. Submit. With wait=0 the answer is 202 straight away.
curl -i -X POST "https://data.semust.com/v1/serp?wait=0" \
  -H "SEMUST-API-USER: your_username" \
  -H "SEMUST-API-PASSWORD: your_password" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 0c5d2e8a-61f4-4b39-a7de-58c90f2b4e17" \
  -d '{"keyword": "seo tools", "language": "en", "country": "US"}'

# HTTP/1.1 202 Accepted
# Location: /v1/tasks/4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39
# Retry-After: 3
# ...

# 2. Collect. Wait Retry-After seconds and repeat while the status is 202.
#    The final answer is the same response the POST would have returned.
curl -i https://data.semust.com/v1/tasks/4b7e2f1c-9a3d-4e8b-b6c2-0d5f8a1e7c39 \
  -H "SEMUST-API-USER: your_username" \
  -H "SEMUST-API-PASSWORD: your_password"