Rate limits

Documentation Index

Fetch the complete documentation index at: https://developers.scrapeunblocker.com/llms.txt

Use this file to discover all available pages before exploring further.

• Concurrent requests - how many in-flight requests we’ll accept at once.
• Monthly request quota - total calls per billing period.

​

When you hit the limit

1. Run a fixed worker pool sized to your concurrency limit.
2. On 429, back off for the duration in the Retry-After header (if present), otherwise 5 seconds.
3. Resume.

import time
import requests

def call(url, key):
    while True:
        r = requests.post(
            "https://api.scrapeunblocker.com/getPageSource",
            params={"url": url},
            headers={"x-scrapeunblocker-key": key},
            timeout=120,
        )
        if r.status_code == 429:
            wait = int(r.headers.get("Retry-After", "5"))
            time.sleep(wait)
            continue
        return r

​

Monthly quota

​

Need more throughput?