Web Unblocker — Anti-Bot Scraper (Cloudflare, DataDome, PerimeterX, Akamai)

Fetch any URL and get the page back, even when it sits behind an anti-bot wall. Point it at a URL and it returns the final HTML (and optionally clean Markdown/text and a screenshot), tells you which protection guards the site, and retries through a hardened browser on a fresh IP when it gets blocked.

Built for the most common complaint in web scraping: "I'm blocked from this website, what are my options?"

How it works

1. Fast path — an HTTP request with curl_cffi (real Chrome TLS/JA3 impersonation). Cheap and instant for unprotected pages.
2. Browser render — if the fast path is blocked or challenged, the page is rendered in Camoufox (a hardened Firefox with humanized fingerprints) which clears most JavaScript anti-bot challenges from Apify IPs.
3. Retry on a new IP — if it's still blocked, it retries on a fresh browser context with a new residential proxy IP, up to maxRetries times.

Anti-bot detection

Every result tells you what was protecting the site, matched from response headers + body fingerprints:

• Cloudflare (cf-ray, "Just a moment", challenge-platform)
• DataDome (x-datadome, captcha-delivery)
• PerimeterX / HUMAN (_px* cookies, px-captcha)
• Akamai Bot Manager (_abck, ak_bmsc)
• Imperva / Incapsula (incap_ses, x-iinfo)
• Generic CAPTCHA (reCAPTCHA / hCaptcha)

What you get

Field	Description
url / finalUrl	Requested URL and the URL after redirects
success	Whether a real (non-challenge) page was returned
statusCode	HTTP status of the final response
protectionDetected	Anti-bot vendor detected, or null
bypassed	true when a protection was present and cleared
method	http or browser
html	Full page HTML (when output includes HTML)
markdown / text	Cleaned content with boilerplate removed (optional)
wordCount	Word count of cleaned text (optional)
screenshotUrl	Link to a full-page JPEG in the key-value store (optional)
scrapedAt	ISO 8601 timestamp

Input

{
  "startUrls": [{ "url": "https://example.com/protected" }],
  "renderJs": "auto",
  "outputFormat": "html",
  "waitForSelector": "",
  "waitMs": 2000,
  "screenshot": false,
  "maxRetries": 2,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

Field	Type	Default	Description
startUrls	array	—	URLs to fetch
renderJs	string	auto	auto (HTTP first, browser if blocked), always, or never
outputFormat	string	html	html, markdown, text, or all
includeLinks	boolean	true	Keep hyperlinks in Markdown
maxChars	integer	0	Truncate Markdown/text (0 = no limit)
waitForSelector	string	—	CSS selector to wait for (browser render)
waitMs	integer	2000	Wait after load when no selector (browser render)
screenshot	boolean	false	Save a full-page screenshot
maxRetries	integer	2	Retries on a fresh IP when blocked
proxyConfiguration	object	Residential	Proxy — residential strongly recommended

Use cases

• Unblock a site you keep getting 403/429 on and parse the HTML yourself.
• Check what anti-bot a site runs before you invest in building a scraper.
• LLM/RAG ingestion of pages that need JS rendering, as clean Markdown.
• Monitoring & QA of protected pages with screenshots.

🤖 Use with Claude or ChatGPT (MCP)

Run this actor straight from Claude, ChatGPT, Cursor or any MCP client via the Apify MCP server. In Claude Desktop: Settings → Connectors → Add custom connector → https://mcp.apify.com, then ask it to fetch a URL. Or expose just this tool:

{ "mcpServers": { "apify": { "url": "https://mcp.apify.com?tools=get_anything/web-unblocker" } } }

Full guide: Connect Apify actors to Claude & ChatGPT.

Notes

• Only publicly accessible pages are fetched; no logins or paywalled content.
• Residential proxy is what makes retries effective — each retry rotates to a new IP.
• For maximum bypass reliability set renderJs to always.