Cloudflare scrape script
by Faded -



Apr 2015




1 Year of Service
Cloudflare scrape script
Thread starter

A simple Python module to bypass Cloudflare's anti-bot page (also known as "I'm Under Attack Mode", or IUAM), implemented as a Requests adapter. Cloudflare changes their techniques periodically, so I will update this repo frequently.
This can be useful if you wish to scrape or crawl a website protected with Cloudflare. Cloudflare's anti-bot page currently just checks if the client supports Javascript, though they may add additional techniques in the future.

Due to Cloudflare continually changing and hardening their protection page, cloudflare-scrape now uses PyExecJS, a Python wrapper around multiple Javascript runtime engines. This allows the script to easily and effectively impersonate a regular web browser without explicitly parsing and converting Cloudflare's Javascript obfuscation techniques.
The only supported Javascript engines at this time are Node.js and V8 (with or without the PyV8 module). This is due to potential security concerns with the other engines.

Note: This only works when regular Cloudflare anti-bots is enabled (the "Checking your browser before accessing..." loading page). If there is a reCAPTCHA challenge, you're out of luck. Thankfully, the Javascript check page is much more common.
For reference, this is the default message Cloudflare uses for these sorts of pages:
Checking your browser before accessing

This process is automatic. Your browser will redirect to your requested content shortly.

Please allow up to 5 seconds...
Any script using cloudflare-scrape will sleep for 5 seconds for the first visit to any site with Cloudflare anti-bots enabled, though no delay will occur after the first request.

Sorry, but you need at least 25 posts and 5 hours online time to unlock hidden content.


Send an Email

(06-16-2016, 03:28 PM)bingo Wrote: only she know coins one side not other side. life is easy she think

08-05-2015, 12:34 PM
Find Reply
Register to remove ads

The last reply on this thread is older than a month. Please do not unnecessarily bump it.

Users browsing this thread: 1 Guest(s)