HTTP errors in web scraping and automation · error message

HTTP: TooManyRedirects: Exceeded 30 redirects

ConfigurationClient and connection exceptions

Last verified

The exact message
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

What it means

Requests follows redirects automatically, up to Session.max_redirects, which defaults to 30. When a chain is longer than that, it stops and raises TooManyRedirects. In practice the chain is almost never 30 legitimate hops; it is a loop where page A redirects to B and B back to A.

Loops in scripts usually involve cookies. A site sets a cookie on the first response and redirects; the browser stores it and the next request succeeds. A client that does not keep cookies between redirects, or rejects them, gets redirected again forever. Consent walls, login redirects, and locale or device redirects (to a mobile site that redirects back) cause the same pattern. Browsers show the equivalent as ERR_TOO_MANY_REDIRECTS.

What is not wrong: the server is up and reachable.

What it looks like

r = requests.get(url, allow_redirects=False, timeout=30)
while r.is_redirect:
    print(r.status_code, r.headers["Location"])
    r = requests.get(r.headers["Location"], allow_redirects=False, timeout=30)

Why it happens

  • Cookies set during the redirect chain are not persisted, so the site keeps redirecting to set them.
  • A login or consent page that redirects unauthenticated clients back to itself.
  • HTTP to HTTPS and HTTPS to HTTP rules conflicting on the server (common behind misconfigured proxies).
  • Locale, region or device redirects that bounce between variants.

How to fix it

  1. Disable automatic redirects (allow_redirects=False) and print each Location header to see the loop.
  2. Use a requests.Session so cookies persist across the chain.
  3. Request the final canonical URL directly.
  4. If the loop goes through a login or consent page, use the site's documented API or authentication instead.
  5. If you own the site, check for conflicting redirect rules between the CDN, proxy and application.

How to stop it recurring

Always use a Session, store canonical URLs after the first successful fetch, and log redirect chains during development so loops surface before production.

Official documentation: requests.readthedocs.io/en/latest/api/#requests.TooManyRedirects. See all HTTP error messages.

Related errors

Definitions

Quick answers

What does "TooManyRedirects: Exceeded 30 redirects" mean in HTTP?

Requests follows redirects automatically, up to Session.max_redirects , which defaults to 30. When a chain is longer than that, it stops and raises TooManyRedirects . In practice the chain is almost never 30 legitimate hops; it is a loop where page A redirects to B and B back to A. Loops in scripts usually involve cookies.

How do I fix "TooManyRedirects: Exceeded 30 redirects"?

1. Disable automatic redirects (allow_redirects=False) and print each Location header to see the loop. 2. Use a requests.Session so cookies persist across the chain. 3. Request the final canonical URL directly. 4. If the loop goes through a login or consent page, use the site's documented API or authentication instead. 5. If you own the site, check for conflicting redirect rules between the CDN, proxy and application.

Retries and timeouts without writing the loop

Autonoly's HTTP Request node and browser steps have per-node retries, timeouts and continue-on-fail, so a transient 429 or 503 can back off instead of failing the run. It has no proxies or IP rotation: if a site blocks automated traffic, that block applies to Autonoly too.

Also see Error handling in workflows.