HTTP: 403 Forbidden

A 403 means the server understood the request and decided not to serve it. Unlike a 401, sending credentials will not change the answer by itself: the server has made a policy decision about this client, this path or this request. RFC 9110 lets a server explain why in the body, but many sites send only a generic block page.

When a page opens fine in your browser but a script gets 403, the difference is almost always something the site measures about the client: the request rate, the IP range it comes from (cloud and data-center ranges are often treated differently from home connections), missing cookies from an earlier page, or a policy that only admits interactive browsers. A firewall or bot-management layer often issues it before the request reaches the application.

What is not wrong: the URL usually exists and the server is up. A 403 is the site telling you it does not want this client to have this resource, which is a signal to change approach rather than to retry harder.

import requests

r = requests.get("https://example.com/page", timeout=30)
print(r.status_code)            # 403
print(r.headers.get("server"))  # e.g. cloudflare
print(r.text[:500])             # block page names the reason
r.raise_for_status()
# requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://example.com/page

Start every scraping project by reading robots.txt, the terms of service and any published API. Identify your client honestly with a descriptive User-Agent and contact address, keep request rates low and steady, and cache aggressively. If a site uses Cloudflare, the block usually appears as error 1020; rate-based blocks often arrive first as 429. Treat repeated 403s as a request to stop, not an obstacle.