TL;DR
A 402 Payment Required error means the server refuses to serve the resource until a payment or quota condition is met. In web scraping you hit 402 when an API key runs out of credits, a metered proxy or scraping service exhausts its plan, or a site puts content behind a paywall.
Where 402 Comes From
The HTTP spec reserved 402 for future digital payment systems and never standardized its behavior, so every service that uses it defines its own meaning. In practice, three sources dominate scraping logs:
Exhausted API credits. Scraping APIs, LLM providers, and data services return 402 when your balance hits zero. The request is well formed, your key is valid, there is simply nothing left to spend.
Metered infrastructure. Proxy networks and serverless platforms (Cloudflare Workers included) answer with 402 when a plan limit is reached mid-month.
Paywalled content. News sites and data portals occasionally use 402 instead of 401 or 403 for subscriber-only pages.
How to Handle 402 in a Scraper
A 402 is not retryable. Unlike a 429, waiting does not fix it, and unlike a 403, rotating proxies does not either. Retrying only burns time, so the correct handling is to fail fast and alert:
response = requests.get(api_url, headers={"Authorization": f"Bearer {key}"})
if response.status_code == 402:
raise RuntimeError(
"Provider quota exhausted: top up credits or rotate to a backup key"
)Two practices keep 402 from taking down a pipeline. First, monitor remaining credits through the provider's usage endpoint instead of discovering exhaustion in production. Second, separate quota errors from block errors in your metrics, because they need opposite responses: one is a billing action, the other is an evasion action.
402 vs 401 vs 403
- 401 Unauthorized: the server does not know who you are; credentials are missing or wrong.
- 402 Payment Required: the server knows who you are and wants money or quota.
- 403 Forbidden: the server knows who you are and refuses anyway, which in scraping usually means bot detection.
Key Takeaways
- 402 signals a quota or payment condition, almost always from an API or metered service.
- Do not retry: top up, rotate to a backup key, or upgrade the plan.
- Track credit consumption proactively so 402 never surprises a production run.
How ScrapeGraphAI Handles This
ScrapeGraphAI uses transparent credit-based pricing, and the dashboard shows remaining credits in real time with configurable low-balance alerts and auto top-up. When a request cannot be served for quota reasons the API returns an explicit, typed error rather than a bare 402, so pipelines can branch on it cleanly.