Web Scraping API for Every Use Case

The only scraping API designed for autonomous AI agents. No proxies. No maintenance. Just reliable data extraction.

Get Your Free API Key

Turn any website into an API with just a few lines of code

Partners

API Endpoints

from scrapegraph_py import Client
from scrapegraph_py.logger import sgai_logger

sgai_logger.set_logging(level="INFO")

# Initialize the client with your API key
client = Client(api_key="your-api-key")

# Extract product information from an e-commerce page
response = client.smartscraper(
    website_url="https://example-shop.com/product",
    user_prompt="Extract the product name, price, description, availability, and customer ratings"
)

response_with_schema = client.smartscraper(
    website_url="https://example-shop.com/product",
    user_prompt="Extract product details",
    output_schema=schema
)

client.close()

from scrapegraph_py import Client
from scrapegraph_py.logger import sgai_logger

sgai_logger.set_logging(level="INFO")

# Initialize the client with your API key
client = Client(api_key="your-api-key")

# Extract product information from an e-commerce page
response = client.smartscraper(
    website_url="https://example-shop.com/product",
    user_prompt="Extract the product name, price, description, availability, and customer ratings"
)

response_with_schema = client.smartscraper(
    website_url="https://example-shop.com/product",
    user_prompt="Extract product details",
    output_schema=schema
)

client.close()

API Output

{
  "request_id": "12345-67890-abcdef",
  "result": {
    "product_name": "Wireless Bluetooth Headphones",
    "price": 89.99,
    "in_stock": true,
    "rating": 4.5,
    "description": "Premium noise-cancelling headphones with 30-hour battery life",
    "customer_reviews": [
      {
        "rating": 5,
        "comment": "Excellent sound quality and comfortable fit"
      },
      {
        "rating": 4,
        "comment": "Great value for money, minor connectivity issues"
      }
    ]
  },
  "metadata": {
    "execution_time": 2.3,
    "tokens_used": 1250
  }
}

{
  "request_id": "12345-67890-abcdef",
  "result": {
    "product_name": "Wireless Bluetooth Headphones",
    "price": 89.99,
    "in_stock": true,
    "rating": 4.5,
    "description": "Premium noise-cancelling headphones with 30-hour battery life",
    "customer_reviews": [
      {
        "rating": 5,
        "comment": "Excellent sound quality and comfortable fit"
      },
      {
        "rating": 4,
        "comment": "Great value for money, minor connectivity issues"
      }
    ]
  },
  "metadata": {
    "execution_time": 2.3,
    "tokens_used": 1250
  }
}

Use Cases

Built for Real-World Applications

From e-commerce monitoring to AI agent tools, ScrapeGraphAI powers data extraction for hundreds of businesses worldwide

Price Monitoring Bot

Track competitor prices on Amazon, eBay, and other e-commerce sites. Get alerts when prices drop or inventory changes.

Popular targets:

Amazon productseBay listingsShopify stores

Lead Generation Tool

Extract LinkedIn profiles, Twitter accounts, and contact information at scale without getting blocked.

Popular targets:

LinkedIn profilesTwitter usersCompany contacts

Market Research Dashboard

Aggregate reviews, ratings, and sentiment from multiple sites. Build comprehensive competitor analysis.

Popular targets:

Product reviewsApp ratingsCustomer sentiment

Real Estate Tracker

Monitor property listings on Zillow, Redfin, and local sites. Track price changes and new listings.

Popular targets:

Zillow listingsRedfin dataRental properties

News Aggregator

Collect articles from 100+ news sources. Extract headlines, content, and metadata automatically.

Popular targets:

News articlesBlog postsPress releases

AI Agent Tool

Provide agents with extremely fast web access. Perfect for RAG pipelines, autonomous research, and real-time data enrichment.

Popular targets:

RAG pipelinesAutonomous agentsReal-time data

Don't see your use case? Our AI adapts to any website structure.

Features

AI-Powered Extraction

Extract structured data from any website using advanced AI models.

No Proxies Needed

Built-in proxy rotation and browser automation handles everything.

Simple API

Just provide a URL and describe what data you want to extract.

Multi-Language SDKs

Available for Python, JavaScript, and other major programming languages.

Smart Scraping

Handles dynamic content, JavaScript rendering, and complex page structures.

Fast & Reliable

Optimized for speed with 99.9% uptime and automatic retries.

AI Agent Tool

MCP Server for Lightning-Fast Web Access

Provide agents with extremely fast web access. Perfect for RAG pipelines, autonomous research, and real-time data enrichment.

Popular Use Cases

RAG Pipelines

Enhance your retrieval-augmented generation with real-time web data

Autonomous Agents

Empower agents with instant web access for independent research

Real-time Data

Access live information for dynamic data enrichment and updates

New Integration

Ready to supercharge your AI agents?

Integrate our MCP Server to give your AI agents instant access to web data. Compatible with Claude, OpenAI, and other major AI platforms.

Compatible with Model Context Protocol (MCP) v1.0

Statistics

21K+

Stars on GitHub

27M+

Extracted Webpages

850K+

Unique Users

Pricing

💡 Yearly plans: Billed upfront with 15% savings - All credits provided immediately for the full year

pages/year

Free

$0/ month

Perfect for trying out ScrapeGraphAI.

50 credits
10 requests/minute rate limit
Community support

30,000

pages/year

Starter

$17/ month

Great for individuals and small projects.

60,000 credits per year
30 requests/minute rate limit
Email support

240,000

pages/year

Growth
Most Popular

$85/ month

Ideal for growing businesses.

480,000 credits per year
60 requests/minute rate limit
Priority support

1,500,000

pages/year

Pro

$425/ month

For demanding scraping needs.

3,000,000 credits per year
200 requests/minute rate limit
High speed scraping
Dedicated support

Custom

Enterprise

Custom

Tailored solutions for large organizations.

Personalized credits
Custom rate limit
Dedicated infrastructure
High speed scraping
Dedicated support

Credit Usage per Endpoint

Endpoint	Description	Credits per Page	Best For
SmartScraper	AI-powered structured data extraction from single pages with infinite scrolling support	10	Product details, contact info, structured content, dynamic feeds
SearchScraper	Search and scrape results from search engines	30	Market research, competitor analysis, brand monitoring
SmartCrawler	Crawl and analyze entire websites with depth control	10 AI or 2 base	Site-wide extraction, documentation analysis, competitor intelligence
Markdownify	Convert any webpage to clean Markdown document	2	Documentation, content migration, LLM preparation
AgenticScraper	AI agent that navigates and interacts with websites	15 + 10/step	Multi-step workflows, form filling, data behind interactions

What's Included in Each API Call:

✓HTTP requests, JavaScript rendering, and dynamic content handling
✓Proxy rotation, IP management, and anti-bot bypass
✓AI inference, natural language processing, and data structuring
✓Output formatting in JSON, CSV, or Markdown
✓Optional output schema for structured, validated data extraction

Model Context Protocol

Connect AI agents to ScrapeGraphAI with MCP

Use ScrapeGraphAI directly from Claude Desktop, Cursor, or any MCP-compatible AI assistant. Extract web data with simple natural language commands.

Native Integration

Works seamlessly with Claude Desktop and other MCP clients

Natural Language

Just ask your AI to scrape data - no coding required

Open Source

Community-driven development on GitHub

View on GitHub Read Documentation

MCP Server Integration with Claude Desktop

New

Testimonials

What our users say

Jason Touleyrou

"Recently, I stumbled upon an intriguing tool—Scrapegraph-ai. It's a fascinating blend of code and user prompts designed to streamline website scraping. My first attempt with Scrapegraph-ai was a breeze. It took less than five minutes to set up, and the results were immediate and impressive."

Mehandi Islam

"Web Scraping Made Easy: Automate Data Extraction with ScrapeGraphAI. This tool has revolutionized how we approach data collection."

Soham Kshirsagar

"ScrapeGraphAI is revolutionizing web scraping by leveraging Large Language Models (LLMs) to simplify the data extraction process."

Kuldeep Singh Sidhu

"Are you tired of writing scripts to scrape data from the web? 😓 ScrapeGraphAI is here for you! 🎉 It's an absolute game-changer."

Thomas Janssen

"Meet ScrapeGraphAI, a REVOLUTION in Web Scraping. With ScrapeGraphAI you use a Large Language Model to do the heavy lifting. No more hours spent debugging selectors!"

Nagesh Mustapure

"Unlocking the Power of Web Data with ScrapeGraphAI! With data growing at an exponential rate, leveraging web data effectively is crucial. Enter ScrapeGraphAI—a cutting-edge tool revolutionizing web scraping."

Jason Touleyrou

"Recently, I stumbled upon an intriguing tool—Scrapegraph-ai. It's a fascinating blend of code and user prompts designed to streamline website scraping. My first attempt with Scrapegraph-ai was a breeze. It took less than five minutes to set up, and the results were immediate and impressive."

Mehandi Islam

"Web Scraping Made Easy: Automate Data Extraction with ScrapeGraphAI. This tool has revolutionized how we approach data collection."

Soham Kshirsagar

"ScrapeGraphAI is revolutionizing web scraping by leveraging Large Language Models (LLMs) to simplify the data extraction process."

Kuldeep Singh Sidhu

"Are you tired of writing scripts to scrape data from the web? 😓 ScrapeGraphAI is here for you! 🎉 It's an absolute game-changer."

Thomas Janssen

"Meet ScrapeGraphAI, a REVOLUTION in Web Scraping. With ScrapeGraphAI you use a Large Language Model to do the heavy lifting. No more hours spent debugging selectors!"

Nagesh Mustapure

"Unlocking the Power of Web Data with ScrapeGraphAI! With data growing at an exponential rate, leveraging web data effectively is crucial. Enter ScrapeGraphAI—a cutting-edge tool revolutionizing web scraping."

Jason Touleyrou

"Recently, I stumbled upon an intriguing tool—Scrapegraph-ai. It's a fascinating blend of code and user prompts designed to streamline website scraping. My first attempt with Scrapegraph-ai was a breeze. It took less than five minutes to set up, and the results were immediate and impressive."

Mehandi Islam

"Web Scraping Made Easy: Automate Data Extraction with ScrapeGraphAI. This tool has revolutionized how we approach data collection."

Soham Kshirsagar

"ScrapeGraphAI is revolutionizing web scraping by leveraging Large Language Models (LLMs) to simplify the data extraction process."

Kuldeep Singh Sidhu

"Are you tired of writing scripts to scrape data from the web? 😓 ScrapeGraphAI is here for you! 🎉 It's an absolute game-changer."

Thomas Janssen

"Meet ScrapeGraphAI, a REVOLUTION in Web Scraping. With ScrapeGraphAI you use a Large Language Model to do the heavy lifting. No more hours spent debugging selectors!"

Nagesh Mustapure

"Unlocking the Power of Web Data with ScrapeGraphAI! With data growing at an exponential rate, leveraging web data effectively is crucial. Enter ScrapeGraphAI—a cutting-edge tool revolutionizing web scraping."

Jason Touleyrou

"Recently, I stumbled upon an intriguing tool—Scrapegraph-ai. It's a fascinating blend of code and user prompts designed to streamline website scraping. My first attempt with Scrapegraph-ai was a breeze. It took less than five minutes to set up, and the results were immediate and impressive."

Mehandi Islam

"Web Scraping Made Easy: Automate Data Extraction with ScrapeGraphAI. This tool has revolutionized how we approach data collection."

Soham Kshirsagar

"ScrapeGraphAI is revolutionizing web scraping by leveraging Large Language Models (LLMs) to simplify the data extraction process."

Kuldeep Singh Sidhu

"Are you tired of writing scripts to scrape data from the web? 😓 ScrapeGraphAI is here for you! 🎉 It's an absolute game-changer."

Thomas Janssen

"Meet ScrapeGraphAI, a REVOLUTION in Web Scraping. With ScrapeGraphAI you use a Large Language Model to do the heavy lifting. No more hours spent debugging selectors!"

Nagesh Mustapure

"Unlocking the Power of Web Data with ScrapeGraphAI! With data growing at an exponential rate, leveraging web data effectively is crucial. Enter ScrapeGraphAI—a cutting-edge tool revolutionizing web scraping."