ScrapeGraphAI + Agno: Fast Agents With Web Access

TL;DR

Agno is a lightweight, fast framework for building agents. ScrapeGraphAI ships as a ready-made Agno toolkit, so wiring up web access is a single import.

• Install: pip install -U agno openai scrapegraph-py (Agno from its repo, see below).
• Keys: SGAI_API_KEY and OPENAI_API_KEY.
• Import: ScrapeGraphTools from agno.tools.scrapegraph.
• Enable: pass ScrapeGraphTools(all=True) to your agent, or toggle individual tools.

Why Agno

Agno is built for speed and simplicity. You define an agent, give it a model and some tools, and call it. There's not much ceremony. That makes it a natural home for scraping, where you often just want an agent that can look something up and answer.

The integration leans into that simplicity. Instead of hand-wrapping each SDK method, ScrapeGraphAI provides a ScrapeGraphTools class you drop straight into an agent's tool list. One import, and the agent can scrape, extract, search, and crawl.

Installation

Agno currently installs from its repository, alongside a model provider and the ScrapeGraphAI SDK:

pip install -U "agno @ git+https://github.com/agno-agi/agno.git#subdirectory=libs/agno" openai scrapegraph-py

Set both keys:

export SGAI_API_KEY="your-scrapegraph-key"
export OPENAI_API_KEY="your-openai-key"

A working agent in one block

This is the whole thing. Import the toolkit, attach it to an agent, ask a question:

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.scrapegraph import ScrapeGraphTools
 
agent = Agent(
    model=OpenAIChat(id="gpt-4o"),
    tools=[ScrapeGraphTools(all=True)],
    markdown=True,
)
 
agent.print_response(
    "Use smartscraper on https://example.com to extract the page title and main heading. Return them as JSON.",
    stream=True,
)

all=True switches on every tool in the kit. The agent reads the prompt, picks smartscraper, runs it against the URL, and streams the answer back as it goes.

What's in the kit

ScrapeGraphTools exposes the full ScrapeGraphAI surface under names that map to capabilities:

• smartscraper: structured extraction driven by a prompt.
• markdownify: convert a page to clean markdown.
• searchscraper: web search.
• crawl: multi-page extraction with schema validation.
• scrape: raw HTML retrieval.

Turning tools on selectively

all=True is convenient, but giving an agent every tool isn't always the sharpest choice. A model with fewer, more relevant tools makes better decisions and spends fewer tokens deciding. The toolkit lets you enable only what a given agent needs through boolean flags like enable_smartscraper and enable_markdownify:

tools = [ScrapeGraphTools(enable_smartscraper=True, enable_searchscraper=True)]

There are a few behavior knobs worth knowing too. render_heavy_js handles pages that need full JavaScript rendering, you can pass custom headers, and crawl timeouts are configurable for slower sites.

Wrapping up

Agno keeps agents fast and minimal; the ScrapeGraphTools toolkit keeps adding web access just as minimal. Install the packages, set two keys, and pass the toolkit to your agent. Start with all=True while you explore, then narrow to the specific tools each agent needs once you know the job. For a quick scraping agent, this is about as little code as it gets.

Related Articles

• ScrapeGraphAI + CrewAI: Build Data Collection Agents - Multi-agent crews with the same scraping engine.
• ScrapeGraphAI + LlamaIndex: Agentic Data Extraction - Agent-driven extraction in another framework.
• ScrapeGraphAI Python SDK: Scrape, Extract, Crawl - The SDK the toolkit is built on.
• ScrapeGraphAI MCP Server: Give Your AI the Web - The same capabilities with no code at all.