TL;DR
- Use
scrapefor page content or several outputs from a known URL, such as Markdown and a screenshot. - Use
extractwhen you want specific fields from a URL or HTML/Markdown you already have. - Use
searchto find sources across the web; usecrawlto follow pages within a site. - Use
monitorwhen you need to check a known page repeatedly and track changes.
Both scrape and extract can return structured JSON. Choose between them based on the input you have and whether you need other page formats in the same request.
Run the examples in the companion notebook with your own API key. It extracts fields from sample HTML, then captures Markdown and JSON together from a public practice page. Running both examples uses API credits.
A product price tracker and a documentation importer both collect web data. They need different things: the tracker needs a price and currency; the importer needs readable pages. Choosing the service starts there.
Choose by the input and output
| What you have | What you need | Start with |
|---|---|---|
| One page URL | Markdown, HTML, a screenshot, branding, or several formats | scrape |
| A URL or saved HTML/Markdown | Named fields in structured JSON | extract |
| A question or topic, with sources still to find | Search results and their content, optionally structured answers | search |
| A starting URL on a site | Content from linked pages in a defined section | crawl |
| A known page to revisit | Scheduled captures and differences between checks | monitor |
Read the table from the input column. If you already have a list of product URLs, process those URLs directly. Search becomes useful when you need to discover additional sources. A crawl becomes useful when you need to follow a site's links.
For credentials and full request examples, use the ScrapeGraphAI API guide. The V2 announcement covers the release and migration.
Scrape vs extract: decide what to keep from the page
Suppose you're importing a help article into a retrieval system. Use scrape with Markdown so your app can store the text. If the reviewer also needs to see the page layout, request a screenshot in the same call.
If your database expects name, price, and currency, choose extract with a prompt and schema describing those fields. It accepts a URL, but it also accepts saved HTML or Markdown, which matters when another part of your app has already fetched the page.
scrape also has a json format with its own prompt and schema. A product record plus a screenshot can therefore come from one scrape request. You don't need to chain scrape and extract just because the output contains JSON.
A practical choice:
| Required result | Choice |
|---|---|
| Article text for a retrieval system | scrape with Markdown |
| Product fields from saved HTML | extract |
| Product fields from a URL | extract, or scrape with JSON |
| Product fields and a screenshot of the source | scrape with JSON and screenshot formats |
The Scrape formats documentation lists the combinations. The Extract documentation covers URL, HTML, and Markdown inputs.
A worked choice: product data from saved HTML
Imagine your catalog importer already has the HTML for a product. Its database needs a product name, numeric price, currency, and stock status. That is an extract job: discovery and fetching have already happened.
This small HTML fixture keeps the example reproducible. It is sample product data, not a live retailer result. Set SGAI_API_KEY to your API key before running the request.
curl --fail-with-body https://v2-api.scrapegraphai.com/api/extract \
-H "SGAI-APIKEY: $SGAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"html": "<article><h1>Trail Bottle 750 ml</h1><p>Price: EUR 24.00</p><p>In stock</p></article>",
"prompt": "Extract the product name, numeric price, three-letter currency code, and whether it is in stock. Use null for any field missing from the HTML.",
"schema": {
"type": "object",
"properties": {
"name": { "type": ["string", "null"] },
"price": { "type": ["number", "null"] },
"currency": { "type": ["string", "null"] },
"inStock": { "type": ["boolean", "null"] }
},
"required": ["name", "price", "currency", "inStock"],
"additionalProperties": false
}
}'The test returned the following extracted object. The API returns it inside the response's json field, alongside request metadata.
{
"name": "Trail Bottle 750 ml",
"price": 24,
"currency": "EUR",
"inStock": true
}The schema separates a numeric price from its currency and allows an unknown value. A missing stock status stays separate from an explicit false. Validate the result against the source before storing it; a schema checks the shape of data, not whether every extracted value is correct.
The notebook also runs scrape against a Books to Scrape product page with Markdown and JSON formats in one request. That example returned the book title, a numeric price, and its currency alongside the page text.
Keep the source URL and capture time with the record in your own database. They're useful when someone asks where a price came from. If you only have a URL, replace the html input with url and let Extract fetch the page.
Search vs crawl: find sources or follow a site
Use search when the input is a question such as "retailers selling insulated bottles in Italy." It returns web results with fetched content. Add a prompt and schema if you want to extract structured information from those results. The Search documentation explains that combined request.
Use crawl when you have already chosen a site and need its linked pages. A store category or a documentation section is a useful starting point. Set page and depth limits, and use URL patterns to keep the job within the section you want.
A crawl can request JSON per page as well as Markdown. If those records already meet your requirements, you can use them directly. There is no need to send every page through a second extraction step.
For a catalog importer, search might identify candidate stores once. After you choose a store, crawl the relevant category and review which pages are products. Keep that approved set of URLs for subsequent runs. A search result list can vary between requests, so it is a poor substitute for a stable product inventory.
Crawls run as jobs. Wait for completion and retrieve the page results before treating an import as finished. See the Crawl documentation for status checks and fetching the resulting content.
Add monitor when the question becomes "what changed?"
A one-time product import needs today's values. A stock alert needs another check later and a way to compare the result.
For a known product URL, monitor runs scheduled captures and records differences between runs. Choose JSON with the fields you care about if the alert concerns price or stock. Choose a screenshot when visual changes are the thing you need to inspect.
Monitor can deliver tick payloads to a webhook. Your application can inspect the change information and decide whether to notify someone. A useful alert might require a price change above a chosen threshold; that is a business rule for your application.
Choose a schedule that matches the decision. Checking every minute adds work when a daily update would be sufficient. Monitor also supports pausing a watch when it is no longer needed. The Monitor documentation covers schedules, activity, and webhooks.
Check one record before expanding the job
Start the catalog example with one saved page and the fields your database needs. Check those values manually. Next, try a small set of representative pages, including one with a missing field. Add source discovery or a scheduled watch when the product requires it.
For a documentation importer, the same approach leads to a different choice: crawl a small section with Markdown output, inspect the retrieved text, then expand the scope. You can add link or screenshot formats when something in the application consumes them.
Formats, page counts, search results, and scheduled checks affect usage. Use the pricing calculator with the workload you actually intend to run. A monitoring budget needs the number of URLs and checks per day; a one-time import needs the expected page count and formats.
If a page needs JavaScript rendering, configure fetching for that page. Rendering controls how the page is collected; it does not change the decision between content capture and structured extraction. Start with the default fetch settings and add the controls your source requires.
Common questions
Does every workflow need all five services?
No. A job that converts saved HTML into product records can use extract alone. A job that stores a screenshot of one URL can use scrape. Add another service when the input, output, or schedule calls for it.
Should I scrape Markdown before extracting JSON?
Use extract directly when you have a URL and only want structured fields. If Markdown is already stored, pass that content to Extract. If you want JSON and other page formats together, request them through scrape.
Can I keep the same fields when I start monitoring?
Yes. A monitor's JSON format accepts a prompt and schema. Reuse the field definitions you have validated, then inspect the differences between checks in monitor activity.
Try your first request
Run the saved-HTML example, then replace the fixture with one of your own pages. Compare the extracted fields with that source before feeding the result into your application. For page captures, search, or crawls, start with the matching service in the ScrapeGraphAI dashboard.