TL;DR
CSS selectors and XPath are two ways to locate elements in an HTML page. CSS selectors are shorter and more readable and cover most scraping needs. XPath is more powerful: it can navigate upward to parents, select by text content, and express conditions CSS cannot. Use CSS by default and XPath when you need its extra reach.
The Same Job, Two Syntaxes
Both CSS selectors and XPath answer the same question: given a page, which elements do I want? They differ in expressiveness and readability, not in purpose. Most parsers (BeautifulSoup with soup.select, lxml, Scrapy, Playwright) support both, so the choice is per-query, not per-project.
# CSS selector: concise, reads left to right
prices = tree.css("div.product span.price::text")
# XPath: verbose, but can do things CSS cannot
prices = tree.xpath('//div[@class="product"]//span[@class="price"]/text()')Where CSS Wins
CSS selectors are shorter, more familiar (they are the same selectors used in stylesheets), and easier to read and maintain. For the common cases (select by tag, class, id, attribute, or descendant relationship) CSS is the cleaner choice, which is why it is the default in most modern scraping.
Where XPath Wins
XPath can express things CSS simply cannot:
- Navigate to parents and ancestors. CSS only goes down the tree; XPath goes up (
../..), useful when the element you can identify is a child of the one you actually want. - Select by text content.
//a[contains(text(), "Next")]targets an element by what it says, which CSS cannot do. - Complex conditions. Positional and boolean predicates (
[position() > 1 and @data-active]) give fine-grained control.
When a page's structure forces you to select by text or reach a parent, XPath is the tool.
Practical Advice
Default to CSS for readability, and drop to XPath for the specific queries that need parent traversal or text matching. Mixing both in one scraper is normal and idiomatic. Whichever you use, remember both are selector-based and break when a site changes its markup.
Key Takeaways
- CSS selectors are shorter and readable; XPath is more powerful.
- Only XPath can traverse to parents and select by text content.
- Default to CSS, use XPath for its extra reach; both break on markup changes.
How ScrapeGraphAI Handles This
ScrapeGraphAI's extract endpoint removes selectors entirely: you describe the data you want and it reads the page by meaning, so neither a CSS class rename nor an XPath structure change breaks your extraction.