What a CSS selector tester is for
Writing a selector is guesswork until you run it. You think .product .price will pick up every price on a listing page, then your scraper returns three results instead of forty because half the prices sit inside a different wrapper. This tool closes that loop: paste the HTML, write the selector, and see the matches immediately — before the selector goes anywhere near a scraper.
The matching here is done by the browser's own querySelectorAll, not an approximation, so what you see is exactly what your code will get.
CSS selector syntax cheat sheet
| Selector | Matches |
|---|---|
.price | every element with class price |
article.product | <article> elements that also have class product |
.product .price | a .price anywhere inside a .product |
.product > .price | a .price that is a direct child of .product |
[data-id] | any element with a data-id attribute |
[data-id='a2'] | exact attribute value |
a[href^='/p/'] | attribute starts with |
a[href$='.pdf'] | attribute ends with |
a[href*='product'] | attribute contains |
tr:nth-child(2) | the second tr among its siblings |
li:first-child / li:last-child | first / last sibling |
p:not(.intro) | every p except those with class intro |
h2, h3 | either (a selector list) |
.a + .b | .b immediately after .a |
.a ~ .b | any .b that follows .a |
CSS or XPath — which should you use?
Use CSS for almost everything. It is shorter, it is what browser devtools give you, and every scraping library supports it.
Reach for XPath when you need something CSS genuinely cannot express:
Selecting by text content.
//a[contains(text(),'Next')]has no CSS equivalent.Walking upward. XPath has
parent::andancestor::; CSS has no parent selector that works in scraping libraries.Selecting an attribute itself rather than the element, e.g.
//a/@href.
Everything else is easier in CSS. The tester above switches between the two so you can compare the same intent in both.
Why a selector works in devtools but not in your scraper
This is the most common scraping bug, and the cause is nearly always the same: the page you inspected is not the page your scraper downloaded.
Devtools shows you the DOM *after* JavaScript has run. A plain HTTP request returns the raw HTML *before* it runs. If the content is rendered client-side, your selector matches nothing because the elements do not exist yet in what you fetched.
The URL box above exists to settle this in one step. Paste the page address and it loads the raw HTML the server returned — the same bytes requests.get() or curl would hand you. If your selector matches there, the selector is fine. If it matches in devtools but not here, the content is JavaScript-rendered and no amount of fixing the selector will help.
Three things to check, in order:
- Load the URL above, or view the raw source —
view-source:in the URL bar, orcurlthe page. If your target is not in there, the content is JavaScript-rendered. - Check for iframes. Content inside an
<iframe>is a separate document; a selector on the parent will never reach it. - Look for generated class names. Framework builds emit classes like
css-1x2y3zthat change on every deploy. Anchor on a stable attribute —data-testid, an id, or the element structure — instead.
If the content is JavaScript-rendered, you need something that executes the page rather than just downloading it. That is what a browser-based agent does, and it is why browser automation reaches pages that a plain HTTP scraper cannot.
From a working selector to a running scraper
Once the selector matches what you want, the code panel above gives you the same selector in parsel, BeautifulSoup, Playwright and Cheerio. That handles a page.
The part nobody writes about is what happens on page two hundred: pagination, rate limits, sessions that expire, layouts that change on a Tuesday, and the retry logic around all of it. An Autonoly agent runs the selector on a schedule, handles the pages behind logins, and writes results straight to a sheet or database — so the selector stays the only part you have to think about.