What this XPath tester does
XPath is a query language for navigating a document tree. It is more powerful than CSS in three specific ways — matching on text, moving upward through ancestors, and selecting attribute nodes directly — and more awkward for everything else.
This tester evaluates your expression with the browser's own document.evaluate, so the results match what a real XPath engine returns rather than an approximation.
XPath syntax cheat sheet
| Expression | Matches |
|---|---|
//div | every div at any depth |
/html/body/div | absolute path from the root |
//div[@class='product'] | exact attribute value |
//div[contains(@class,'product')] | attribute contains — use this when an element has several classes |
//a[contains(text(),'Next')] | element whose text contains a string |
//a[text()='Next'] | exact text match |
//a/@href | the attribute node itself, not the element |
//div[@class='p']/text() | the text node |
//ul/li[1] | first li — XPath indexes from 1, not 0 |
//ul/li[last()] | last li |
//ul/li[position()<3] | first two |
//span[@data-key] | element that simply has the attribute |
//div[@id='a']//span | descendant at any depth |
//span/parent::div | move up to the parent |
//h2/following-sibling::p | siblings after this node |
//div[@a='1' and @b='2'] | two conditions |
The three things that catch people out
XPath indexes from 1. //li[1] is the first item, not the second. Every other language you use indexes from 0; this one does not.
`contains(@class, 'x')` is usually what you want, not `@class='x'`. An element with class="product featured" does not match @class='product', because that tests the whole attribute string. contains() does substring matching — with the caveat that contains(@class,'act') also matches class="inactive". For precision, match with surrounding spaces: contains(concat(' ',normalize-space(@class),' '),' product ').
`//` anywhere means "at any depth", including in the middle. //div//span matches a span nested ten levels inside a div. That is usually helpful and occasionally far too broad.
Relative or absolute?
An absolute path like /html/body/div[3]/div[2]/ul/li[4]/a is what devtools gives you when you right-click and copy XPath. It breaks the first time anyone adds a wrapper div anywhere above your target.
Relative XPath — //a[@class='product-link'] — anchors on something meaningful about the element rather than its exact position in the tree, and survives layout changes. Use it unless you have a reason not to.
Why an XPath works in the browser but not in your scraper
The usual culprit is JavaScript. The browser evaluates XPath against the rendered DOM; a plain HTTP scraper evaluates it against the raw HTML that came back from the server. If the content is injected by JavaScript, it is simply not present in what your scraper downloaded.
Check with view-source: or curl. If your nodes are not in the raw HTML, you need something that runs the page — which is what browser automation does.
The other two causes: content inside an <iframe> is a separate document and needs to be entered first, and default namespaces on XHTML or XML documents mean //div matches nothing until you register a prefix.
Beyond a single page
The expression is the easy part. Running it across thousands of pages, on a schedule, through logins and pagination and the occasional layout change, is the work. An Autonoly agent takes the expression you built here and does that part — writing results into a sheet, a database, or whatever comes next in the workflow.