Master XPath: How to Use contains() and contains(text()) to Find Elements

XPath is a powerful tool for querying and navigating XML and HTML documents, making it essential for browser automation and web scraping. Many automation and web scraping frameworks, including Selenium, Cypress, Playwright, and Scrapy, support XPath for locating HTML elements.

Whether you're using XPath for automated tasks or scraping data, mastering XPath is key to accurately locating elements. In this article, we'll focus on two important XPath functions—contains() and contains(text())—to help you refine your queries. Let’s get started!

What is XPath

XPath, or XML Path Language, is an expression language used to navigate and select elements in an HTML or XML document. It provides a way to locate elements on a web page based on their tag name, attributes, position, text content, and more in the document’s hierarchy.

The XPath expression is constructed according to the position of the HTML element in the document's hierarchy. It's like a map that leads you to the target from a starting point. Therefore, you can use XPath to locate an element on a web page when its ID, class, name, and other attributes are unavailable and you can’t use DOM.

Here's a quick overview of XPath syntax:

• / selects all direct child elements of the current element.
• // selects all descendant elements of the current element, regardless of where they are in the document.
• @ selects the attributes of the current element.
• * selects any element.

For example, the XPath expression //div[@class='header'] selects all <div> elements with the class "header".

🐰 Hare Hint: While // is powerful, it can slow down your queries as it searches the entire document. Avoid overusing // and use more specific paths when possible.

What is contains() in XPath

The contains() function allows you to locate an element by matching part of its attribute or text content with a specific string. This is especially useful when dealing with dynamic attributes like IDs or classes that include generated parts (e.g., id="user_12345"). In these cases, you can use contains() to match any elements with an ID that includes "user_".

How to Use contains() in XPath

The contains() function helps you locate elements by checking if part of their attribute or text content matches a specified string. It takes two arguments:

contains(arg1, arg2)

• arg1: The attribute or text content to search.
• arg2: The substring you're looking for within arg1.

The function returns true if arg2 is found within arg1; otherwise, it returns false.

When used within [] after a tag name in an XPath expression, it helps locate an HTML element on a webpage by checking if its attribute contains a specific text. Here's the basic syntax:

//tag[contains(@attribute, 'value')]

For example, if you want to find all <button> elements with a class that includes the word "secondary", you can use contains() like this:

//button[contains(@class, 'secondary')]

This selects all button elements whose class name contains the substring "secondary", such as:

<button class="submit-btn-secondary">Submit</button>
<button class="submit-btn-secondary-large">Submit Form</button>

🐰 Hare Hint: XPath is case-sensitive by default. Make sure your strings match the exact case used in the HTML attributes or text. Otherwise, use translate() to convert to lowercase for case-insensitive matching.

How to Use contains(text()) in XPath

The contains(text()) function works like contains(), but it specifically searches within the text content of an element instead of its attributes. This is useful when you need to locate an element based on partial text content. Here's the basic syntax:

//tag[contains(text(), 'partialText')]

• partialText: The substring of the text content you want to match.

For example, if you want to find all <h1> elements that include the word "How", you can use contains() like this:

//h1[contains(text(), 'How')]

This selects all h1 elements whose text contains the substring "How", such as:

<h1>How to Use contains() in XPath</h1>
<h1>How to Use contains(text()) in XPath</h1>

🐰 Hare Hint: Sometimes, text nodes might have leading or trailing spaces. To handle this,  you can use contains(text()) together with normalize-space() to remove extra spaces and then check for the substring, e.g., //p[contains(normalize-space(text()), 'learn more ')].

Using Operators to Combine the contains() Expressions

When working with complex HTML documents, there may be scenarios where you need to locate elements based on multiple conditions. In these cases, combining contains() expressions with operators like and and or can make your XPath queries more precise and efficient.

Here are the logical operators that you can use to chain multiple conditions together:

• and: Combines two conditions within a single XPath expression and returns true only if both are satisfied.
• or: Combines two conditions within a single XPath expression and returns true if at least one condition is satisfied.
• | (union): Combines multiple XPath expressions into one, allowing you to select elements that match any of the specified expressions, regardless of their context or position in the document.

Example: Using “and” with contains()

Suppose you want to find a <div> element that has both a specific class and text content containing a particular word. For example, if you're looking for a <div> with a class name that includes "card" and text that contains "Flash Sale", you can use:

//div[contains(@class, 'card') and contains(text(), 'Flash Sale')]

This will select elements like:

<div class="special-card">Flash Sale - Limited Time!</div>

Example: Using “or” with contains()

If you want to select elements that match either of two conditions, you can use or. For example, to find <button> elements with text containing either "Submit" or "Confirm":

//button[contains(text(), 'Submit') or contains(text(), 'Confirm')]

This would match buttons like:

<button class="btn-primary">Submit</button>
<button class="btn-secondary">Confirm</button>

Example: Using “|” with contains()

Suppose you want to select all <h1> elements that contain the text "How" or all <h2> elements that contain the text "What", here's how you would write the XPath expression:

//h1[contains(text(), 'How')] | //h2[contains(text(), 'What')]

This would match elements like:

<h1>What is XPath</h1>
<h2>How to Use XPath</h2>

🐰 Hare Hint: Although the or and | operators seem similar, they serve different purposes. The or operator checks multiple conditions for the same element , while the | operator combines results from multiple XPath expressions, targeting different elements or parts of the document.

Different Behaviours Among XPath Versions

XPath has multiple versions, and different browsers may use different ones. Later versions, like XPath 2.0, extend the capabilities of XPath 1.0 by supporting a wider range of data types. However, newer versions remain backward compatible, meaning most XPath 1.0 expressions can still be used in XPath 2.0 and beyond.

Since there are slight differences between these versions, the functions behave differently as well.

XPath 1.0

When using the contains(arg1, arg2) function, arg1 can accept multiple items, including a node set containing a group of nodes. In this case, the node set is converted to a string, using the string value of the first node only.

For example, using the //*[contains(text(), 'target string')] expression, the element below can be located, since "target string" is the first node.

<div>
	<p>target string <br/>other strings</p>
</div>

If it’s not in the first node, the function will return false and the element cannot be located.

<div>
	<p>other strings <br/>target string</p>
</div>

XPath 2+

In XPath 2.0 and above, it is an error to use contains(arg1, arg2) with more than one item as the first argument. As a result, the XPath expression from the previous example will return an error and cannot be used to locate the target string in both cases.

Instead of using the previous XPath expression, use this:

//*[text()[contains(., 'target string' )]]

In XPath, the dot (".") refers to a single node. When used in contains(), it matches the string with each individual node, regardless of its position. Therefore, in XPath 2.0 and above, it can be used to find the first element where "target string" is the first node.

Since the position of the string doesn't matter, it will return true for the second element too. This also applies to XPath 1.0.

Using XPath in Browser Automation and Web Scraping

XPath plays an important role in web automation and scraping, especially in tools like Selenium and other automation frameworks. Its ability to accurately locate web page elements using attributes, text, or structure makes it a valuable skill for browser automation and web scraping.

While mastering XPath is key for using automation frameworks like Selenium, Cypress, and Playwright, no-code tools like Roborabbit offer a simpler alternative. It lets you automate web tasks with minimal coding and eliminates the need for complex XPath expressions. Here’s an example of how you can use Roborabbit for web scraping:

Whether you're a developer or a non-technical user, learning XPath can be helpful for automating web tasks or scraping data. However, if you're looking for an easier way to accomplish these tasks, consider trying Roborabbit—it offers 100 free credits when you sign up!