When used creatively, XPaths can help improve the efficiency of auditing large websites. Consider this another tool in your SEO toolbelt.
There are endless types of information you can unlock with XPaths, which can be used in any category of online business.
Some popular ways to audit large sites with XPaths include:
In this guide, we’ll cover exactly how to perform these audits in detail.
What Are XPaths?
Simply put, XPath is a syntax that uses path expressions to navigate XML documents and identify specified elements.
This is used to find the exact location of any element on a page using the HTML DOM structure.
We can use XPaths to help extract bits of information such as H1 page titles, product descriptions on ecommerce sites, or really anything that’s available on a page.
While this may sound complex to many people, in practice, it’s actually quite easy!
How to Use XPaths in Screaming Frog
In this guide, we’ll be using Screaming Frog to scrape webpages.
Screaming Frog offers custom extraction methods, such as CSS selectors and XPaths.
It’s entirely possible to use other means to scrape webpages, such as Python. However, the Screaming Frog method requires far less coding knowledge.
(Note: I’m not in any way currently affiliated with Screaming Frog, but I highly recommend their software for web scraping.)
Step 1: Identify Your Data Point
Figure out what data point you want to extract.
For example, let’s pretend Search Engine Journal didn’t have author pages and you wanted to extract the author name for each article.
What you’ll do is:
- Right-click on the author name.
- Select Inspect.
- In the dev tools elements panel, you will see your element already highlighted.
- Right-click the highlighted HTML element and go to Copy and select Copy XPath.
At this point, your computer’s clipboard will have the desired XPath copied.
Step 2: Set up Custom Extraction
In this step, you will need to open Screaming Frog and set up the website you want to crawl. In this instance, I would enter the full Search Engine Journal URL.
- Go to Configuration > Custom > Extraction
- This will bring up the Custom Extraction configuration window. There are a lot of options here, but if you’re looking to simply extract text, match your configuration to the screenshot below.
Step 3: Run Crawl & Export
At this point, you should be all set to run your crawl. You’ll notice that your custom extraction is the second to last column on the right.
When analyzing crawls in bulk, it makes sense to export your crawl into an Excel format. This will allow you to apply a variety of filters, pivot tables, charts, and anything your heart desires.
3 Creative Ways XPaths Help Scale Your Audits
Now that we know how to run an XPath crawl, the possibilities are endless!
We have access to all of the answers, now we just need to find the right questions.
- What are some aspects of your audit that could be automated?
- Are there common elements in your content silos that can be extracted for auditing?
- What are the most important elements on your pages?
The exact problems you’re trying to solve may vary by industry or site type. Below are some unique situations where XPaths can make your SEO life easier.
1. Using XPaths with Redirect Maps
Recently, I had to redesign a site that required a new URL structure. The former pages all had parameters as the URL slug instead of the page name.
This made creating a redirect map for hundreds of pages a complete nightmare!
So I thought to myself, “How can I easily identify each page at scale?”
After analyzing the various page templates, I came to the conclusion that the actual title of the page looked like an H1 but was actually just large paragraph text. This meant that I couldn’t just get the standard H1 data from Screaming Frog.
However, XPaths would allow me to copy the exact location for each page title and extract it in my web scraping report.
In this case I was able to extract the page title for all of the old URLs and match them with the new URLs through the VLOOKUP function in Excel. This automated most of the redirect map work for me.
2. Auditing Ecommerce Sites with XPaths
Sometimes, stakeholders will need product level audits on an ad hoc basis. Sometimes this covers just categories of products, but sometimes it may be the entire site.
Using the XPath extraction method we learned earlier in this article, we can extract all types of data including:
- Product name
- Product description
- Review data
- Image URLs
- Product Category
- And much more
This can help identify products that may be lacking valuable information within your ecommerce site.
3. Auditing Blogs with XPaths
This is a more common method for using XPaths. Screaming Frog allows you to set parameters to crawl specific subfolders of sites, such as blogs.
However, using XPaths, we can go beyond simple meta data and grab valuable insights to help identify content gap opportunities.
Categories & Tags
One of the most common ways SEO professionals use XPaths for blog auditing is scraping categories and tags.
This is important because it helps us group related blogs together, which can help us identify content cannibalization and gaps.
This is typically the first step in any blog audit.
This step is a bit more Excel-focused and advanced. How this works, is you set up an XPath extraction to pull the body copy out of each blog.
Fair warning, this may drastically increase your crawl time.
Whenever you export this crawl into Excel, you will get all of the body text in one cell. I highly recommend that you disable text wrapping, or your spreadsheet will look terrifying.
Next, in the column to the right of your extracted body copy, enter the following formula:
In this formula, A1 equals the cell of the body copy.
To scale your efforts, you can have your “keyword” equal the cell that contains your category or tag. However, you may consider adding multiple columns of keywords to get a more accurate and robust picture of your blogging performance.
This formula will present a TRUE/FALSE Boolean value. You can use this to quickly identify keyword opportunities and cannibalization in your blogs.
We’ve already covered this example, but it’s worth noting that this is still an important element to pull from your articles.
When you blend your blog export data with performance data from Google Analytics and Search Console, you can start to determine which authors generate the best performance.
To do this, sort your blogs by author and start tracking average data sets including:
- Impressions – Search Console
- Clicks – Search Console
- Sessions – Analytics
- Bounce Rate – Analytics
- Conversions – Analytics
- Assisted Conversions – Analytics
Share Your Creative XPath Tips
Do you have some creative auditing methods that involve XPaths? Share this article on Twitter or tag me @seocounseling and let me know what I missed!
All screenshots taken by author, October 2019
How to increase organic traffic: 14 Practical tips
- Organic traffic is the best shareware way to attract visitors who already want to make a deal. You should conduct a good SEO analysis and take care of the quality of your site to increase it.
- You can get more organic visits if you develop a strategy, eliminate technical errors of your site, use its good mobile version, make correct external and internal optimization, optimize URLs, update the site content regularly, develop a blog with unique content, analyze competitors, and promote your site through social networks, press releases, newsjacking, emails, and messengers.
- The correct implementation of the above-mentioned tasks will provide a long-lasting result for you.
Ordinary users trust SERP more than advertising and links marked as “ads”. Correctly performed optimization, troubleshooting and the use of promising channels will quickly bring a good result.
14 Practical tips to increase organic traffic
You can get organic visits using a set of working methods, tools, and recommendations. The best 14 ways are summarized in the review below.
1. Developing a strategy to increase organic traffic
The solution to any problem begins with the development of a strategy to leave room for financial and time planning. Strategy development is carried out in stages:
- You should set goals and objectives at first
- Then, identify weaknesses using a comprehensive site audit and analyze the competitive environment
- As the next step, you can eliminate identified errors and problems
- Also, you need to select priority methods to attract organic traffic and increase the position of the site in SERP
- Then, map the work and budget, prepare a content plan (golden rule for a content plan – 60/30/10 – third-party content 60%, unique content 30%, advertising 10%)
- If you need, you should select specialists and form technical tasks
- As the last step, perform tasks, analyze results using Google Analytics
Attracting organic traffic is a rather lengthy process that distinguishes it from contextual advertising. Ads start working immediately after launch. However, organic traffic will work for a long time without any additional investments.
2. Elimination of technical errors of the site
You can identify and eliminate technical errors of the site using the following methods:
- Surface self-check
- Comprehensive site audit with the help of professionals
- Usage of paid and free services. It’s an optimal solution for those who want to get a quick result with minimal financial investments. Services allow you to identify SEO errors and ones in other key positions. The best of them are Semrush, Ahrefs, and SEOptimizer
You should eliminate identified errors by yourself or with the help of professionals. It’ll make your website more attractive to users and search engines. After the site audit, you can get rid of duplicates, speed up the download of the site, identify affiliates, and solve other problems.
3. Mobile version of the site
More than 65% of internet users prefer to select and order products from mobile devices. You can’t lose such a huge audience and should take care of your site mobile version. It allows you to increase target audience coverage several times, increase sales and subscriptions. You can create a separate mobile version or use an adaptive design of your main site. In the last case, there will be an automatic adjustment to the screens of different devices.
4. Correct external and internal website optimization
It’s aimed to obtain links from third-party sites. External links that aren’t protected from indexing transfer a part of a donor weight to the acceptor site. When working on building an external link mass, you should consider:
- Donor site trust, spam level of backlinks. The first index should be high, the second one – low
- Rules of posting links. It’s recommended to surround them with content
- Donor site topics (should be related)
- Frequency of placement. You should increase the link juice gradually. It’s especially important for young sites that have a low level of trust in search engines. A sharp increase can lead to the pessimization of the acceptor site
It helps to make the site relevant to those queries you carry out the promotion. It consists of:
- Keyword list collection
- Keywords grouping
- Preparing and publishing content optimized with LSI and SEO
- Formation and optimization of meta tags: title and description, headings and subheadings, image tags
- Creation of robots.txt files and sitemap.xml (if it’s not generated automatically)
- Interlinking and other related work
It’s important to ensure that meta tags and content are supplemented with relevant keywords but are not spammed. Otherwise, you can fall under search engine filters.
5. URLs optimization
You can complement URLs with keywords. It makes them more understandable for website visitors. When optimizing URLs, it’s recommended:
- Use from three to five relevant words, longer links will be cut off in the SERP
- Use hyphens rather than underscores
- Take into account spam indicators. Keywords from URLs are added to the overall frequency on the page
Optimized URLs look more attractive so visitors click on them more likely.
6. Regular content updates
Content updates are a rather important factor which influences on ranking. We speak about updating previously posted materials as well as publishing new ones. It helps to keep pace, increase credibility, have a positive effect on indexing.
You should carry out updates regularly following the content plan. It allows you to work with new keywords and attract organic traffic from search engines.
A blog is a valuable resource necessary for attracting organic traffic not only for commercial but also for information requests. We used to carefully choose the goods before the deal. A blog with interesting and relevant content increases chances that after reading the review, the visitor will perform the target action.
On the blog, you can publish news, information materials, as well as infographics, video reviews – everything that can attract attention and encourage visitors to make a deal. When writing articles for a blog, you can use the links to the catalog. So that the client can immediately buy the product they like without spending time searching the site.
8. Expertise and uniqueness of the content
Usage of non-unique content is a deliberately losing thing. As a result of it, you can get a claim from the copyright holder. Therefore, it’s necessary to create and optimize your content that will provide organic visits. This rule applies not only to texts but also to photos, pictures, videos. In the case of publishing someone else’s content, you must obtain the permission of the copyright holder and give a link to the source.
There is one more caveat – expertise, which plays an important role in ranking issues. Search engines don’t focus on quality optimization but on the semantic uniqueness and benefit that the content of the site can bring to the visitor. The content should answer the question that the user enters in the search bar. If the materials contain outdated, uninteresting, or knowingly untruthful data, the visitor will leave the site. An increasing number of failures will hurt ranking.
9. Promotion in social networks
Social networks are an effective tool with which you can manage opinions and drive traffic to your website. You can create a group for communication with potential customers and publish their announcements, information about promotions, discounts, updates of the assortment, and other content that encourages them to click on the link. Before starting the campaign on social networks, you need to analyze groups of your competitors, look at the situation with ordinary user’s eyes. If the posts are interesting, the subscribers will start to like and share them. This will provide additional free advertising and reach.
10. Competitive analysis
To be the first, you should know what is happening in the competition. To solve this problem, you need to use an audit which will help:
- Define a keywords cluster
- Keep abreast of all events, updates and new products introduced by competitors
- Form advertising budgets and solve other strategic tasks
For audit, you can use online services, questionnaires, secret shoppers, newsletter subscription, analysis of social networks groups, and other tools. You can use the information you’ve got to improve and optimize your website.
11. Press releases on third party resources
Regular publication of press releases on popular sites will help to solve several problems. The first one is traffic attraction, the second – external optimization. News sites visitors click the links willingly. The only negative aspect is that it’s difficult to place such publications. You should make the most of your efforts to get a positive result in outreach and lead generating.
12. Using newsjacking
Newsjacking is one of the varieties of guerrilla marketing that provides unobtrusive advertising. The latter is served against the background of an important event not being a priority. The plus is that users will often visit the site using both search queries and aggregators or news portals. The main rule is to link the offer with a really interesting and important event. Otherwise, the tool will not work.
13. Email marketing setup
From year to year, newsletters demonstrate their effectiveness. They allow you not only to communicate with customers but also to receive visits to the site. To configure the newsletter, you must have your contact base. To collect the latter, you need to place a simple registration or subscription form on the site consisting of a minimum number of lines. After that, you can establish communication with customers, notifying them of promotions, catalog updates, and other important events.
14. Mailing in messengers
Mailing in messengers is similar to emails. However, messages in Facebook Messenger, Snapchat, or WhatsApp have a higher percentage of opening. A smartphone is always near the person, such messages are more familiar and convenient. Therefore, you should not ignore the potential of this channel. Before starting such mailing, it’s necessary to ask the client whether he/she doesn’t mind receiving advertising materials. Otherwise, the sender (you) may be blocked.
To round up
Correct external and internal optimization, work in social networks and messengers, competitive analysis, technical errors eliminating, and usability improving is priority tasks to increase organic traffic. You can perform some tasks on your own. Other ones will have to be entrusted to professionals. The correct implementation of these tasks will provide a long-lasting result, an increase in organic traffic, sales, and an influx of hot customers.
Bing Maps API, Google and YouTube ads and targeted SEO
SEO2 weeks ago
How to effectively think of SEO at every stage
Video Marketing1 week ago
Top 10 Most Common Video Mistakes to Avoid
SEO2 weeks ago
Three tactics to support your efforts
SEO3 weeks ago
Five SEO content types to power and grow your business through 2020
SEO2 weeks ago
Top five B2B digital marketing tips during COVID-19 times
SEO2 weeks ago
Studying the anatomy of a successful high-conversion landing page
Video Marketing2 weeks ago
How Red Hat Created a Shorty-Nominated Podcast About Open Source Technology
SEO3 weeks ago
Eight great tips for internal site search optimization