Connect with us


How to Use XML Sitemaps to Boost SEO



Jes Scholz

As the web evolves, so too does Google and SEO.

This means what is considered best practice is often in flux. What may have been good counsel yesterday, is not so today.

This is especially true for sitemaps, which are almost as old as SEO itself.

The problem is, when every man and their dog has posted answers in forums, published recommendations on blogs and amplified opinions with social media, it takes time to sort valuable advice from misinformation.

So while most of us share a general understanding that submitting a sitemap to Google Search Console is important, you may not know the intricacies of how to implement them in a way that drives SEO key performance indicators (KPIs).

Let’s clear up the confusion around best practices for sitemaps today.

In this article we cover:

What Is an XML Sitemap

In simple terms, an XML sitemap is a list of your website’s URLs.

It acts as a roadmap to tell search engines what content is available and how to reach it.

sitemap vs website crawling

In the example above, a search engine will find all nine pages in a sitemap with one visit to the XML sitemap file.

On the website, it will have to jump through five internal links to find page 9.

This ability of an XML sitemap to assist crawlers in faster indexation is especially important for websites that:

  • Have thousands of pages and/or a deep website architecture.
  • Frequently add new pages.
  • Frequently change content of existing pages.
  • Suffer from weak internal linking and orphan pages.
  • Lack a strong external link profile.

Side note: Submitting a sitemap with noindex URLs can also speed up deindexation. This can be more efficient than removing URLs in Google Search Console if you have many to be deindexed. But use this with care and be sure you only add such URLs temporarily to your sitemaps.

Key Takeaway

Even though search engines can technically find your URLs without it, by including pages in an XML sitemap you’re indicating that you consider them to be quality landing pages.

While there is no guarantee that an XML sitemap will get your pages crawled, let alone indexed or ranked, submitting one certainly increases your chances.

XML Sitemap Format

A one-page site using all available tags would have this XML sitemap:

xml sitemap format and tags

But how should an SEO use each of these tags? Is all the metadata valuable?

Loc (a.k.a. Location) Tag

This compulsory tag contains the absolute, canonical version of the URL location.

It should accurately reflect your site protocol (http or https) and if you have chosen to include or exclude www.

For international websites, this is also where you can implement your hreflang handling.

By using the xhtml:link attribute to indicate the language and region variants for each URL, you reduce page load time, which the other implementations of link elements in the <head> or HTTP headers can’t offer.

Yoast has an epic post on hreflang for those wanting to learn more.

Lastmod (a.k.a. Last Modified) Tag

An optional but highly recommended tag used to communicate the file’s last modified date and time.

John Mueller acknowledged Google does use the lastmod metadata to understand when the page last changed and if it should be crawled. Contradicting advice from Illyes in 2015.

The last modified time is especially critical for content sites as it assists Google to understand that you are the original publisher.

It’s also powerful to communicate freshness, but be sure to update modification date only when you have made meaningful changes.

Trying to trick search engines that your content is fresh, when it’s not, may result in a Google penalty.

Changefreq (a.k.a. Change Frequency) Tag

Once upon a time, this optional tag hinted how frequently content on the URL was expected to change to search engines.

But Mueller has stated that “change frequency doesn’t really play that much of a role with sitemaps” and that “it is much better to just specify the time stamp directly”.

Priority Tag

This optional tag that ostensibly tells search engines how important a page is relative to your other URLs on a scale between 0.0 to 1.0.

At best, it was only ever a hint to search engines and both Mueller and Illyes have clearly stated they ignore it.

Key Takeaway

Your website needs an XML sitemap, but not necessarily the priority and change frequency metadata.

Use the lastmod tags accurately and focus your attention on ensuring you have the right URLs submitted.

Types of Sitemaps

There are many different types of sitemaps. Let’s look at the ones you actually need.

XML Sitemap Index

XML sitemaps have a couple of limitations:

  • A maximum of 50,000 URLs.
  • An uncompressed file size limit of 50MB.

Sitemaps can be compressed using gzip (the file name would become something similar to sitemap.xml.gz) to save bandwidth for your server. But once unzipped, the sitemap still can’t exceed either limit.

Whenever you exceed either limit, you will need to split your URLs across multiple XML sitemaps.

Those sitemaps can then be combined into a single XML sitemap index file, often named sitemap-index.xml. Essentially, a sitemap for sitemaps.

For exceptionally large websites who want to take a more granular approach, you can also create multiple sitemap index files. For example:

  • sitemap-index-articles.xml
  • sitemap-index-products.xml
  • sitemap-index-categories.xml

But be aware that you cannot nest sitemap index files.

For search engines to easily find every one of your sitemap files at once, you will want to:

  • Submit your sitemap index(es) to Google Search Console and Bing Webmaster Tools.
  • Specify your sitemap index URL(s) in your robots.txt file. Pointing search engines directly to your sitemap as you welcome them to crawl.

xml sitemap in robots.txt

You can also submit sitemaps by pinging them to Google.

But beware:

Google no longer pays attention to hreflang entries in “unverified sitemaps”, which Tom Anthony believes to mean those submitted via the ping URL.

XML Image Sitemap

Image sitemaps were designed to improve indexation of image content.

In modern day SEO, however, images are embedded within page content, so will be crawled along with the page URL.

Moreover, it’s best practice to utilize JSON-LD markup to call out image properties to search engines as it provides more attributes than an image XML sitemap.

Because of this, an XML image sitemap is unnecessary for most websites. Including an image sitemap would only waste crawl budget.

The exception to this is if images help drive your business, such as a stock photo website or ecommerce site gaining product page sessions from Google Image search.

Know that images don’t have to be on the same domain as your website to be submitted in a sitemap. You can use a CDN as long as it’s verified in Search Console.

XML Video Sitemap

Similar to images, if videos are critical to your business, submit an XML video sitemap. If not, a video sitemap is unnecessary.

Save your crawl budget for the page the video is embedded into, ensuring you markup all videos with JSON-LD as a

Google News Sitemap

Only sites registered with Google News should use this sitemap.

If you are, include articles published in the last two days, up to a limit of 1,000 URLs per sitemap, and update with fresh articles as soon as they’re published.

Contrary to some online advice, Google News sitemaps don’t support image URL.

Google recommends to use image or og:image to specify your article thumbnail for Google News.

Mobile Sitemap

This is not needed for most websites.

Why? Because Mueller confirmed mobile sitemaps are for feature phone pages only. Not for smartphone-compatibility.

So unless you have unique URLs specifically designed for featured phones, a mobile sitemap will be of no benefit.

HTML Sitemap

XML sitemaps take care of search engine needs. HTML sitemaps were designed to assist human users to find content.

The question becomes, if you have a good user experience and well crafted internal links, do you need a HTML sitemap?

Check the page views of your HTML sitemap in Google Analytics. Chances are, it’s very low. If not, it’s a good indication that you need to improve your website navigation.

HTML sitemaps are generally linked in website footers. Taking link equity from every single page of your website.

Ask yourself. Is that the best use of that link equity? Or are you including an HTML sitemap as a nod to legacy website best practices?

If few humans use it. And search engines don’t need it as you have strong internal linking and an XML sitemap. Does that HTML sitemap have a reason to exist? I would argue no.

Dynamic XML Sitemap

Static sitemaps are simple to create using a tool such as Screaming Frog.

The problem is, as soon as you create or remove a page, your sitemap is outdated. If you modify the content of a page, the sitemap won’t automatically update the lastmod tag.

So unless you love manually creating and uploading sitemaps for every single change, it’s best to avoid static sitemaps.

Dynamic XML sitemaps, on the other hand, are automatically updated by your server to reflect relevant website changes as they occur.

To create a dynamic XML sitemap:

  • Ask your developer to code a custom script, being sure to provide clear specifications
  • Use a dynamic sitemap generator tool
  • Install a plugin for your CMS, for example the Yoast SEO plugin for WordPress

Key Takeaway

Dynamic XML sitemaps and a sitemap index are modern best practice. Mobile and HTML sitemaps are not.

Use image, video and Google News sitemaps only if improved indexation of these content types drive your KPIs.

XML Sitemap Indexation Optimization

Now for the fun part. How do you use XML sitemaps to drive SEO KPIs.

Only Include SEO Relevant Pages in XML Sitemaps

An XML sitemap is a list of pages you recommend to be crawled, which isn’t necessarily every page of your website.

A search spider arrives at your website with an “allowance” for how many pages it will crawl.

The XML sitemap indicates you consider the included URLs to be more important than those that aren’t blocked but aren’t in the sitemap.

You are using it to tell search engines “I’d really appreciate it if you’d focus on these URLs in particular.”

Essentially, it helps you use crawl budget effectively.

By including only SEO relevant pages, you help search engines crawl your site more intelligently in order to reap the benefits of better indexation.

You should exclude:

  • Non-canonical pages.
  • Duplicate pages.
  • Paginated pages.
  • Parameter or session ID based URLs.
  • Site search result pages.
  • Reply to comment URLs.
  • Share via email URLs.
  • URLs created by filtering that are unnecessary for SEO.
  • Archive pages.
  • Any redirections (3xx), missing pages (4xx) or server error pages (5xx).
  • Pages blocked by robots.txt.
  • Pages with noindex.
  • Resource pages accessible by a lead gen form (e.g., white paper PDFs).
  • Utility pages that are useful to users, but not intended to be landing pages (login page, contact us, privacy policy, account pages, etc.).

I want to share an example from Michael Cottam about prioritizing pages:

Say your website has 1,000 pages. 475 of those 1,000 pages are SEO relevant content. You highlight those 475 pages in an XML sitemap, essentially asking Google to deprioritize indexing the remainder.

Now, let’s say Google crawls those 475 pages, and algorithmically decides that 175 are “A” grade, 200 are “B+”, and 100 “B” or “B-”. That’s a strong average grade, and probably indicates a quality website to which to send users.

Contrast that against submitting all 1,000 pages via the XML sitemap. Now, Google looks at the 1,000 pages you say are SEO relevant content, and sees over 50 percent are “D” or “F” pages. Your average grade isn’t looking so good anymore and that may harm your organic sessions.

But remember, Google is going to use your XML sitemap only as a clue to what’s important on your site.

Just because it’s not in your XML sitemap doesn’t necessarily mean that Google won’t index those pages.

When it comes to SEO, overall site quality is a key factor.

To assess the quality of your site, turn to the sitemap related reporting in Google Search Console (GSC).

Key Takeaway

Manage crawl budget by limiting XML sitemap URLs only to SEO relevant pages and invest time to reduce the number of low-quality pages on your website.

Fully Leverage Sitemap Reporting


The sitemaps section in the new Google Search Console is not as data rich as what was previously offered.

It’s primary use now is to confirm your sitemap index has been successfully submitted.


If you have chosen to use descriptive naming conventions, rather than numeric, you can also get a feel for the number of different types of SEO pages that have been “discovered” – aka all URLs found by Google via sitemaps as well as other methods such as following links.

In the new GSC, the more valuable area for SEOs in regard to sitemaps is the Index Coverage report.

How to Use XML Sitemaps to Boost SEO

The report will default to “All known pages”. Here you can:

  • Address any “Error” or “Valid with warnings” issues. These often stem from conflicting robots directives. One solved, be sure to validate your fix via the Coverage report.
  • Look at indexation trends. Most sites are continually adding valuable content, so “Valid” pages (aka those indexed by Google) should steadily increase. Understand the cause of any dramatic changes.
  • Select “Valid” and look in details for the type “Indexed, not submitted in sitemap”. These are pages where you and Google disagree on their value. For example, you may not have submitted your privacy policy URL, but Google has indexed the page. In such cases, there’s no actions to be taken. What you need to be looking out for are indexed URLs which stem from poor pagination handling, poor parameter handling, duplicate content or pages being accidently left out of sitemaps.

Afterwards, limit the report to the SEO relevant URLs you have included in your sitemap by changing the drop down to “All submitted pages”. Then check the details of all “Excluded” pages.

Reasons for exclusion of sitemap URLs can be put into four action groups:

  1. Quick wins: For duplicate content, canoncials, robots directives, 40X HTTP status codes, redirects or legalities exclusions put in place the appropriate fix.
  2. Investigate page: For both “Submitted URL dropped” and “Crawl anomaly” exclusions investigate further by using the Fetch as Google tool.
  3. Improve page: For “Crawled – currently not indexed” pages, review the page (or page type as generally it will be many URLs of a similar breed) content and internal links. Chances are, it’s suffering from thin content, unoriginal content or is orphaned.
  4. Improve domain: For “Discovered – currently not indexed” pages, Google notes the typical reason for exclusion as they “tried to crawl the URL but the site was overloaded”. Don’t be fooled. It’s more likely that Google decided “it’s not worth the effort” to crawl due to poor internal linking or low content quality seen from the domain. If you see a larger number of these exclusions, review the SEO value of the page (or page types) you have submitted via sitemaps, focus on optimizing crawl budget as well as review your information architecture, including parameters, from both a link and content perspective.

Whatever your plan of action, be sure to note down benchmark KPIs.

The most useful metric to assess the impact of sitemap optimization efforts is the “All submitted pages” indexation rate – calculated by taking the percentage of valid pages out of total discovered URLs.

Work to get this above 80%.

Why not 100%? Because if you have focussed all your energy on ensuring every SEO relevant URL you currently have is indexed, you likely missed opportunities to expand your content coverage.

Note: If you are a larger website who has chosen to break their site down into multiple sitemap indexes, you will be able to filter by those indexes. This will not only allow you to:

  1. See the overview chart on a more granular level.
  2. See a larger number of relevant examples when investigating a type of exclusion.
  3. Tackle indexation rate optimization section by section.

Key Takeaway

In addition to identifying warnings and errors, you can use the Index Coverage report as an XML sitemap sleuthing tool to isolate indexation problems.

XML Sitemap Best Practice Checklist

Do invest time to:

✓ Include hreflang tags in XML sitemaps

✓ Include the <loc> and <lastmod> tags

✓ Compress sitemap files using gzip

✓ Use a sitemap index file

✓ Use image, video and Google news sitemaps only if indexation drives your KPIs

✓ Dynamically generate XML sitemaps

✓ Ensure URLs are included only in a single sitemap

✓ Reference sitemap index URLs in robots.txt

✓ Submit sitemap index to both Google Search Console and Bing Webmaster Tools

✓ Include only SEO relevant pages in XML sitemaps

✓ Fix all errors and warnings

✓ Analyze trends and types of valid pages

✓ Calculate submitted pages indexation rates

✓ Address causes of exclusion for submitted pages

Now, go check your own sitemap and make sure you’re doing it right.

Image Credits

Feature Image: Paulo Bobita
All screenshots taken by author

Continue Reading
Click to comment

You must be logged in to post a comment Login

Leave a Reply


5 Essential Schema Plugins for WordPress



Julia McCoy

Not all website elements are created equal.

Some matter – and some matter a ton.

So what are your site’s standout elements? Here are five:

  • Customer reviews.
  • Ratings.
  • Product pricing.
  • Recipe ingredients (if your website is about cooking).
  • Your company’s operating hours.

It’s pretty obvious to you how important these elements are to your site.

But how can you make search engines understand that?

The answer is with schema markup and rich snippets.

What Are Schema Markup & Rich Snippets?

Rich snippets are the extra pieces of information that appear around your website’s link.

Schema markup is HTML code used to create rich snippets.

Take a look at this example.

5 Essential Schema Plugins for WordPress

Let’s say you want to buy a Darth Maul toy lightsaber.

You go on Google and type “Star Wars Darth Maul Lightsaber.”

The photo above shows you the results.

As you can see, you don’t only get links to the different websites selling the toy.

You also get a bunch of additional information to help you decide which link to click.

Amazon shows you star ratings and reviews.

Entertainment Earth includes the toy’s price, length, height, and weight.

Valuable pieces of information, right?

These are your rich snippets.

And if you add them to your site, you’ll naturally boost it higher on Google’s SERPs.

How Rich Snippets Boost Your SEO Ranking

Technically, rich snippets don’t directly affect your site’s ranking on Google’s SERPs.

However, they can still get you ranked higher on Google.

For instance, let’s say you want to bake a Black Forest cake for a loved one’s birthday.

You go on Google and type “Black Forest Cake Recipe.”

Here are the results you get:

5 Essential Schema Plugins for WordPress

As a busy cake-lover, which recipe would you most likely click? One that is rated 5 stars and takes 2 hours and 45 minutes to bake? Or one that is rated 4.9 stars and takes 3 hours and 45 minutes to bake?

Naturally, you’d choose the first option.

The star rating, votes, and description (all rich snippets) encourage you to go to for your recipe.

This is how your rich snippets increase your click-through rate.

And the higher your click-through rate, the more likely Google will notice you.

So how do you add rich snippets to your site?

The Top 5 Schema Plugins for WordPress

The great news is if you’re using WordPress, you don’t have to touch a single line of code.

Simply choose a schema plugin, and creating rich snippets will be easy.

Here are the top five marketers love.

1. Schema Pro

5 Essential Schema Plugins for WordPress

Schema Pro makes adding rich snippets to your site fast and easy.

In a matter of minutes, you can add your preferred configurations to all your pages and posts.

Schema Pro supports 13 useful schema types. These are:

  • Reviews (music, movies, products, books, etc.).
  • Services.
  • Recipes (you can create your own attractive schema rich card that’ll boost your click-through rate).
  • Software applications (add reviews and star ratings to give your applications a boost).
  • Video objects.
  • Books.
  • People.
  • Products (give searchers detailed information on what you’re selling).
  • Local businesses.
  • Articles (news, blogs, etc.).
  • Courses.
  • Job postings.
  • Events.

Price: $79/month or $249/lifetime

2. All in One Schema Rich Snippets

5 Essential Schema Plugins for WordPress

All in One Schema Rich Snippets is one of the simplest plugins you can find for schema markup.

Although it’s simple, it provides you with snippets for reviews, ratings, events, articles, and software applications.

One great thing about this plugin is you can use it for free.

It doesn’t have a ton of fancy designs to select from, but it does have the basics you need for rich snippets on your site.

The downside of using this plugin is it doesn’t support automation.

You’ll have to add schema markup to each page of your site manually.

What’s interesting about All in One Schema Rich Snippets is it’s made by the same maker as Schema Pro.

In fact, its dashboard heavily advertises Schema Pro.

So how to choose between the two?

Use Schema Pro if you’re running a bigger online business.

The price will be worth it for the automation, unique designs, and schema markup support for local businesses.

If you’re just starting out and want to try something for free?

Go with All in One Schema Rich Snippets.

Price: Free

3. Schema and Structured Data for WP & AMP

5 Essential Schema Plugins for WordPress

Schema and Structured Data for WP & AMP supports 33 schema types.

Three unique ones include:

  • How To (list the steps in your how-to article to be featured in your rich snippets)
  • Q&A (if your article is in a question and answer format, you can feature the most relevant questions and answers in your rich snippets)
  • Audio object (add details about audio you upload like date of upload, length, etc. to your rich snippets)

The best part? If the schema type you’re looking for isn’t part of the 33 this plugin offers, you can request a customized type!

Here are other features this plugin offers:

  • Reviews pulled from over 75 sites.
  • Your own customized review rating boxes with schema markup.
  • Compatibility with other schema plugins like SEO Pressor and WP SEO Schema.

Price: Personal: $99, Webmaster: $149, Freelancer: $299, Agency: $499

4. WP Review Plugin

5 Essential Schema Plugins for WordPress

If you run an ecommerce site, WP Review is a good plugin option for you.

This is because its main focus is increasing your consumer interaction with product and service ratings and reviews.

If you’re a blogger, this plugin is also great because you can use it to rank and rate tools, software, recipes, or anything else you review.

Here are the top features you’ll get with WP Review:

  • Rating systems using points, percentage, or stars.
  • Unlimited color changes for individual reviews.
  • Lightweight, won’t slow your site down.
  • Supported by most WordPress themes (both free and paid).
  • 2 templates for review boxes to choose from.

WP review is easy to install and configure, making schema markup simple for the least tech-savvy person.

Price: Personal (one site): $49/year or $236/lifetime, Developer (more than one site): $149/year or $596/lifetime, Agency (affiliate marketer): $299/year or $1,196/lifetime.

5. WP SEO Structured Data Schema

5 Essential Schema Plugins for WordPress

WP SEO Structured Data Schema is a free plugin that contains all the basics you need for schema markup and rich snippets.

It supports schema types including organizations, local businesses, videos, events, and ratings.

You can also add geo coordinates, people’s names, logos, business descriptions, working hours, and more.

Price: Free

How to Find the Schema Plugin for WordPress That Fits You

Schema markup and rich snippets are essential to your site SEO.

They make your site look attractive.

They encourage people to click your link.

They help you stand out in the sea of other websites.

So how can you choose the one that’s right for you?

It’s simple.

All you must do is determine your needs.

Do you have a simple site that doesn’t require detailed rich snippets? Go for a free, user-friendly plugin.

Are you an affiliate marketer with five websites and blogs? An investment in a plugin with the latest features will be worthwhile.

As soon as you’ve chosen and installed the right plugin, you’re ready to use a powerful tool that’ll make your click-through rate soar.

More Resources:

Image Credits

All screenshots taken by author, December 2019

Continue Reading


The dangers of misplaced third-party scripts



The dangers of misplaced third-party scripts

I was recently helping one of my team members diagnose a new prospective customer site to find some low hanging fruit to share with them.

When I checked their home page with our Chrome extension, I found a misplaced canonical tag. We added this type of detection a long time ago when I first encountered the issue.

What is a misplaced SEO tag, you might ask?

Most SEO tags like the title, meta description, canonical, etc. belong in the HTML HEAD. If they get placed in the HTML BODY, Google and other search engines will ignore them.

If you go to the Elements tab, you will find the SEO tags inside the <BODY> tag. But, these tags are supposed to be in the <HEAD>!

Why does something like this happen?

If we check the page using VIEW SOURCE, the canonical tag is placed correctly inside the HTML HEAD (line 56, while the <BODY> is in line 139.).

What is happening here?!

Is this an issue with Google Chrome?

The canonical is also placed in the BODY in Firefox.

We have the same issue with Internet Explorer.

Edge is no exception.

We have the same problem with other browsers.

HTML parsing vs. syntax highlighting

Why is the canonical placed correctly when we check VIEW SOURCE, but not when we check it in the Elements tab?

In order to understand this, I need to introduce a couple of developer concepts: lexical analysis and syntax analysis.

When we load a source page using VIEW SOURCE, the browser automatically color codes programming tokens (HTML tags, HTML comments, etc).

In order to do this, the browser performs basic lexical analysis to break the source page into HTML tokens.

This task is typically performed by a lexer. It is a simple, and low-level task.

All programming language compilers and interpreters use a lexer that can break source text into language tokens.

When we load the source page with the Elements tab, the browser not only does syntax highlighting, but it also builds a DOM tree.

In order to build a DOM tree, it is not enough to know HTML tags and comments from regular text, you also need to know when a tag opens and closes, and their place in the tree hierarchy.

This syntactic analysis requires a parser.

An English spellchecker needs to perform a similar, two-phased analysis of the written text. First, it needs to translate text into nouns, pronouns, adverbs, etc. Then, it needs to apply grammar rules to make sure the part of speech tags are in the right order.

But why are the SEO tags placed in the HTML body?

Parsing HTML from Python

I wrote a Python script to fetch and parse some example pages with errors, find the canonical anywhere in the HTML, and print the DOM path where it was found.

After parsing the same page that shows misplaced SEO tags in the HTML Body, I find them correctly placed in the HTML head.

What are we missing?

Invalid tags in the HTML head

Some HTML tags are only valid in the HTML BODY. For example, <DIV> and <SPAN> tags are invalid in the HTML head.

When I looked closely at the HTML HEAD in our example, I found a script with a hardcoded <SPAN>. This means, the script was meant to be placed in the <BODY>, but the user incorrectly placed it in the head. 

Maybe the instructions were not clear, the vendor omitted this information or the user didn’t know how to do this in WordPress.

I tested by moving the script to the BODY but still faced the misplaced canonical issue.

After a bit of trial and error, I found another script that when I moved it to the BODY, the issue disappeared.

While the second script didn’t have any hardcoded invalid tags, it was likely writing one or more to the DOM.

In other words, it was doing it dynamically.

But, why would inserting invalid tags, cause the browser to push the rest of the HTML in the head to the body?

Web browser error tolerance

I created a few example HTML files with the problems I discussed and loaded them in Chrome to show you what happens.

In the first example, I commented out the opening BODY tag. This removes it.

You can see that Chrome added one automatically. 

Now, let’s see what happens if I add a <DIV> inside the HTML HEAD, which is invalid.

This is where it gets interesting. Chrome closed the HTML HEAD early and pushed the rest of the HEAD elements to the body, including our canonical tag and <DIV>.

In other words, Chrome assumed we forgot an opening <BODY> tag!

This should make it clear why misplaced tags in the HEAD can cause our SEO tags to end up in the BODY.

Now, let’s look at our second case where we don’t have a hardcoded invalid tag, but a script might write one dynamically.

Here you see that if a script writes an invalid tag in the HTML head, it will cause the browser to close it early as before. We have exactly the same problem!

We didn’t see the problem with our Python parser because lxml (the Python parsing library) doesn’t try to fix HTML errors.

Why do browsers do this?

Browsers need to render pages that our Python script doesn’t need to do. If they try to render before correcting mistakes, the pages would look completely broken.

The web is full of pages that would completely break if web browsers didn’t accommodate for errors.

This article from HTML5Rocks provides a fascinating look inside web browsers and helps explain the behavior we see in our examples.

“The HTML5 specification does define some of these requirements. (WebKit summarizes this nicely in the comment at the beginning of the HTML parser class.)

Unfortunately, we have to handle many HTML documents that are not well-formed, so the parser has to be tolerant about errors.

We have to take care of at least the following error conditions:

The element being added is explicitly forbidden inside some outer tag. In this case, we should close all tags up to the one which forbids the element, and add it afterward.

Please read the full article or at least make sure to read at least the section on “Browser’s Error Tolerance” to get a better context.

How to fix this

Fortunately, fixing this problem is actually very simple. We have two alternatives. A lazy one and a proper one.

The proper fix is to track down scripts that insert invalid HTML tags in the head and move them to the HTML body.

The lazy and quickest fix is to move all SEO tags (and other important tags) before any third party scripts. Preferably, right after the opening <HEAD> tag.

You can see how I do it here.

We still have the same invalid tag and script in the HTML head and the SEO tags are also in the head.

Is this a common problem?

I’ve been seeing this issue happening for many years now, and Patrick Stox has also reported seeing the same problem happening often to enterprise sites.

One of the biggest misconceptions about technical SEO is that you do it once and you are done. That would be the case if the sites didn’t change, users/developers didn’t make mistakes and/or Googlebot behavior didn’t change either.

At the moment that is hardly the case.

I’ve been advocating technical SEOs learn developer skills and I hope this case study illustrates the growing importance of this.

If you enjoyed this tip, make sure to attend my SMX West session on Solving Complex JavaScript Issues And Leveraging Semantic HTML5 next month. Among other things, I will share advanced research on how Googlebot and Bingbot handle script and HTML issues like the ones I mentioned here.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Hamlet Batista is CEO and founder of RankSense, an agile SEO platform for online retailers and manufacturers. He holds U.S. patents on innovative SEO technologies, started doing SEO as a successful affiliate marketer back in 2002, and believes great SEO results should not take 6 months.

Continue Reading


9 Best AMP WordPress Plugins for Speed, Search & Tracking



John Lincoln

If you want your website to load quickly on a mobile platform, you should make use of Accelerated Mobile Pages (AMP) technology.

Fortunately, that’s easy to do if you’re using WordPress because there are quite a few AMP plugins available.

Even better: many of them are free.

You will, however, need to invest some time to configure them properly. You might even need to make some design tweaks.

Still, it’s worth the effort if Google rewards your site with a top rank in mobile search.

Quick note, before you try any of these AMP plugins for WordPress, make sure you:

  • Create a backup version of your website.
  • Ensure it works in your hosting environment.
  • Make sure it matches your goals for the site.
  • Have a developer on hand to make sure it can be configured properly.
  • Make sure it works with your theme.

Here are some of the best AMP plugins for WordPress in the market.

1. The Official AMP Plugin for WordPress

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

If you’re unsure which plugin to choose from in this list, you almost certainly can’t go wrong with the one that’s designed by the official AMP project.

Here are some of its capabilities:

  • Support for core themes: The plugin is compatible with the “core” themes (such as Twenty Twenty).
  • Compatibility tool: Sometimes, it’s not possible to automate AMP markup insertion. When that happens, the plugin will show which components are causing problems so you can address them manually.
  • CSS tree-shaking: The tool will remove unused cascading style sheets (CSS) so you can stay under the AMP-mandated 50k limit.
  • AMP Stories: While still in beta as of this writing, the plugin enables you to create, edit, and publish AMP Stories.

You can also configure the tool to follow one of three template modes:

  • Standard: Use the AMP plugin for your entire site. There’s no need to separate AMP and non-AMP content.
  • Reader: Shows pages with a simplified design that meets AMP standards. Each page has a canonical URL as well as an AMP URL.
  • Transitional: Delivers AMP and non-AMP experiences with the same look and feel.

The plugin also receives frequent updates. If you’re someone who’s into always getting “the latest and greatest,” that’s another argument in its favor.

2. AMP for WP – Accelerated Mobile Pages

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

One of the more highly rated options in this list is the AMP for WP plugin.

Yes, it even has a better average rating than the one produced by the official AMP project.

AMP for WP also offers a rich feature set that includes:

  • AdSense support.
  • Contact Form 7 support.
  • Email opt-in support.
  • Call To Action support.

In other words, if you want to transform your website into something resembling a mobile app and use it for digital marketing, this plugin might be your best bet.

The tool also plays nicely with some of the more popular WordPress SEO plugins, including Yoast SEO, All in One SEO, SEOPress, Rank Math, and The SEO Framework.

Additionally, AMP for WP also integrates with WooCommerce. If you’re selling products online, that feature alone can give you a nice competitive advantage.

The developers of the plugin offer premium support if you’d need a hand getting things up and running.

3. Schema & Structured Data for WP & AMP

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

How would you like to add markup to your pages so that they appear as rich snippets in search results?

If so, then have a look at the Schema & Structured Data for WP & AMP plugin.

Please note: this option isn’t an AMP plugin per se. However, it supports AMP. That’s the important takeaway here.

The tool also supports 33 different schema types, including blog posts, news articles, local business details, recipes, products, and videos.

If the schema type you want to use isn’t supported, the developers say that you can request it and they’ll add it for you.

The tool even supports conditional display fields, so you get to decide which posts, pages, or other content gets marked up.

You can even import markup data from other schema plugins, including SEO Pressor and WP SEO Schema.

The premium version of the tool enables you to add reviews and offers priority support.

4. PWA for WP & AMP

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

If you’d like to transform your website into a progressive web app (PWA), take a look at the PWA for WP & AMP plugin.

Use the tool to take your user experience up a notch with the latest and greatest web tech bells and whistles.

For starters, and most importantly for this article, the plugin offers full AMP support.

It also comes with UTM tracking, multi-site support, and a cache expiration option.

PWA for WP & AMP also offers a service worker feature that optimizes your website. That translates to faster load times for visitors.

And maybe best of all: the tool includes offline support. Visitors can load the site in a jiffy even when they aren’t connected to WiFi.

The premium version of PWA for WP & AMP gives you access to a loading icon library, calls to action, and data analytics.

5. Glue for Yoast SEO & AMP

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

If you have the Yoast SEO plugin and the Official AMP Plugin for WordPress then you’re going to need the Glue for Yoast SEO & AMP plugin as well.

Why? Because it’s the “glue” that enables the two plugins to work together.

It also gives you an easy way to customize your AMP content.

Additionally, Glue adds rudimental styling so you can retain at least a little bit of the site’s look and feel for AMP visitors.

It’s developed by the same developer who created the Yoast SEO plugin.

6. AMP WP – Google AMP for WordPress

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

This one appears to be a fairly new entry in the library of AMP plugins. As of now, it’s only got a thousand active installations.

Still, the AMP WP plugin demonstrates promise. According to the product description, you can get as much as a 5x faster load with the tool.

Also, the developer claims that it’s 100% compatible with Yoast SEO.

Here are a few other features you get with the plugin:

  • GDPR support (for Europe).
  • Slider support.
  • Related posts view.
  • Support for third-party analytics (like Google Analytics).
  • Ability configure AMP for specific taxonomies (tags and categories).
  • Optional sticky header.
  • Image lightboxes.

7. weeblrAMP CE

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

The weeblrAMP plugin goes beyond what other plugins offer by empowering you to create a nearly complete AMP version of your entire site.

Additionally, weeblrAMP integrates with numerous other tools, including:

  • Disqus
  • Google Analytics
  • Google Tag Manager
  • Facebook pixel
  • Yoast SEO
  • Jetpack
  • WooCommerce
  • Contact Form 7
  • Gravity Forms

Also, the plugin also enables you to fully customize the AMP experience with template overrides.

As of now, weeblrAMP only has 700 active installs. However, it’s got a 4.5-star rating.

8. AMP It Up!

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

AMP It Up! bills itself as another “set it and forget it” plugin. All you have to do is install and activate it and you’ve magically got an AMP-compliant website.

Truth be told, though, nothing is that simple. Especially in web development.

You’re going to have to do some manual configuration.

The tool offers support for:

  • GDPR
  • Google Analytics
  • YouTube
  • BrightCove

You can even add your own custom JavaScript to your AMP-enabled website. That gives you some flexibility with the functionality.

Additionally, AMP It Up! automatically adds social share buttons to your pages. You won’t have to fish around for another plugin for that purpose.

Also, the tool will automatically convert content with several pictures into an AMP-compliant carousel.

AMP It Up! only has a few hundred installs at this point but it’s got a 4-star rating.

9. AMP Stories for WordPress

9 Best AMP WordPress Plugins for Speed, Search &#038; Tracking

Want a way to create fast-loading full-screen content on the web?

You can use AMP Stories to get website visitors more engaged with your brand.

And yes, they’re just like Instagram Stories.

You can include images, animations, videos, audio, and text in your own AMP Stories.

If you like the idea of using AMP Stories, you should take a look at the AMP Stories for WordPress plugin. It enables you to create a Story by just filling out a form on a post.

The tool also enables you to include a URL on your Story. That can come in handy if you want to redirect people to another part of your site (for example, to place an order).

The pro version of AMP Stories for WordPress includes bookend management, support for extra content at the end of Stories, and Facebook integration.

The tool currently enjoys a 5-star rating.

Wrapping It Up

In this mobile era, you need a website that loads quickly on mobile devices.

Fortunately, that’s easy to do with AMP.

What isn’t so easy is to manually implement AMP technology on your website.

However, if you’re using WordPress, there are plenty of plugins that will do it for you.

If you haven’t already added AMP support to your WordPress site, why not grab one of the plugins listed here and get the ball rolling today?

More Resources:

Image Credits

All screenshots taken by author, December 2019

Continue Reading


Copyright © 2019 Plolu.