Connect with us

WordPress

WordPress 5.3 Will Change How it Blocks Indexing

Published

on

WordPress 5.3 Will Change How it Blocks Indexing


WordPress announced an important change to how it will block search engines from indexing websites. This change abandons the traditional Robots.txt solution in favor of the Robots Meta Tag approach. The change brings WordPress in line with the reason for blocking Google, which is to keep the blocked pages from showing in Google’s search results.

This is the Robots Meta Tag that WordPress will use:

<meta name=’robots’ content=’noindex,nofollow’ />

Blocking Google From Indexing

It has long been a standard practice to use Robots.txt to block the “indexing” of a website.

The word “indexing” meant crawling of the site by GoogleBot. By using the Robots.txt blocking feature you could stop Google from downloading the specified web page and, it was assumed,  Google would be unable to show your pages in the Search Results.

But that robots.txt directive only stopped Google from crawling the page. Google was still free to add it to its index if it was able to discover the URL.

So to block a site from appearing in the index, a publisher would block Google from “indexing” the pages. Which wasn’t consistently effective.

WordPress 5.3 Will Truly Prevent Indexing

WordPress adapted the Robots.txt approach. But that’s changing in version 5.3.

When a publisher currently selects “discourage search engines from indexing this site” what that does is add an entry to the site’s robots.txt that prohibits Google from crawling a site.

Starting with WordPress 5.3, WordPress will adopt the more reliable Robots Meta Tag approach for preventing the indexing of a website.

This change will affect the “discourage search engines from indexing this site” setting. 

This change is an improvement. WordPress publishers can be more secure in knowing that the blocked web pages will not be shown in Google’s search results.

Screenshot of WordPress 5.3 announcement, with the following text: "In WordPress 5.3 the method used to discourage indexing will change on sites enabling the option “discourage search engines from indexing this site” in the WordPress dashboard. These changes were made as part of ticket #43590. These changes are intended to better discourage search engines from listing a site rather than only preventing them from crawling the site."Screenshot of the announcement of the change coming to WordPress 5.3.

Why Did WordPress Use Robots.txt?

WordPress relied on Robots.txt for blocking the indexing of a website because that’s how everybody kept pages from showing in Google’s search results. That was the standard way of doing it.

Yet even though everybody did it that way, as has been explained, it was an unreliable approach.

The word “indexing” has two meanings:

  1. Indexing means crawling, as when Googlebot visits and downloads web pages.
  2. Indexing can also mean adding a web page to Google’s database of web pages (which is called The Index).

Blocking Google from “indexing” a web page will keep it from seeing the web page but Google could still index the web page and add it to Google’s index. Make sense?

Robots.txt Versus Robots Meta Tag

Keeping a web page out of Google’s index was not the intent of the Robots.txt solution. Doing that is the job of the Robots Meta Tag.

So it’s good to see WordPress embrace the Robots Meta Tag as the solution to blocking web pages from showing in the search engines.

WordPress 5.3 is scheduled to be released in November 2019.

Read the WordPress announcement:

Changes to Prevent Search Engines Indexing Sites

Read Google’s Authoritative Documentation

  1. Robots meta tag and X-Robots-Tag HTTP header specifications
  2. Block search indexing with ‘noindex’



Continue Reading
Click to comment

You must be logged in to post a comment Login

Leave a Reply

WordPress

Here’s how to set up the Google Site Kit WordPress plugin

Published

on

Here's how to set up the Google Site Kit WordPress plugin


On Oct. 31, Google announced the launch of its Site Kit WordPress plugin that, “enables you to set up and configure key Google services, get insights on how people find and use your site, learn how to improve, and easily monetize your content.”

This plugin allows you to easily connect the following Google Services in a dashboard format within your WordPress backend:

  • Search Console
  • Analytics
  • PageSpeed Insights
  • AdSense
  • Optimize
  • Tag Manager

It brings the convenience of accessing your site’s performance data while logged into the backend of the site. This is great for webmasters, developers and agencies who are often an admin for their own site or a client’s WordPress site. However, it does not offer the robust and dynamic capabilities of a Google Data Studio report or dashboard to sort data so it may not be ideal for a digital marketing manager or CMO.

With that said, it wouldn’t hurt to implement this plugin as it’s actually a nifty tool that can help you stay on top of your site’s performance metrics. It’s also another way to give Google more access to your site which can have some in-direct benefits organically. 

Here is what the Google Site Kit plugin looks like within the WordPress plugin directory.

Installing and setting up Google Site Kit

To utilize the plugin, simply click install and activate as you would any other WordPress plugin. You will then be prompted to complete the set up.

Step 1

Click on the “Start Setup” button.

Step 2

You will be prompted to give access to your site’s Google Search Console profile, which means you need to sign in to the Gmail account that has access to your site’s Search Console profile.

Step 3

Once logged in you need to grant permissions for Google to access the data in your Search Console profile.

Step 4

Once you’ve granted all the respective permissions, you will get a completion notification and can then click on “Go to my Dashboard.”

Step 5

Once you’re in the Dashboard you will see options to connect other services such as Analytics, AdSense and PageSpeed insights. You can now choose to connect these services if you like. If you go to the settings of the plugin you will see additional connection options for Optimize and Tag Manager.

Here is what the dashboard looks like with Search Console, analytics and PageSpeed Insights enabled. You can see a clear breakdown of the respective metrics.

The plugin allows you to dive into each reporting respectively with navigation options on the left to drill down into Search Console and analytics.

There is also an admin bar feature to see individual page stats.

In summary, this is a great plugin by Google but keep in mind it’s just version 1.0. I’m excited to see what features and integrations the later versions will have!


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


About The Author

Tony Edward is a director of SEO at Tinuiti and an adjunct instructor of search marketing at NYU. Tony has been in the online marketing industry for over 10 years. His background stems from affiliate marketing and he has experience in paid search, social media and video marketing. Tony is also the founder of the Thinking Crypto YouTube channel.

Continue Reading

WordPress

Bing Announces Link Penalties – Search Engine Journal

Published

on

Roger Montti


Bing announced a new link penalties. These link penalties are focused on taking down private blog networks (PBNs), subdomain leasing and manipulative cross-site linking.

Inorganic Site Structure

An inorganic site structure is a linking pattern that uses internal site-level link signals (with subdomains) or cross-site linking patterns (with external domains) in order to manipulate search engine rankings.

While these spam techniques already existed, Bing introduced the concept of calling them “inorganic site structure” in order to describe them.

Bing noted that sites legitimately create subdomains to keep different parts of the site separate, such as support.example.com. These are treated as belonging to the main domain, passing site-level signals to the subdomains.

Bing also said sites like WordPress create standalone sites under subdomains, in which case no site level signals are passed to the subdomains.

Examples of Inorganic Site Structure

An inorganic site structure is when a company leases a subdomain in order to take advantage of site-level signals to rank better. There have been

Private blog networks were also included as inorganic site structure

Domain Boundaries

Bing also introduced the idea of domain boundaries. The idea is that there are boundaries to a domain. Sometimes, as in the case of legitimate subdomains (ex. support.example.com), those boundaries extend out to the subdomain. In other cases like WordPress.com subdomains the boundaries do not extend to the subdomains.

Private Blog Networks (PBNs)
Bing called out PBNs as a form of spam that abuse website boundaries.

“While not all link networks misrepresent website boundaries, there are many cases where a single website is artificially split across many different domains, all cross-linking to one another, for the obvious purpose of rank boosting. This is particularly true of PBNs (private blog networks).”

Subdomain Leasing Penalties

Bing explained why they consider subdomain leasing a spammy activity:

“…we heard concerns from the SEO community around the growing practice of hosting third-party content or letting a third party operate a designated subdomain or subfolder, generally in exchange for compensation.

…the practice equates to buying ranking signals, which is not much different from buying links.”

At the time of this article, I still see a news site subdomain ranking in Bing (and Google). This page belongs to another company. All the links are redirected affiliate type links with parameters meant for tracking the referrals.

According to Archive.org the subdomain page was credited to an anonymous news staffer. Sometime in the summer the author was switched to someone with a name who is labeled as an expert, although the content is still the same.

So if Bing is already handing out penalties that means Bing (and Google who also ranks this page) still have some catching up to do.

Cross-Site Linking

Bing mentioned sites that are essentially one site that are broken up into multiple interlinking sites. Curiously Bing said that these kinds of sites already in violation of other link spam rules but that additional penalties will apply.

Here’s the kind of link structure that Bing used as an example:

illustration of a spammy link networkAll these sites are interlinking to each other. All the sites have related content and according to Bing are essentially the same site. This kind of linking practice goes back many years. They are traditionally known as interlinked websites. They are generally topically related to each other.

Bing used the above example to illustrate interlinked sites that are really just one site.

That link structure resembles the structure of interlinked websites that belong to the same company. If you’re planning a new web venture, it’s generally a good idea to create a site that’s comprehensive than to create a multitude of sites that are focused on just a small part of the niche.

Curiously, in reference to the above illustration, Bing said that kind of link structure was already in violation of link guidelines and that more penalties would be piled on top of those:

“Fig. 3 – All these domains are effectively the same website.
This kind of behavior is already in violation of our link policy.

Going forward, it will be also in violation of our “inorganic site structure” policy and may receive additional penalties.

Takeaway

It’s good news to hear Bing is improving. Competition between search engines encourage innovation and as Bing improves perhaps search traffic may become more diversified as more people switch to Bing as well as other engines like DuckDuckGo.

Read Bing’s announcement: Some Thoughts on Website Boundaries



Continue Reading

WordPress

Google Releases its Site Kit WordPress Plugin Out of Beta

Published

on

Matt Southern


Google has released version 1.0 of its Site Kit plugin for WordPress, which means its officially out of beta after 6 months.

In the time since the developer preview of Site Kit was released, Google says it drastically simplified the setup, fixed bugs, and polished the main user flows.

Site Kit allows WordPress users to access data from Google products right from their site’s dashboard. The plugin aggregates data from Google Search Console, Google Analytics, PageSpeed Insights, and AdSense.

Google Releases its Site Kit WordPress Plugin Out of Beta

With Site Kit there’s no additional code editing required, which makes it easy to set up products like Google Analytics for those without any developer experience.

Anyone can install Site Kit, but Google emphasizes that it’s especially useful for professionals who work on sites for clients. The reasons why include:

  • Clients and other teams can easily access data from Google products by logging into the WordPress dashboard.
  • Clients will see performance states and improvement recommendations directly from Google
  • Site Kit allows you to set roles and permissions and make sure only relevant people can see the data.

To get the most out of Site Kit, Google recommends reviewing the main dashboard on at least a weekly basis. You can also check the stats of individual pages by navigating to the page and clicking on Site Kit in the admin bar.

Google Releases its Site Kit WordPress Plugin Out of Beta

With this data, Google recommends comparing the top performing pages and seeing how people found them. This can help you discover trends, such as which topics get the most engagement on Twitter, which get the most engagement on Facebook, and so on.

To get started with Site Kit, simply install it from your WordPress dashboard.



Continue Reading

Trending

Copyright © 2019 Plolu.