Connect with us


SMX Advanced Overtime: Your questions answered about webspam and penalties



Frédéric Dubut and Fili Wiese speaking at SMX Advanced in Seattle in June. This session was so popular they will be teaming up again to talk about the latest news with Bing and Google penalties and algorithms at SMX East in New York on Nov. 13.

Frédéric Dubut (lead of the spam team at Bing) and I spoke together in a first-ever Bing and ex-Google joint presentation at SMX Advanced about how Google and Bing go about webspam, penalties and algorithms. We did not have the time to address every question from the attendees during the Q&A and so we wanted to follow up here. Below are questions submitted during our session about Google and Bing penalties along with our responses.

Q: Did the disavow tool work for algo penalties or was it mostly for manual action?
A: The disavow tools from Bing and Google most definitely help with manual spam actions. In fact, it is crucial to resolve link related manual spam actions/penalties with the disavow tool. At the same time, if your website has a history of active link building, the disavow tools are also a great way of getting rid of those low-quality links that you can’t remove and are now in violation of the Google or Bing Webmaster Guidelines. While there is no such thing as algorithmic penalties from Google’s side, disavow link data will be used by both Bing and Google as a potential data point for testing the various algorithms that power rankings.

Q: Thoughts or tips on combating spam user’s posts on UGC sections of a site? (reviews, forums, etc.)

A: Vigilance is key while combating user-generated spam and monitoring communities for brand protection purposes. There are some quick and easy ways of mass reviewing or limiting abuse. For example, using CSRF tokens or batch review user submissions by loading the last 100 posts onto one page and skim over them to find the abusive ones, then move on to the next 100, etc. You can also decide to always review any posts with a link before publishing, or you can use commercial tools like akismet or reCaptcha to limit spammer activity. If you don’t think you can commit any resources at all to moderating your UGC sections, you may also consider not allowing the posting of any links. It is important to remember that no tool will stop human ingenuity, which is why committing resources, including trained outreach for employees, is a must if the risk associated with user-generated spam is to be reduced.

Q: How can you tell if someone buys links?

A: It is all about intent and trends. In general, it doesn’t take a thorough manual review of every single link to detect something suspicious. Most often, one quick look at the backlink data is enough to raise suspicions and then reviewing the backlink profile in detail delivers the smoking gun.

Q: With known issues regarding javascript indexing, how are you dealing with cloaking since the fundamentals around most SSR and dynamic solutions seem to mirror cloaking? Is it hard to tell malicious versus others?

A: Actually, if we focus on the intent, why a certain solution is put in place, it is rather easy. In a nutshell, if something is being done so that search engines can be deceived and substantially different content is displayed to bots versus users, that is cloaking, which is a serious violation of both Google and Bing Webmaster Guidelines. However if you want to avoid risking being misunderstood by search engine algorithms and at the same time provide a better user experience with your Javascript-rich website, make sure that your website follows the principles of progressive enhancement.

Q: Can a site be verified in GSC or BWT while a manual penalty is applied?

A: Definitely. In the case of Bing Webmaster Tools, if you want to file a reconsideration request and don’t have an account yet, we highly recommend creating one in order to facilitate the reconsideration process. In the case of Google Search Console, you can log in with your Google account, verify your site as a domain property and see if any manual actions are applied anywhere on your domain.

Q: Is there a way that I can “help” Google find a link spammer? We have received thousands of toxic backlinks with the anchor text “The Globe.” If you visit the site to look for contact info they ask for $200K to remove the backlinks so we spend a lot of time disavowing.

A: Yes, absolutely. Google Webmaster Guidelines violations, including link spamming, can be reported to Google through a dedicated channel: the webspam report. On top, there are Google Webmaster Help forums, which are also monitored by Google Search employees and where bringing such issues to their attention stands an additional chance to trigger an investigation.

To report any concern to Bing, including violations to Bing Webmaster Guidelines, you can use this form.

Q: Does opening a link in a new tab (using target=_blank) cause any issues / penalties / poor quality signals? Is it safe to use this attribute from an SEO perspective or should all links open in the current tab?

A: Opening a link in a new tab has zero impact on SEO. However think about the experience you want to give to your users when you make such decisions, as links opening in new tabs can be perceived as annoying at times.

Q: Should we be proactively disavowing scrapper sites and other spam looking links that we find (not part of a black hat link building campaign)? Does the disavow tool do anything beyond submitting leads to the spam team? Or are those links immediately discredited from your backlink profile once that file is updated?

A: Definitely, if this is a significant part of your website backlink profile. Spam links need to be dealt with in order to mitigate the risk of a manual penalty, algorithms being triggered or even undesirable Google or Bing Search team attention. The disavow tool primarily serves the purpose of being a backlink risk management tool for you and enabling you to distance your website from shady backlinks. However, a submitted disavow file is merely a suggestion for both Google and Bing and not a very reliable lead for active spam fighting. Whether search engines abide by the submitted disavow file or use it in part or not at all is up to each search engine.

Q: How is a cloaking penalty treated? At the page level, sitewide. Can it be algo treated? Or purely manual?

A: Cloaking is a major offense to both Google and Bing, given its utterly unambiguous intent, which is a deception of the search engine and the user. Both engines are targeting cloaking in several complementary ways – algorithmically, with manual penalties, as well as other means of action. The consequence of deceptive user-agent cloaking is typically complete removal from the index. Google and Bing will be trying to be granular in their approach, however, if a website’s root is cloaking or the deception is too egregious, the action will be taken at the domain level.

Q: If you receive a manual penalty on pages on a subdomain, is it possible that it would affect the overall domain? If so, what impact could be expected?

A: It is possible indeed. The exact impact depends on the penalty applied and how it impairs a website’s overall SEO signals once it has manifested itself. This is something that needs to be investigated on an individual site level. If you end up in a situation where you have a penalty applied to your website, your rankings will be impaired and your site’s growth limited. The best course of action is to apply for reconsideration with the search engine in question.

Q: Do Bing and Google penalize based on inventory out of stock pages? For example, I have thousands of soft 404s on pages like these. How do you suggest to best deal with products that go out of stock on large e-commerce sites?

A: No, neither Google nor Bing penalizes sites with large volumes of 404 Not Found pages. Ultimately, when you have any doubt about the legitimacy of a specific technique, just ask yourself if you’d be comfortable sharing it with a Google or Bing employee. If the answer is no, then it is probably something to steer clear of.

The problem here is that with a lot of soft 404s, search engines may trust your server and/or content signals significantly less. As a result, this has the potential to have a major impact on your search visibility. One of the best ways to deal with out of stock items is to be using smart 404’s, which offer users a way to still find suitable available alternatives to the item currently unavailable while at the same time serving a 404 HTTP status code or noindex to users and bots alike. Talk to an SEO professional to discuss what the best strategy is for your website because there are a number of additional factors (e.g. the size of the website, available products and duration of unavailability) which can have a big impact on picking the right SEO strategy.

Have more questions?

Do you have more questions for us? You are in luck because at SMX East this year we will present the latest about Bing and Google penalties and algorithms. Be sure to join us at SMX East!

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Fili is a renowned technical SEO expert, ex-Google engineer and was a senior technical lead in the Google Search Quality team. At SearchBrothers he offers SEO consulting services with SEO audits, SEO workshops and successfully recovers websites from Google penalties. Fili is also a frequent speaker at SMX and other online marketing events.

Continue Reading
Click to comment

You must be logged in to post a comment Login

Leave a Reply


Google desktop favicon search results study



In 2019, Google introduced a new format in mobile search results which included a small “favicon” icon from the website, to the left of the snippet. This week, Google announced that a similar format was being launched in desktop results.

Yard carried out a user study on mobile results in September that concluded that some users think that favicon results are ads. When the desktop results launched we carried out a similar study to identify the effect of the changes.


We asked a set of 250 users to look at 12 images of desktop search results and answer the question “Does this image contain any adverts?” with a “yes” or “no.” These results were compared to results of the old format, without favicons, for the same search queries.

Search query: Car insurance

There were 4 paid ads at the top of the search results. In the old format, 73% of users identified that there was advertising on the page. In the new “favicon” format, this changes slightly to 71%.

Search query: Online programming courses

“Online programming courses” was the query used as an example by Google so it seemed right to test it thoroughly. We looked at three versions – with a single ad at the top, with no ads and with a single ad at the bottom.

Single ad at the top

Users identified that there was advertising on the page 67% of the time with the new favicon format, an increase from 63% on the old format.

Single ad at the bottom

There was no change for the result where the ad was at the bottom of the page. This suggests that the new “Ad” marker might be clearer to users, given it’s more visible at the top of the page than at the bottom.

No ads

The most significant change found was when there were no ads at all. The number of users who identified that there were no ads dropped from 65% to 57% with the new format.

Search query: home insurance

The pattern of users seeing ads that weren’t there was found consistently throughout the study. The percentage of users correctly identifying that there were no ads here dropped from 62% to 55% for “home insurance.”

Search query: Flights to New York

The biggest drop we noted was for “flights to new york” where only 58% correctly identified that there were no ads on the old format, this plummeted to 42% with the new format.


The effect that we noted on mobile, where recognition of advertising remains the same or even increases a little with the introduction of favicons is also seen on desktop. It is possible that the number of people recognizing advertising from the correct signals (e.g. “ad” markers) drops as it is clear that some users think the favicons indicate ads.

It seems likely also that some users may have missed the black ad markers due to their similar appearance to favicons but this is difficult to confirm using this methodology.

The number of people who identify ads on Google search results was alarmingly low with the old format. The new format doesn’t seem to have changed that percentage significantly but has made identification of the actual ads less accurate. In other words, people think more things are ads when they are not and may identify real ads less often.

With this change on mobile and desktop, Google has reached the limits of hiding “ad” markers and has chosen instead to change the paradigm, making natural results appear more like regular results.

The logical conclusion of this approach is that the idea that search results are paid for is normalized, gradually increasing the amount of advertising space which is considered acceptable to users.

That assumes, of course, that we shouldn’t accept Hanlon’s razor to never attribute to malice that which can be adequately explained by stupidity. In this case, Google is smart enough to know exactly what it is doing and has tested these changes thoroughly before rolling them out.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Richard Falconer is an experienced SEO and managing director of Yard, a UK based global agency that produces brand performance through data.

Continue Reading


OneSearch, Verizon Media’s new search engine, sounds awfully familiar



Verizon Media has launched OneSearch, which appears to be a direct DuckDuckGo competitor, capitalizing on heightened awareness around privacy and tracking issues. The new search engine will serve results powered by Bing and run contextual ads rather than behaviorally targeted ads that rely on cookies.

Microsoft and Verizon Media. OneSearch’s organic search results are provided by Microsoft’s Bing search engine and its contextual ads will be served by Microsoft Advertising (formerly Bing Ads) under the Verizon Media and Microsoft Advertising partnership. However, at the time of publishing, ads do not appear to be showing on search results pages.

Privacy features. OneSearch does not employ cookies to track online behavior for targeting or retargeting users with ads.

The search engine encrypts search terms entered by the user and builds the search URL using those encrypted search terms. The encryption key expires after one hour, which may stop third parties from accessing search history via browser history after the one-hour period.

“Advanced Privacy Mode,” which is enabled by default, must be turned on to encrypt search terms and expire search history links.

Trending Now. When a user clicks on the search box, a list of “Trending Now” searches appears (before the user begins inputting their search terms).

“Trending Now uses search logs to determine if there are entities (people, places or things) for which searches have been peaking recently,” a Verizon Media spokesperson told Search Engine Land. “We compare the number of searches for that entity within a short, recent, period of time with the average baseline for that entity over an extended period of time. A pool of those that are showing the most dramatic uplift are selected, and we rotate through that pool.”
Why we care. OneSearch offers another search option for privacy-oriented users. DuckDuckGo, which also serves contextual search ads that do not rely on cookies, has gained in popularity over the years and will appear on Google’s search choice screen for Android users in the EU, which may increase its share of the mobile search market. OneSearch will have to differentiate itself if it seeks to overtake similar competitors such as DuckDuckGo, let alone more well-known search engines.

The way that OneSearch leverages trending searches also lends itself to the content that other Verizon-owned companies, such as Yahoo, produce.

“We crawl the web for content, which includes Verizon Media websites, and pull the most recent, relevant content from that pool,” a Verizon Media spokesperson told Search Engine Land. “If you tend to see Yahoo content, that just means Yahoo has a good depth of recent, relevant articles that pertain to the content.”

About The Author

George Nguyen is an Associate Editor at Third Door Media. His background is in content marketing, journalism, and storytelling.

Continue Reading


Case study: The true value of informational content for e-commerce SEO



The true value of informational content for e-commerce SEO is often difficult to prove. Some content marketers and SEOs are convinced that informational content can serve as a link-worthy asset that will attract natural links from other websites. The acquired links are then supposed to help improve the SEO performance of the entire online shop.

But how do we isolate factors to demonstrate that it is indeed the informational content that contributes to a website’s overall SEO performance, and not other optimizations and developments? This article presents a case that supports the theory of informational content helping commercial pages rank better and generate additional sales.

Are you in a hurry? Jump straight to the TL;DR summary of this article.

Background: SEO strategy and performance

The e-commerce site we are looking at in this case study was first launched in 2011, but the current SEO strategy was only implemented in 2017. In a nutshell, it consists of the following activities:

Technical SEO:

  • Making the website’s content (60.000 product pages and 80 category pages) crawlable and indexable for search engines
  • Heavy focus on improving page speed for users and search engine crawlers


  • Reorganizing the existing informational content pages and adding new ones
  • Adding contextual internal links from informational content to category pages

Active link building has never played a role in this website’s SEO efforts. All links that are currently pointing to the domain were either by-products of other marketing activities and partnerships or were generated naturally by people voluntarily linking to the website’s content.

This approach has yielded great results for the business over the past two and a half years. Since the implementation of the new SEO strategy, the domain’s visibility in Google’s SERPs, as calculated by Sistrix, has grown significantly. The growth was only interrupted by a major setback caused by the so-called Medic Update in the summer of 2018, but it recovered during the March 2019 Core Update:

The growth in visibility has translated into an increase in sales generated via organic search from EUR 359k in 2016, the year before the SEO strategy was implemented, to EUR 914k in 2019:

The revenue numbers for organic search traffic were tracked with Google Analytics and calculated with the help of a custom attribution model that does not only take into account the last interaction before the sale, but also factors in all previously tracked interactions of a user with the website.

Using the same attribution method, we can also show the contributions of the different page types as landing pages to the overall revenue generated through organic search. In 2019, the informational content pages only attracted traffic worth 2,36% of the overall SEO revenue (screenshot from the Google Analytics Model Comparison Tool):

One area in which the informational content pages have shown a very good performance is the natural acquisition of backlinks. Out of the ten pages on the website that have the highest number of links from other domains pointing to them, five are informational content pages, four are category pages and one is the home page.

Looking at the available information, it is hard to tell which role the informational content pages really play for the great overall SEO performance of the shop. Their direct contribution to sales is small, but they do have a decent share of the website’s backlinks pointing to them. Do these backlinks really help the commercial shop pages rank better and generate more sales through organic search traffic?

Removing all informational content pages

In late 2019, a drastic business decision brought about some major changes for the company. Despite the great performance over the past few years, the shop is just a relatively small player within a bigger organization. The company belongs to a bigger corporation, which also owns a big direct competitor of the shop. In order to cut costs, it was decided to move the entire handling of the online shop over to the big direct competitor.

The new owners are planning to replace the entire shop with their own systems, and they only want to keep the domain name and the logo of the current shop. They have made it very clear that they are not planning to use any of the informational content that has been built up over the past years. Because of this, the company that is currently still in control of the shop decided to remove the content and save it for other projects that they might work on in the future.

Roughly 25 informational content pages were removed and the URLs were redirected to the shop’s home page. The redirects were not implemented in the hope that rankings would be transferred to the home page. In a hopeless situation, setting up redirects instead of 404 or 410 status codes was just a spontaneous decision without any specific motivation. No other changes were made to the shop in the weeks before and after the informational content was removed.

Impact on the website’s rankings

For the first few days after the informational content pages were removed, Google seemed to be quite forgiving. Some of the URLs of the removed pages kept ranking although they were redirecting all traffic to the shop’s home page. This might also be due to the fact that the pages were not crawled immediately, so it took Google a while to detect and process all changes.

Ten days after the content was removed, all rankings for the directory that previously contained the informational content pages were completely gone:

Please note that the daily visibility curve in the above screenshot might not represent the drop with complete accuracy. Although Sistrix calculates the visibility daily, it seems that they do not scrape the SERPs for every single one of the millions of keywords in their database every day, so ranking changes might only have an impact on the daily visibility graph a few days after they occur.

Interestingly, the overall visibility of the domain also took a serious blow, with losses far bigger than just the visibility of the removed directory. Within three weeks after the removal of the informational content pages, the shop had lost almost one third of its overall visibility, although the removed content previously only made up roughly 1% of the domain’s visibility:

The home page and several category pages lost lots of their page 1 rankings for commercial intent search queries with high search volumes. The following screenshot from the Sistrix ranking changes report shows an extract of the most important rankings that were lost after the informational content pages were removed:

Please note that the shop did not sell sunglasses and that the language was not English. The keywords and the URLs in the above screenshot were changed in order to protect the identity of the business, but the search intents were maintained and all other numbers in the screenshot (search volume, CPC, positions, etc.) are real.

Are the ranking drops and visibility losses of the home page and category pages directly linked to the removal of the informational content pages, and if so, how and why?

Possible reasons for the ranking drop

The most convincing theory for why the rankings of the home page and category pages dropped after the informational content pages were removed is related to links. As mentioned above, the informational content pages had a fair amount of links from other domains pointing to them. Within their content, there were also contextual links pointing to the home page and category pages, which were supposed to pass on the relevance of the backlinks pointing to the informational content pages to pages that could cater to more commercial search intents.

After the removal of the informational content pages, the backlinks pointing to them lost all of their relevance and the internal links pointing from the informational content pages to other pages on the domain were also completely lost.

However, there are a number of other factors that might play a role and there are also some unknowns that should be addressed.

The URLs of the informational content pages were redirected to the home page, which probably results in the backlinks pointing to the original pages no longer passing all of their relevance to the redirect targets, as their content is completely different from the originally linked pages. Google officials have confirmed that redirects to less relevant pages can be treated as so-called “soft 404s.”

The question arises whether this situation would have played out differently if the URLs of the informational content pages had given back 404 or 410 status codes instead of redirecting to the home page. Would a backlink to a 404 page or to a URL that gives back a 410 status code lose less relevance than a “soft 404” caused by a redirect to a non-matching target?

It would seem that in this particular case, it would not make a difference if the pages gave back a 404 or 410 status code instead of redirecting to the home page. The only way to save some of the relevance of the backlinks might have been to redirect each removed URL to a similar piece of content, which was not an option in this case.

It might also be tempting to connect this case to theories about the topical relevance of the entire domain or the more recent SEO buzzword “E-A-T”. One might argue that informational content has a value in itself that goes beyond backlinks and internal links passing on relevance to commercial pages. While these ideas should not be discarded entirely, they are quite vague and even more difficult to prove.

Another unknown that needs to be addressed are external factors that might have had an impact on the website’s rankings. The ranking changes that happened after the informational content pages were removed might be coincidental and not directly related to the removal of the content. It is always difficult to completely exclude the possibility of Google algorithm changes or updates being the real cause of observed ranking changes.

The next section of this case study delivers more indicators that support the belief that the ranking changes were not coincidental, but indeed related to the removal of the informational content pages.

Recovery after putting the content back

About 3 weeks after removing the informational content pages, the company that was still in control of the website decided to put the entire content back temporarily. This decision was made for the sake of SEO science, in order to test if the ranking loss was indeed related to the removal of the informational content pages. Also, they wanted to hand over the website in the best possible state, even if the new owners had not changed their mind about using the informational content on the new website.

The following screenshot of the daily Sistrix visibility shows what happened to the domain’s overall visibility in Google’s search results after the informational content pages were put back:

Three weeks after putting all informational content pages back in place, the overall visibility of the website has fully recovered and gone back to the level it had before the informational content pages were removed. The home page and category pages regained most of the top rankings for commercial intent queries that they had before the removal of the content, except for some slight changes that are within the usual range you would expect over a period of six weeks, due to normal fluctuations and seasonality.

What do we learn from this case?

The main takeaway from observing this case is that the data strongly suggests that the informational content pages do indeed help the home page and category pages rank better for queries with commercial intent.

The exact distribution of factors remains unclear, but it seems likely that links from other domains pointing to the informational content pages together with internal links pointing from the informational content pages to commercial pages play a major role in the improved SEO performance of the commercial pages.


  • An e-commerce site with 60.000 product pages, 80 category pages and 25 informational content pages removed all informational content pages and redirected the URLs to the home page.
  • Before their removal, the informational content pages only generated 2,36% of sales via organic search traffic, but a significant share of the domain’s backlinks pointed to them.
  • After the removal of the informational content pages, the shop lost about one-third of its overall visibility as the home page and category pages lost most of their top rankings for commercial intent search queries.
  • About three weeks later, the informational content was put back, in order to test if the ranking drops were indeed related to the removal of the content.
  • Another three weeks later, the domain’s visibility was fully recovered and the home page and category pages regained their good rankings for commercial intent search queries.
  • The data suggests that the informational content pages do indeed help the home page and category pages rank better for search queries with commercial intent.
  • The most likely explanation lies in the relevance of backlinks from other domains pointing to the informational content pages, which is then passed on to commercial pages via contextual internal links.

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.

About The Author

Eoghan Henn is a freelance technical marketer and the co-founder of searchVIU, a website migration SEO tool provider. He also teaches digital marketing at the University of Santiago de Compostela. Before moving to Spain and starting his current roles, he worked as a consultant for digital marketing agencies in Germany and Belgium. His areas of specialization include international and technical SEO, Google Tag Manager implementations and web analytics.

Continue Reading


Copyright © 2019 Plolu.