fbpx
Connect with us

SEO

Leverage Python and Google Cloud to extract meaningful SEO insights from server log data

Published

on


For my first post on Search Engine Land, I’ll start by quoting Ian Lurie:

Log file analysis is a lost art. But it can save your SEO butt!

Wise words.

However, getting the data we need out of server log files is usually laborious:

  • Gigantic log files require robust data ingestion pipelines, a reliable cloud storage infrastructure, and a solid querying system
  • Meticulous data modeling is also needed in order to convert cryptic, raw logs data into legible bits, suitable for exploratory data analysis and visualization

In the first post of this two-part series, I will show you how to easily scale your analyses to larger datasets, and extract meaningful SEO insights from your server logs.

All of that with just a pinch of Python and a hint of Google Cloud!

Here’s our detailed plan of action:

#1 – I’ll start by giving you a bit of context:

  • What are log files and why they matter for SEO
  • How to get hold of them
  • Why Python alone doesn’t always cut it when it comes to server log analysis

#2 – We’ll then set things up:

  • Create a Google Cloud Platform account
  • Create a Google Cloud Storage bucket to store our log files
  • Use the Command-Line to convert our files to a compliant format for querying
  • Transfer our files to Google Cloud Storage, manually and programmatically

#3 – Lastly, we’ll get into the nitty-gritty of Pythoning – we will:

  • Query our log files with Bigquery, inside Colab!
  • Build a data model that makes our raw logs more legible 
  • Create categorical columns that will enhance our analyses further down the line
  • Filter and export our results to .csv

In part two of this series (available later this year), we’ll discuss more advanced data modeling techniques in Python to assess:

  • Bot crawl volume
  • Crawl budget waste
  • Duplicate URL crawling

I’ll also show you how to aggregate and join log data to Search Console data, and create interactive visualizations with Plotly Dash!

Excited? Let’s get cracking!

System requirements

We will use Google Colab in this article. No specific requirements or backward compatibility issues here, as Google Colab sits in the cloud.

Downloadable files

  • The Colab notebook can be accessed here 
  • The log files can be downloaded on Github – 4 sample files of 20 MB each, spanning 4 days (1 day per file)

Be assured that the notebook has been tested with several million rows at lightning speed and without any hurdles!

Preamble: What are log files?

While I don’t want to babble too much about what log files are, why they can be invaluable for SEO, etc. (heck, there are many great articles on the topic already!), here’s a bit of context.

A server log file records every request made to your web server for content.

Every. Single. One.

In their rawest forms, logs are indecipherable, e.g. here are a few raw lines from an Apache webserver:

Daunting, isn’t it?

Raw logs must be “cleansed” in order to be analyzed; that’s where data modeling kicks in. But more on that later.

Whereas the structure of a log file mainly depends on the server (Apache, Nginx, IIS etc…), it has evergreen attributes:

  • Server IP
  • Date/Time (also called timestamp)
  • Method (GET or POST)
  • URI
  • HTTP status code
  • User-agent

Additional attributes can usually be included, such as:

  • Referrer: the URL that ‘linked’ the user to your site
  • Redirected URL, when a redirect occurs
  • Size of the file sent (in bytes)
  • Time taken: the time it takes for a request to be processed and its response to be sent

Why are log files important for SEO?

If you don’t know why they matter, read this. Time spent wisely!

Accessing your log files

If you’re not sure where to start, the best is to ask your (client’s) Web Developer/DevOps if they can grant you access to raw server logs via FTP, ideally without any filtering applied.

Here are the general guidelines to find and manage log data on the three most popular servers:

We’ll use raw Apache files in this project.

Why Pandas alone is not enough when it comes to log analysis

Pandas (an open-source data manipulation tool built with Python) is pretty ubiquitous in data science.

It’s a must to slice and dice tabular data structures, and the mammal works like a charm when the data fits in memory!

That is, a few gigabytes. But not terabytes.

Parallel computing aside (e.g. Dask, PySpark), a database is usually a better solution for big data tasks that do not fit in memory. With a database, we can work with datasets that consume terabytes of disk space. Everything can be queried (via SQL), accessed, and updated in a breeze!

In this post, we’ll query our raw log data programmatically in Python via Google BigQuery. It’s easy to use, affordable and lightning-fast – even on terabytes of data!

The Python/BigQuery combo also allows you to query files stored on Google Cloud Storage. Sweet!

If Google is a nay-nay for you and you wish to try alternatives, Amazon and Microsoft also offer cloud data warehouses. They integrate well with Python too:

Amazon:

Microsoft:

Create a GCP account and set-up Cloud Storage

Both Google Cloud Storage and BigQuery are part of Google Cloud Platform (GCP), Google’s suite of cloud computing services.

GCP is not free, but you can try it for a year with $300 credits, with access to all products. Pretty cool.

Note that once the trial expires, Google Cloud Free Tier will still give you access to most Google Cloud resources, free of charge. With 5 GB of storage per month, it’s usually enough if you want to experiment with small datasets, work on proof of concepts, etc…

Believe me, there are many. Great. Things. To. Try!

You can sign-up for a free trial here.

Once you have completed sign-up, a new project will be automatically created with a random, and rather exotic, name – e.g. mine was “learned-spider-266010“!

Create our first bucket to store our log files

In Google Cloud Storage, files are stored in “buckets”. They will contain our log files.

To create your first bucket, go to storage > browser > create bucket:

The bucket name has to be unique. I’ve aptly named mine ‘seo_server_logs’!

We then need to choose where and how to store our log data:

  • #1 Location type – ‘Region’ is usually good enough.
  • #2 Location – As I’m based in the UK, I’ve selected ‘Europe-West2’. Select your nearest location
  • #3 Click on ‘continue’

Default storage class: I’ve had good results with ‘nearline‘. It is cheaper than standard, and the data is retrieved quickly enough:

Access to objects: “Uniform” is fine:

Finally, in the “advanced settings” block, select:

  • #1 – Google-managed key
  • #2 – No retention policy
  • #3 – No need to add a label for now

When you’re done, click “‘create.”

You’ve created your first bucket! Time to upload our log data.

Adding log files to your Cloud Storage bucket

You can upload as many files as you wish, whenever you want to!

The simplest way is to drag and drop your files to Cloud Storage’s Web UI, as shown below:

Yet, if you really wanted to get serious about log analysis, I’d strongly suggest automating the data ingestion process!

Here are a few things you can try:

  • Cron jobs can be set up between FTP servers and Cloud Storage infrastructures: 
  • FTP managers like Cyberduck also offer automatic transfers to storage systems, too
  • More data ingestion tips here (AppEngine, JSON API etc.)

A quick note on file formats

The sample files uploaded in Github have already been converted to .csv for you.

Bear in mind that you may have to convert your own log files to a compliant file format for SQL querying. Bigquery accepts .csv or .parquet. 

Files can easily be bulk-converted to another format via the command line. You can access the command line as follows on Windows:

  • Open the Windows Start menu
  • Type “command” in the search bar
  • Select “Command Prompt” from the search results
  • I’ve not tried this on a Mac, but I believe the CLI is located in the Utilities folder

Once opened, navigate to the folder containing the files you want to convert via this command:

CD 'path/to/folder’

Simply replace path/to/folder with your path.

Then, type the command below to convert e.g. .log files to .csv:

for file in *.log; do mv "$file" "$(basename "$file" .*0).csv"; done

Note that you may need to enable Windows Subsystem for Linux to use this Bash command.

Now that our log files are in, and in the right format, it’s time to start Pythoning!

Unleash the Python

Do I still need to present Python?!

According to Stack Overflow, Python is now the fastest-growing major programming language. It’s also getting incredibly popular in the SEO sphere, thanks to Python preachers like Hamlet or JR.

You can run Python on your local computer via Jupyter notebook or an IDE, or even in the cloud via Google Colab. We’ll use Google Colab in this article.

Remember, the notebook is here, and the code snippets are pasted below, along with explanations.

Import libraries + GCP authentication

We’ll start by running the cell below:

It imports the Python libraries we need and redirects you to an authentication screen.

There you’ll have to choose the Google account linked to your GCP project.

Connect to Google Cloud Storage (GCS) and BigQuery

There’s quite a bit of info to add in order to connect our Python notebook to GCS & BigQuery. Besides, filling in that info manually can be tedious!

Fortunately, Google Colab’s forms make it easy to parameterize our code and save time.

The forms in this notebook have been pre-populated for you. No need to do anything, although I do suggest you amend the code to suit your needs.

Here’s how to create your own form: Go to Insert > add form field > then fill in the details below:

When you change an element in the form, its corresponding values will magically change in the code!

Fill in ‘project ID’ and ‘bucket location’

In our first form, you’ll need to add two variables:

  • Your GCP PROJECT_ID (mine is ‘learned-spider-266010′)
  • Your bucket location:
    • To find it, in GCP go to storage > browser > check location in table
    • Mine is ‘europe-west2′

Here’s the code snippet for that form:

Fill in ‘bucket name’ and ‘file/folder path’:

In the second form, we’ll need to fill in two more variables:

The bucket name:

  • To find it, in GCP go to: storage > browser > then check its ‘name’ in the table
  • I’ve aptly called it ‘apache_seo_logs’!

The file path:

  • You can use a wildcard to query several files – Very nice!
  • E.g. with the wildcarded path ‘Loggy*’, Bigquery would query these three files at once:
    • Loggy01.csv
    • Loggy02.csv
    • Loggy03.csv
  • Bigquery also creates a temporary table for that matter (more on that below)

Here’s the code for the form:

Connect Python to Google Cloud Storage and BigQuery

In the third form, you need to give a name to your BigQuery table – I’ve called mine ‘log_sample’. Note that this temporary table won’t be created in your Bigquery account.

Okay, so now things are getting really exciting, as we can start querying our dataset via SQL *without* leaving our notebook – How cool is that?!

As log data is still in its raw form, querying it is somehow limited. However, we can apply basic SQL filtering that will speed up Pandas operations later on.

I have created 2 SQL queries in this form:

  • “SQL_1st_Filter” to filter any text
  • “SQL_Useragent_Filter” to select your User-Agent, via a drop-down

Feel free to check the underlying code and tweak these two queries to your needs.

If your SQL trivia is a bit rusty, here’s a good refresher from Kaggle!

Code for that form:

Converting the list output to a Pandas Dataframe

The output generated by BigQuery is a two-dimensional list (also called ‘list of lists’). We’ll need to convert it to a Pandas Dataframe via this code:

Done! We now have a Dataframe that can be wrangled in Pandas!

Data cleansing time, the Pandas way!

Time to make these cryptic logs a bit more presentable by:

  • Splitting each element
  • Creating a column for each element

Split IP addresses

Split dates and times

We now need to convert the date column from string to a “Date time” object, via the Pandas to_datetime() method:

Doing so will allow us to perform time-series operations such as:

  • Slicing specific date ranges 
  • Resampling time series for different time periods (e.g. from day to month)
  • Computing rolling statistics, such as a rolling average

The Pandas/Numpy combo is really powerful when it comes to time series manipulation, check out all you can do here!

More split operations below:

Split domains

Split methods (Get, Post etc…)

Split URLs

Split HTTP Protocols

Split status codes

Split ‘time taken’

Split referral URLs

Split User Agents

Split redirected URLs (when existing)

Reorder columns

Time to check our masterpiece:

Well done! With just a few lines of code, you converted a set of cryptic logs to a structured Dataframe, ready for exploratory data analysis.

Let’s add a few more extras.

Create categorical columns

These categorical columns will come handy for data analysis or visualization tasks. We’ll create two, paving the way for your own experiments!

Create an HTTP codes class column

Create a search engine bots category column

As you can see, our new columns httpCodeClass and SEBotClass have been created:

Spotting ‘spoofed’ search engine bots

We still need to tackle one crucial step for SEO: verify that IP addresses are genuinely from Googlebots.

All credit due to the great Tyler Reardon for this bit! Tyler has created  searchtools.io, a clever tool that checks IP addresses and returns ‘fake’ Googlebot ones, based on a reverse DNS lookup.

We’ve simply integrated that script into the notebook – code snippet below:

Running the cell above will create a new column called ‘isRealGbot?:

Note that the script is still in its early days, so please consider the following caveats:

  • You may get errors when checking a huge amount of IP addresses. If so, just bypass the cell
  • Only Googlebots are checked currently

Tyler and I are working on the script to improve it, so keep an eye on Twitter for future enhancements!

Filter the Dataframe before final export

If you wish to further refine the table before exporting to .csv, here’s your chance to filter out status codes you don’t need and refine timescales.

Some common use cases:

  • You have 12 months’ worth of log data stored in the cloud, but only want to review the last 2 weeks
  • You’ve had a recent website migration and want to check all the redirects (301s, 302s, etc.) and their redirect locations
  • You want to check all 4XX response codes

Filter by date 

Refine start and end dates via this form:

Filter by status codes

Check status codes distribution before filtering:

Code:

Then filter HTTP status codes via this form:

Related code:

Export to .csv 

Our last step is to export our Dataframe to a .csv file. Give it a name via the export form:

Code for that last form:

Pat on the back if you’ve followed till here! You’ve achieved so much over the course of this article!

I cannot wait to take it to the next level in my next column, with more advanced data modeling/visualization techniques!

I’d like to thank the following people:

  • Tyler Reardon, who’s helped me to integrate his anti-spoofing tool into this notebook!
  • Paul Adams from Octamis and my dear compatriot Olivier Papon for their expert advice
  • Last but not least, Kudos to Hamlet Batista or JR Oakes – Thanks guys for being so inspirational to the SEO community!

Please reach me out on Twitter if questions, or if you need further assistance. Any feedback (including pull requests! :)) is also greatly appreciated!

Happy Pythoning!

This year’s SMX Advanced will feature a brand-new SEO for Developers track with highly-technical sessions – many in live-coding format – focused on using code libraries and architecture models to develop applications that improve SEO. SMX Advanced will be held June 8-10 in Seattle. Register today.


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


About The Author

Charly Wargnier is a seasoned digital marketing consultant based in the UK, leaning on over a decade of in-the-trenches SEO, BI and Data engineering experience. Charly has worked both in-house and agency-side, primarily for large enterprises in Retail and Fashion, and on a wide range of fronts including complex technical SEO issues, site performance, data pipelining and visualization frameworks. When he isn’t working, he enjoys coding for good and spending quality time with his family – cooking, listening to Jazz music and playing chess, in no particular order!



Continue Reading
Click to comment

You must be logged in to post a comment Login

Leave a Reply

SEO

Simple guide to creating an expert roundup post that drives website traffic

Published

on


30-second summary:

  • Roundup posts are pieces of content in which a list of selected experts give their insights on the same topic, in short descriptions that include their opinions, predictions, or reviews.
  • Creating an expert roundup post for your website or blog can take some preparation and organizing efforts, but it brings undeniable long-term benefits in terms of traffic, authoritativeness, and peer recognition.
  • In the following guide, I will take you through every step of creating an enticing expert roundup post for your website.

Publishing valuable content is a constant challenge when it comes to the formats and topics to cover. As a blogger, digital marketer, or content creator, you already know how much thought goes into offering your audience fresh, engaging content on a regular basis. Readers appreciate formats they are familiar with and can consume easily. A roundup post is an example of a successful approach to topics of interest in your industry.

Roundup posts are pieces of content in which a list of selected experts give their insights on the same topic, in short descriptions that include their opinions, predictions, or reviews.

Creating an expert roundup post for your website or blog can take some preparation and organizing efforts, but it brings undeniable long-term benefits in terms of traffic, authoritativeness, meaningful relationships, and peer recognition. By gathering a group of experts to answer the same question, you will not only generate relevant content for your website but build a strong relationship basis with experts in your industry.

Having a list of selected experts answer a well-placed question gives you a valuable piece of content that is highly shareable, so let’s see what it takes to do it right. In the following guide, I will take you through every step of creating an enticing expert roundup post for your website.

1. Brainstorm potential questions

The first step you need to take after deciding to publish an expert roundup post is to find the perfect question to ask the experts. This will be the key element of your post, and it will dictate whether it will be successful or not.

The perfect question might not be easy to find, but take enough time to find it. Brainstorm as much as you need before you decide who to invite in. All the further efforts of finding influencers and experts could be in vain if the topic you choose does not fit the roundup format, or doesn’t spark interest in your readers’ minds. So I’d recommend you find a question that resonates well with both your readers and experts.

Things to consider when brainstorming

To better understand what kind of questions are fit for a roundup post, you should picture the end result. You want to have your experts give your readers a piece of their own judgment, advice, or insight on a subject that your readers are familiar with. It won’t be a 101, a critical debate, or brain-picking for ultra-specialized information.

Your question needs to be:

  • Easy enough to give your respondents room to elaborate and get ample answers from them without the need for extra-questions.
  • General enough to give you a reasonably long list of influencers, experts, and peers. Go to niche and you might be able to talk to a handful of people about it.
  • Original enough to get your readers curious about the topic, and what experts have to say about it.

What about the topic will you be asking about? Naturally, it has to be specific to your website or niche and what you usually write about. The key is to find a subject that your audience is curious or interested in. Perhaps a trend, or a subject that usually sparks debate or behind-the-scenes type of information that regular posts don’t really get into.

You have the chance to get insights about the latest trends everyone is wondering about, or tips and tricks, good practices that expert peers have discovered through their experience and expertise.

Examples of questions:

  • What’s one piece of advice you’d give to beginner bloggers?
  • What’s one thing you would’ve done differently when starting your blog?
  • What do you think the future of blogging is?

How to get ideas

Easier said than done? If you don’t already have a topic you’ve been pondering about, compile a list of possible questions for your roundup with a little research.

Use tools like the Ahrefs Content Explorer to find trends in your industry, and subjects that seem to attract a lot of readers. You can filter results by their social shares, number, and quality of referring industries so that this tool and similar ones can give you a good idea of what subject should be pursued.

With these things in mind fuel that creative engine and start putting ideas on paper, whether they seem perfect candidates or just potential pursuers. It’s best to have a long list to start from when drafting the winning question for your roundup post.

2. Find talented experts

After finding your question, you should have a good idea about the expertise of your respondents. Assuming that you are active in your industry for a reasonable time, you should already know who the experts in your niche are. You want to compile an extensive list of experts of at least 50+ because not all will reply to your inquiry.

Let’s make a profile for the ideal respondent in your expert roundup post:

  • They are directly related to the industry you are writing about
  • They have a good follower base and an audience that regards them as influencers
  • They have contributed to roundup posts in the past
  • They are continually sharing thought-provoking, original ideas on their social media and on personal or business blogs
  • They have authority in the field: company owners and founders, top positions in companies of the industry, public speakers, success bloggers, and more

A practical, fast way of identifying possible candidates for your roundup post is to check other roundup posts in the industry. Does this approach seem lazy at a first glance? The redundancy of a roundup article doesn’t come from the list of people contributing to it, but from the very topic, you will choose for it.

As long as you are able to provoke your respondents to bring something original to the table, selecting them from other roundup articles is absolutely fine.

Depending on your topic, you might find tens of experts already showing potential for accepting your invitation. But don’t put your eggs in one basket: there are other ways of finding strong, authoritative voices in your niche.

A simple search on social media can give you a good idea of who is interested in the topic you have selected and fits the ideal profile described above. Twitter and Facebook are also great platforms where you can find experts in your industry.

For our roundup post about blogging tips for beginners, we have gathered content from CEOs and founders of content marketing websites, authors, bloggers, and podcasters in digital marketing. They were all able to give us valuable insight into what blogging is like for beginners, and what they should do to thrive.

Web search is another simple solution to putting together your expert list. We were able to find several experts by simply typing in our keyword or phrase onto Google. Find bloggers who have been covering your subject, or similar ones, and dig a bit more in their previous posts, to have an idea of who you’re going to contact.

Ahrefs, BuzzSumo, and Hootsuite are other awesome tools to research hot topics and authoritative blogs, as they display real-time data on their referring industries, traffic value, and the number of shares they get for their posts.

3. Find their contact information

Once you have a list of experts, bloggers, and influencers who can give you valuable insight into the subject you want to cover, it’s time to start gathering their contact information.

It’s best to keep a database of their information, on a simple Google or excel sheet with their names, email addresses, URL to their website, the date when you contacted them, and a column where you check if they submitted content or not. You can get a little more advanced by using a CRM or email outreach tool like Mailshake.

Keeping your contact information organized will speed up the preparation process and will help you avoid awkward situations like sending an invitation twice, or forgetting to do a follow-up with them.

Some of the experts you’re trying to reach won’t have a visible email address but you can use tools like Hunter.io to find them by simply entering their first name, last name, and their domain name. It will give you a list of the results it found. Ideally, you would launch your invitation privately, but if you still can’t find their email address, don’t hesitate to send a tweet that mentions your plan or a simple message via the other social media platforms.

Here are more tips on how to find contact information for people who you want to reach, and what good practices you should follow.

4. Reach out to your experts

If it’s the first time you are contacting someone, it’s a good idea to look into the good practices of a cold email. Roundup posts are great for getting quality backlinks, and the persons you will contact are aware of the positive influence their contribution can have on your traffic and domain. But they can’t endorse a website or a blog that doesn’t prove to be valuable on a constant basis.

We can talk about cold email outreach best practices for days on end but that would take too long. What I highly recommend is that you be genuine, polite, and kind when reaching out to your experts. This goes a long way and they’ll be able to tell when someone is being genuine since they receive hundreds of spam emails every day.

I also recommend you personalize each and every one of your emails. Yes, this will take time but you will have a higher conversion rate than if you were to send the same bulk email to everyone.

I don’t have a template to help you get started, but below I have provided a screenshot of an email I sent out to one of the experts that we included in our roundup. Feel free to use it for some inspiration and to help garner some ideas.

Another fantastic way of reaching out to experts is by joining and engaging with them via their live streams. We used this tactic to reach one of our experts who had not replied via email. It worked out, and he gave us some awesome advice while on his live stream.

5. Put it all together

Getting enough contributions from the experts you have contacted is a great achievement in the process of creating a stellar roundup post. But your job is not finished yet. Putting together the content you have just received from your guests is very important, as it will have to be a high-quality presentation that they will gladly share on their channels, therefore getting you some exposure to new audiences.

Things you want to include

  • A headshot
  • Their reply
  • Short bio
  • Social media and website handles
  • And your own comments

As you can see in our roundup post example, each contributor’s section starts with a professional picture of the contributor, the content that they have submitted, finished with our own thoughts on their commentary. We have added easy-to-follow social media icons that take you to their Twitter, Instagram, Facebook, or LinkedIn profiles, the titles they hold, and links to their main projects: companies they work for, blogs, YouTube channels, and others.

We wanted to make it easy for our readers to follow and engage with our experts via their blogs, businesses, and social media accounts.

While you want to emphasize the value of each contributor, you must also have your reader’s interest in mind, and clearly answer a need or a question your audience has.

6. Promote your post

Promoting your posts should be done in the interest of all your contributors, yourself, and your audience. The effort you have invested in compiling these pieces of content will ideally be rewarded with significant organic traffic, a good number of quality backlinks, and most importantly the start of new and meaningful relationships.

You should also edit the content carefully so that each contributor gets the same level of attention and appreciation. Take your time to thank each one of them for their contribution and don’t hesitate to personalize your message with your personal impression of their content.

Backlinks are certainly welcome, but asking them explicitly might not be appreciated by all the persons you will contact. The best ways of getting your contributors to share the roundup post are to simply ask them if they could share it with their audience.

Infographics, tweets, quotes included in an image, newsletters, and paid ads are all great ideas for getting your content everywhere and promoting it like crazy.

Conclusion: Creating an expert roundup post is totally worth it

Publishing an expert roundup post might not be everyone’s style of content, but for certain industries and domains, it can be a long-term valuable resource, both for your audience and your peers. And, of course, for you.

Keeping your focus on the value your post should bring to your readers will help you choose an enticing topic, ask the right question, and select the right people to answer it. While no one can ignore the advantages a roundup post has for contributors and creators alike, backlinks and traffic should not be your singular concern.

Ultimately, the success of such a compilation is given by the shares, referrals, and traffic you get from your audience. Create a fantastic expert roundup post by asking a question your readers are interested in, and your contributors can easily answer.

Give this type of content the time and effort it needs and it will prove to be a fruitful initiative, both amongst your peers and as a relevant post for your website.



Source link

Continue Reading

SEO

The anatomy of a negative SEO attack

Published

on


30-second summary:

  • SEO can just as easily destroy websites’ rankings as it can build them up.
  • Newer websites or startups with smaller backlink profiles are the most vulnerable to negative SEO attacks.
  • Webmasters need to regularly monitor their backlink profile to make sure their site is not keeping company with any questionable web properties.
  • Negative SEO can be remediated through manual outreach or Google’s disavow tool, but high-quality link building campaigns are the best way to minimize the impact of low-quality links.

In the early days of search engine optimization, a variety of black-hat techniques allowed SEOs to dominate the first page of search. Cloaking, keyword stuffing, backlink spam, and other strategies could catapult websites to the first page. But those days are long gone. Google’s algorithms are extremely powerful and can easily result in a negative SEO attack. Not only will black-hat strategies no longer work – they will destroy your site’s rankings and even prevent your domain from ranking permanently. 

So for those out there on the internet who are not interested in seeing your domain move up the first page, black hat SEO is an easy way to harm your website. Many new site owners are so eager to get any backlinks that they can, they allow low-quality links to populate their profile without ever thinking about where those links are coming from, or why those other site owners linked to them in the first place. 

Negative SEO attacks are real. I’ve helped many clients recover from them. They can come from competitors, hackers, or seemingly out of nowhere, but without a quick response, a website’s reputation with search engines can be permanently harmed. 

Although Google algorithm updates or technical issues with your website can impact your keyword rankings, an unexpected drop could be a sign of negative SEO. The good news is, the anatomy of a negative SEO attack is clearly recognizable. If you take quick action, you can protect your website and minimize the damage.

The websites that are most vulnerable to SEO 

The reality is, every time your site moves up a spot in the SERPs, you knock another site down. It’s not fun to imagine that other people would use negative SEO to harm your efforts, but if you offer great service or product that could take business or traffic away from someone else, then your site is at risk.

Any website can experience a negative SEO attack, but local businesses and startups with less than 300 referring domains are the most vulnerable. The smaller your backlink profile, the more impactful any low-quality or unnatural links will be. If 50% of your links are spammy and you’re a brand new site, Google crawlers are going to look at your backlink profile and assume your site is trying to cheat your way to the top. 

For new webmasters, in particular, it’s critical to pay close attention to every backlink you acquire. This is also true when you pay for the services of a link building company. Some site owners are hesitant to pursue link building because they have had negative experiences with SEOs in the past who engaged in these spammy techniques that ended up tearing their site down rather than building it up. 

As your backlink profile grows, spammy links will not have as much of an impact on your domain authority or rankings. Still, it’s good to keep an eye on the referring domains and anchor text diversity of your backlink profile.

How to identify a negative SEO attack

There are a variety of common negative SEO techniques that people may use to harm your website. After handling negative SEO attacks with my own clients, these are the most common types I’ve come across and that I encourage webmasters to be on the lookout for.

1. Toxic backlinks

Backlinks from low-quality sites that have low domain authority, little relevance to your industry, or very little site traffic should always be suspect. If you receive a large influx of these low-quality links, they may be coming from a link farm that has the infrastructure to build a massive amount of links quickly. If you’re a new site with a large percentage of toxic links, Google will likely assume you’ve been participating in black hat manipulation.

A negative SEO attack can be cause by toxic poor quality backlinks

2. Comment spam links

One way SEOs used to manipulate their site authority was by leaving backlinks in the comment section of blogs or forum sites. If you suddenly receive backlinks in the comment section of older blogs with no relevance or traffic, someone might have placed them there maliciously. If it’s an SEO agency that placed the link and you paid for it, fire them immediately. Google indexes those links in the comment section, and it will not look favorably upon your site if you have a lot of these unnatural backlinks.

Spam comments could cause a negative SEO attack

3. Exact match or unnatural anchor text

Natural anchor text will most often include your brand name, the services or products your business offers, or more generic wording like, “Click Here.” If all of your anchor text has the exact keyword you’re trying to rank for, that will come across as manipulation to Google. If the anchor text is irrelevant, it will confuse Google bots about the content of your site. It’s important to pay attention to the most common ways that other sites link to yours so if new links don’t share at least some similarity, you can investigate them accordingly.

4. Fake negative reviews

Although negative reviews don’t have as drastic of an impact on your site authority as your backlink profile, Google does crawl and render those sites when considering whether to rank web pages. Local and small businesses with bad reviews, in particular, will not rank, so in addition to reviewing your backlink profile on a regular basis, site owners should also be monitoring the important review sites in their industry. Most major review sites allow you to report reviews if you have reason to believe they are fake.

There are other types of negative SEO that I haven’t listed here such as content scraping, links hidden in images, and more, but the above are very easily identified using Google Search Console or any type of backlink analyzer. Familiarizing yourself with the many ways that others may try to link to your site in a harmful way will help you be able to identify those problematic links right after they show up in your backlink profile.

How to perform negative SEO remediation

Digging yourself out of a negative SEO attack is never fun, but it can be done. If you’re being a responsible webmaster and monitoring your backlink profile regularly, you should have a solid understanding of what a healthy backlink profile for your website looks like, and will therefore be able to recognize the moment that something appears off.

If you believe that the influx of links is indeed the result of nefarious intentions, you have a few options to repair the damage, and hopefully, before Google penalizes your site. Some of these options are more expensive than others, but if you’re not an experienced webmaster, it is probably best to get the guidance of an SEO expert. If you remove the wrong links, you can end up performing negative SEO on your own website by mistake.

1. Request removal

The first step with any link is to reach out to the webmaster to ask for the link to be removed. Admittedly, this is not always successful. However, before you move on to option two, you want to make sure you have exhausted every effort to have the link removed before requesting Google to get involved. If the link was the result of comment spam, the owner of the blog may be willing to moderate or delete the comment. There have been webmasters who have charged my clients a fee to have links removed. Depending on the price you’re willing to pay, you can choose to do so or move on to other options.

2. Disavow file

In 2012, Google added the disavow tool in Google Search Console to give webmasters more agency in their off-site SEO. The reality is, no one can fully control the websites that choose to link to theirs in a harmful way, so it’s not really fair for search engines to penalize your site as a result. Google recognized this and created the disavow tool, however, they still advise site owners to use it sparingly.

A disavow file is essentially a list of links that you want invalidated on your domain, or that you don’t want Google to consider when evaluating the quality of your website. There are detailed instructions on how to submit a disavow file in the Google Search Console help center. Take note though that these links aren’t actually removed, Google just no longer takes them into consideration the next time they are crawled and indexed. If you’re using an SEO software that measures the quality of your backlink profile, you will likely have to submit the disavow file there as well if you want their metrics to accurately reflect how Google understands your site.

3. Link building campaigns

High-quality, contextual link building is different from black-hat SEO in that it uses original content to earn links on relevant, industry-specific publications. The best SEO agencies will increase site authority the right way, through techniques that are Google compliant and don’t harm your rankings in the long-term. If you are not actively trying to earn high-quality links for your website, not only are you missing out on the opportunity to improve your overall keyword rankings, you place your site in a more vulnerable position. If you pursue consistent link acquisition and build up a healthy backlink profile before a negative SEO attack occurs, you are more well-positioned to avoid a Google penalty.

It is certainly frustrating and unfair when negative SEO occurs, but there is really nothing that a webmaster can do to prevent it. So in the case of negative SEO, preparation is the best medicine. Knowing what to look for will help you be more prepared to take immediate action and minimize the damage. 

Manick Bhan is the founder and CTO of LinkGraph, an award-winning digital marketing and SEO agency that provides SEO, paid media, and content marketing services. He is also the founder and CEO of SearchAtlas, a software suite of free SEO tools. You can find Manick on Twitter @madmanick.





Source link

Continue Reading

SEO

7 changes make a big difference with your Digital Marketing Strategy

We know that the face of marketing has changed from past few years as people are more informed while making purchases, thanks to the information available online. These people become potential customers, and so companies prefer digital marketing over all other forms of marketing.

Published

on

DMM-d7229151

We know that the face of marketing has changed from past few years as people are more informed while making purchases, thanks to the information available online. These people become potential customers, and so companies prefer digital marketing over all other forms of marketing. Businesses need a robust digital marketing strategy since they are not harnessing the full power of this online trend. For companies to be successful in 2020, digital marketing needs to be more than just posting on Facebook. They will need to improve their existing digital marketing continually.

2020-State of Digital Marketing

Online and digital media has changed – it is no longer a world of random things and what is hot now may change later. However, there are a few simple facts that we need to consider your general digital marketing efforts:

Newcomers will enter, but there will not be new social networks every other day.

Digital marketing strategy should rely on current and upcoming new data.

Why you need a robust digital marketing strategy

It will help you focus and come up with a plan to determine how many followers you want and by how much you need to increase the conversion rate

A strategy will inspire you to get your digital marketing game healthy and be in competition.

A well-written plan will regularly check in on your current methods and see if they are working as per plan. You can optimise and adapt accordingly.

A digital marketing strategy will limit the duplication of content or resources.

It will help you experiment with new approaches to gain and keep customers.

By creating a strategy, you will be able to understand what your customers want and provide services accordingly.

Let us discuss in detail some changes that can make a big difference to your Digital Marketing strategy in 2020

1) Plan new SEO friendly content

One of the best ways of improving SEO is to create new content regularly. The content should be mapped to relevant keywords . Include an SEO friendly title that will directly impact the quality score and ranking. You need to ensure that the material consists of researched keywords in the first 100 words of an article to follow best practices and get credit for your business.

2) Revamp your social media presence

One way to engage your audience and refresh your social media presence is to leverage the trends and bring your brand’s personality to life. We can do it in the following ways

Evaluate your content and ask yourself these questions: How is it different? What engaging content can I create?

Moreover, start conversations with influencers, research for relevant topics and provide insight on articles concerning your brand.

With social media, you can keep on trying to get the best content mix, and with platformance analytics, you can understand the what is working and what is not.

3) AdWords and Content Shock

AdWords is one of the most effective methods of online advertising since the business of all sizes can use it to increase leads and customers. With AdWords, you can spend less and still understand how your audience is searching. AdWords is similar to SEO which takes time to evolve your keyword research and strategy to yield results. Google will reward you with an increase in leads to your site if you stick to it. When the creation and distribution of content are costly than the ROI it generates, it is known as Content Shock. It becomes a great asset for small players since they don’t reach the big numbers from the start. Hence, it becomes the tremendous start for small players as they do not need to scale up their digital marketing strategy.

4) Use Analytics to inform A/B testing

Analytics drives digital strategy, and one of the most important things to do with analytics is to engage in A/B testing. You can use the incoming data to apply A/B testing strategy. Finally, A/B testing is the best way to understand your customers. Resources like Google Analytics, Social Media and AdWords, are the best way to improve your digital efforts.

5) Understand the changing environment

Your biggest enemy when it comes to digital marketing is complacency. Due to changing consumer views, marketers need to identify obsolete strategies and adjust accordingly. The brand now focuses on providing content when the user needs it instead of creating content hoping users will need it. This approach requires a strategy. For a full proof strategy in 2020, marketers need to focus on three main elements: Artificial Intelligence, Social Media and Customer. If you look closely at these elements, they influence all the trends like augmented reality, influencer analytics, video marketing, personalisation, native advertising and content marketing.

6) Encourage personalised interactions

Based on a report, 73% of customers will do business with your company if you provide a customised experience. Personalising your communication helps to engage with your customers. Sending personalised emails, modifying templates, using recipient’s name, personally responding on social media and segmenting the email list based on persona are a few ways that can work very well.

7) Build your network

Try to gain information and experience of others in your field. By building your network, you can predict and work as per changing times. The network will help you prepare for challenges in future.

Your 2020 Digital Marketing strategy needs to be like the playbook which is tailored to the team and team’s competition. A simple principle in the digital world is if you don’t establish a baseline of where your company stands concerning the competitors, you will have no idea of the areas you would need to focus on beating them. Your company will really benefit if you make little changes in your digital marketing strategy by keeping track of upcoming trends and changes in the environment. If this seems like a lot, give VajraGlobal a call.

Continue Reading

Trending

Copyright © 2020 Plolu.