If you have a website for your business it is more important than ever to understand how search engines work so you can improve your online visibility and, just as importantly, avoid being penalised for breaking Google’s webmaster guidelines.

To the uninitiated Google can seem to move in strange and mysterious ways when it comes to ranking websites in search, which is being a little unfair to the search giant.

Google’s primary aim to return the best possible results for each unique search hasn’t changed over the years. It wants to make sure that it delivers the best results and ensure that people continue to use its search engine.

What has changed is its response to people attempting to game its way to the top of the rankings. This is reflected in its comprehensive webmaster guidelines about what to do (and not to do) on your website, and the efforts that Google has put into developing ways to identify and punish websites that break these guidelines.

Breaking the rules

This is all well and good, but if you don’t know the rules it can be easy to break them, and that’s where the innocent can fall foul of the search engines.

Take one of my clients, an SME that specialises in communication skills training for corporate clients. In order not to reveal anything confidential, let’s refer to them as Comms R Us.

Looking to increase the number of inquiries from their website they had sought out an SEO specialist to help them increase their online leads.

Looking at their website it became evident that the majority of traffic that they were receiving was branded, people searching for ‘Comms R Us’ and they weren’t ranking for non-branded search terms. Obviously it was unlikely that they were going to get any new leads if they weren’t ranking for generic searches for their services.

On further investigation it became evident that they had fallen foul of Panda, one of Google’s quality algorithms. Panda is a Google algorithm that was designed to weed out websites with duplicate, low quality content.

This is where it gets difficult.

Google doesn’t tell you when you have received an algorithmic penalty like Panda, and the only way that you can tell is by looking at your website traffic, and if there is a drop, checking to see if it coincides with an algorithm update.

If you want to check your own website for potential penalties you can see the history of Google updates here: https://moz.com/google-algorithm-change.

Looking at the traffic it was evident that there was a loss of almost 50% of traffic to the website over the last weekend of March 2012, the weekend of a Panda refresh, and this loss of traffic was non-branded. Comms R Us was still ranking number one for branded search terms.

Just as Google doesn’t tell you when you have a penalty, it doesn’t tell you why you’ve got the penalty either. This is where it gets interesting.

Because of the time of the traffic drop, we can tell that that Panda was the issue, and we know then that the problem was to do with content on the website.

So the next question is what is ‘content’ in the eyes of Panda? And what parts of the content on the website have caused the problem?

There are literally hundreds of reasons that Google can penalise your website, but generally most Panda penalties fall into one of these categories:

  • Duplicated content: pages that are either identical or near identical both on your own site and on other people’s sites
  • Thin content: pages that have low % of unique content – i.e. just a couple of sentences per page and don’t provide any value to the user.
  • Overt keyword stuffing: pages that over-do the targeting of keywords with unnatural repetition, or creating multiple pages for similar keywords.

So what went wrong with Comms R Us?

Comms R Us had put a lot of time and effort into their website. They had invested in a custom design, made their website responsive so it worked on mobile devices, created landing pages for each set of courses, and individual pages for each individual course. They were adding new content to their website by blogging every week.

All their content was unique, they weren’t copying anything either. So where did it all go wrong?

A website analysis pulled up the following areas that we think have contributed to the penalty:

Title tags

The title tag, which is what is displayed in the tab in your browser and is usually the blue clickable link that you see in Google results, is the most important on page element to Google. It is the strongest indicator of what each page is about (although it does need the support of the rest of the page as well).

The team had created a title tag which led with their top three keywords and then the title of the page. For instance the homepage title read like this:

Interpersonal Skills | Communication Skills | Presentation Skills | Home

And when the name of the page was the same as one of the existing keywords you got more duplication (and a title longer than 72 characters:

Interpersonal Skills | Communication Skills | Presentation Skills | Interpersonal Skills

From our perspective this was the most obvious issue with the website. Just to summarise, title tags should:

  • Be unique for each page
  • Be backed up by the content on the page
  • Target one keyword (and maybe one or two secondary keywords/synonyms)
  • Read naturally
  • Entice the reader to click
  • Be under 72 characters in length

As you can see their titles broke pretty much every rule. The first thing we looked to do is redesign the title tags. Here is an example of a new title tag:

Interpersonal Skills Training Courses | Comms R Us

As you can see this new title follows all the rules, and it also adds in some additional keywords to help the page rank for searches relating to their services.

Duplicate content

While the website itself was all unique content, Comms R Us had bought a number of alternative keyword rich domain names which they had pointed back to their website. This action created copies of the website which Google had indexed, creating exact duplicates of the website in Google’s database.

Fixing this issue was simple – simply take the alternative domains offline and park them to remove the issue.

Thin content

This was a tight call, and was the most contentious discussion. While the content was all unique, most of the pages were quite concise, and we did have a number of pages where the only content was a YouTube video. Being a WordPress site we also had lots of category and tag pages, all of which were indexed.

We made the decision to remove all category, tag and video pages from the Google index by noindex/following each page as these were the worst offenders from a duplicate and thinness measurement.

As well as adding some additional content to our course pages, we discussed noindexing some of the blog articles as well, but in the end we decided against this as we felt the content although brief, was appropriate and we didn’t think it would be seen as spammy in the eyes of the search engine.  

The wait

We fixed the majority of issues with the website back in January 2015. But as I write this we are still waiting for a recovery (or not) from the penalty.

Panda is not a continuous algorithm like the main Google ranking algorithm – it runs sporadically.

After a couple of years of Google running the Panda on a regular basis, it hadn’t been run since September 2014, so in effect we had been waiting since January 2015 to see if our changes where going to have the desired effect and return our lost rankings.

The latest refresh started running in late July 2015, and unlike previous refreshes which were all over in a week, this one is designed to run over several months which means more waiting to see if we have done enough to remove the penalty.

As of September 2015 we are yet to see any improvement, a situation that appears to be quite widespread amongst websites attempting to remove the penalties. 

Is this fair?

To be clear, I don’t have a problem with Google and what it is trying to do – after all its aim is to make the web a better place and I think it is doing that. What I do have a problem with is the lack of transparency and (especially) the recovery time involved when a website falls foul of their rules.

From Google’s perspective, I think it is walking a pretty tight rope regarding general business acceptance, although its domination of the search results means that any threat it has from competing search engines is minimal. If I was Google I would be more worried about more anti-trust regulations being bought in.

There have been murmurings that this whole process is just a way to raise their advertising revenue, and if enough sites with influential owners get hit there could be an increase in regulatory pressure, but I think the fact that a penalty can simply destroy a business that relies on online rankings is a far greater issue.

Comms R Us has been lucky that their business was not based on online enquires and leads alone. Others have not been so lucky.

For many organisations, losing your websites rankings and traffic overnight can spell the death knell to their business. That or a huge increase in AdWords budgets. And for those who don’t exclusively rely on online traffic for sales and leads a drop in rankings and traffic can still be a stunt on their growth.

There is an interesting debate to be held about this process. From my perspective there are a couple of things Google could do to be more transparent and help out website owners:

  1. Let us know when we have received a penalty
  2. Let us know what the penalty is for
  3. Run algorithm updates on a regular basis to encourage website to fix their quality issues

I’d be interested in everyone’s comments below particularly those who have been affected by Panda previously.