Recent updates to Google’s algorithms – called Caffeine and Mayday – extend search to cover a wider variety of content and to a deeper level within sites, giving smaller businesses a better chance to catch the eyes of customers

quick facts

• Google’s new indexing system Caffeine, launched in June, is 50% faster at crawling the web
• Its earlier Mayday update to its algorithm looks deeper into sites to ensure content is relevant and refreshed
• Caffeine is leading to a real-time web where popular social media topics can appear in search results within minutes
• Large brochureware sites suffered drops of 10-30% in Google traffic following the update, while smaller sites reported lifts as high as 70%

Google’s latest update to its search engine, Caffeine, has made regularly updating the whole of your site, rather than just the front page, crucial to achieving good search result rankings and attracting more customers. But it’s the combined effect of this with the earlier Mayday update that has had an impact on natural search strategy.

Caffeine, which was announced last summer but didn’t come into full effect until June this year, sees Google looking at more sites and a wider variety of content more quickly. Its indexing systems are now working 50% faster than before, the search giant estimates, bringing the so-called real-time web closer to reality.

By being more timely and scouring a wider variety of content -particularly video, images and social media comments and updates - it gives searchers more relevant results than before. That’s why Google says ’freshness’ is the main benefit for the user.

Louis Halpern, CEO of integrated agency Halpern Cowan, says this means the information brands publish will get indexed and possibly feature in search results far quicker than before. At the same time, though, so too will comments from the media, competitors and customers.

“Caffeine sums up the new era, when your brand isn’t what you say it is, it’s what Google say it is,” Halpern says. “Its indexing goes wider and is quicker, so there are now more social media references on the first page of results, particularly if a subject or brand is trending in Twitter.
“It makes it even more important for brands to project a positive image online, but also underlines how important it is for them to get everything offline working well too, to prevent negative comments getting into search results.”

Going deeper

This widening and quickening of search results comes into focus when Caffeine, an update to how the search engine looks for results, is seen in combination with Mayday, an algorithm update made last spring that dictates how it ranks those results. While Caffeine made Google results ’fresher’, Mayday extended the priority the algorithm gives to updated, relevant information deep within a site.

In the past Google’s system was seen as ensuring the rich get richer. A big brand with a lot of traffic would dominate search results by having a high rank for its main home page, which would then pass traffic to smaller sections which didn’t need to feature updated copy. The sub-page would appear high in natural search results because it was part of a site which had huge traffic levels and lots of inbound links built up over several years.

Doug Platts, head of natural search at social media agency iCrossing, points out that by looking at the relevance, uniqueness and freshness of content deep into the long tail of a site, the Mayday update is good news for brands that can produce a lot of relevant content and actively engage audiences through social media.

“A smaller business can now compete on a far more level playing field because it’s probably in a better position to quickly alter and update its site content,” Platts says. “You just need to concentrate on what you’re good at and make sure pages are well indexed and refreshed constantly with new information, such as announcements, blogs, video, pictures and twitter feeds.”

Indeed, Gareth Owen, head of SEO at media agency Steak, believes Mayday threw down a challenge to big brands to prove they can be as passionate about every part of their website and not just their front page. “A big department store would always rank highly for, say, kettles because it has a big site and is deemed relevant, even though it may only have some old brochureware which hasn’t been touched in ages,” he says. “Combining Caffeine with Mayday means such sites can’t get away with that any longer because internet-savvy businesses will be updating their pages frequently to get picked up by Google. If the changes have done one thing, they’ve shown big businesses they can’t be lazy any more, they have to be passionate about every part of their website. If they’re not, then frankly they don’t deserve to rank.”

British affiliate marketer Kirsty McCubbin reveals that her BlokesUndies and LingerieBrands sites, which have constantly refreshed content, received 30% and 25% more traffic from Google in the month after the Mayday update was implemented. On the other hand, her PersonalisedGiftsUK and FragranceBrands sites, which she admits neglecting to refresh, both saw drops of 70%.

Like any retailer, DSG International wasn’t immune and ecommerce content manager Chris Brown says the blend of Caffeine and Mayday has forced it to work harder at refreshing content deep in its site to maintain a prominent search ranking. This, he believes, is a positive step that ensures consumers get fresh, relevant, timely content both on a brand’s site and within social media, and that by providing text, video and photographic content, the retailer has been rewarded with several new top rankings.

“The Google updates have provided an opportunity for retailers and set a challenge to all of the bigger brands on the web, which is good news for customers,” says Brown. “The updates mean big brands won’t automatically gain better rankings due to their volume of content, visitors and domain authority. They’ll have to prove the worth of every page as being relevant to a search query. Google’s algorithm is about relevancy for the user, so we need to be wherever our customers want to be online, and that means engaging with the platforms they feel comfortable in, like Facebook, Twitter, blogs, forums and YouTube.”

Local advantage

If Mayday can be thought of as looking deeper into the long tail of a site, it also allows companies to concentrate beyond short one- or two-keyword searches and concentrate on longer terms. In particular, local search will prove to be increasingly important, according to Martin Dinham, sales director at SEO specialist Guava.

“Search terms are getting longer as people look for more detailed information, so there’s a huge opportunity for companies to make sure their pages are indexed so they come out well in local search,” he says. “It’ll be hard for smaller sites to compete with big brands, which may be driving huge volumes of traffic through advertising, but they can do well by indexing to appear when people are looking for a popular keyword with a location.”

Hence, while ’red dress’ might be out of a small shop’s reach ’red dress Solihull’ levels the playing field considerably, as long as it works to ensure its Solihull-relevant content is refreshed regularly.

In fact, Caffeine and Mayday, by indexing faster and wider and then shifting the algorithm to reward relevant, refreshed content deep within a site, are widely seen as counterbalancing the Vince algorithm update in spring 2009. This was described by Google as a minor change, not big enough to merit a name, hence it was named retrospectively after the engineer who worked on it most.

But Neil Jackson, search strategy director at search optimisation agency Tamar, says that despite Google’s protestations, search experts almost all agreed the tweak favoured big-name brands by somehow (Google never explains quite how) concentrating more on the page rank, inbound links and authority scores which large brands have had the time and budget to build up.

“Vince is widely seen as having prioritised the big brands and Mayday as redressing the balance by focusing on the long tail of search,” Jackson says. “People are tending to use longer, more specific terms, so Google has recognised this by extending the algorithm into the long tail of content on sites so it can offer relevant results. While the big brands’ dominance over short keyword searches was reinforced by Vince, Mayday is definitely a counterbalance through which Google is reassuring content providers of all sizes that if they have the relevant information throughout their site, they can compete.”

Although Caffeine and Mayday weren’t highlighted by Google as helping to clear up malpractice in the world of search optimisation, Caffeine is already starting to have an impact, according to Guy Levine, CEO of Return On Digital. Traditionally many brands might run the risk of buying ’farmed’ links on directory sites, which serve no purpose other than to make a brand appear to have a lot of inbound links (a must for a high position in search results). They would then wait for the links to be discovered by Google, which would take away the page rank score of the directory so the links would lose their value.

Levine says this waiting game has now speeded up thanks to Caffeine’s greater pace. He says that he and other SEO executives are seeing some evidence of brands, including one of his own clients, receiving minor warning punishments. These include being excluded from search results or a depression in page rank for a few days.

case study: World First gains from niche travel terms

The odds of ranking highly were against small travel insurance company World First as it pitched against personal finance giants which had huge marketing budgets to drive traffic to their sites and had spent many years building up inbound links.

That’s still the case for generic terms like ’travel insurance’, for which World First struggles to get above page four. However, Caffeine and Mayday have allowed it to concentrate on generating content that underlines its niche products and attributes.

“The best way we can compete is to keep refreshing our site with relevant press releases and blogs, which are distributed via RSS as well, and then tweet and retweet information and keep an active profile on Facebook,” says MD Martin Rothwell.

“All of these keep adding content deep into the site, as well as throughout social media sites, which is working really well for us now there’s more of a level playing field with niches and longer-term searches.

“We put out a lot of information about how we covered volcanic ash disruption, which got us some very high rankings very quickly because it was a trending topic,” he adds. “We also do very well in medical search terms for travel insurance. We have top spot for HIV and other conditions which are apt for travel insurance.”