On 14 December 2008
we relaunched the Econsultancy.com site. This involved a subtle name change (“E-consultancy”
became “Econsultancy”), a new logo, a completely new look site with a new
directory structure, a new URL, on servers in a different country. We had to
migrate 10,000s of pages, deleted a load of old ones, and created 10,000s of
new ones.
The background to all this is explained in my interview about the new Econsultancy site – and question 9, about the SEO impact of this large change, is the subject of this post. What has happened to our previously excellent search rankings since the changeover?
A few caveats before we dive in:
- I only really talk about Google in terms of search engines as it is so dominant and this is particularly true for us.
- When I talk about search ‘rankings’, what I really mean is the volume of organically referred search traffic. Ranking obsession is dangerous and increasingly pointless as everyone sees their own personalised results. So ‘ranking’ I use really as a proxy for search-referred site visits.
- I’m not a great believer in PageRank in as far as you can have a high PageRank and not get rankings and vice versa. So when I talk about PageRank here I mean in the vaguest possible sense of ‘reputation’ or ‘credibility’ or ‘authority’.
- I’m also pretty sceptical about anything that Google shows in the main search engine as being anything other than vaguely indicative e.g. the site: or link: commands don’t give data that is even nearly accurate or complete.
Background to Site Changes
The reasons for completely updating the site are explained in my interview about the new Econsultancy site. Further details are given in our press release Econsultancy invests $500,000 into new website. On the domain change from http://www.e-consultancy.com to http://econsultancy.com I’ve also posted a reply to a question on this in our forum: “Why did Econsultancy drop the www?”.
What did we do from an SEO point of view for the site migration?
The short version of ‘what we did’ from an SEO migration point of view was only two things:
Firstly, we used 301 Moved Permanently redirects for all old pages which had corresponding pages on the new site. We did this by exporting all the links (80,000+) that Google told us about via Google Webmaster Tools’ “Links > Pages with external links” and writing a script to do all the redirects where there were database IDs for the old and corresponding new content, making the mapping and redirects quite straight forwards even at this volume. Then we did non-database-d content (e.g. flat HTML pages, including the ‘About’ sections, key section pages etc.) ‘manually’.
Secondly, we verified the new site within Google Webmaster Tools so that we could keep an eye on how it was being crawled and so on. We also set the geographic target audience to ‘United States’ here given our business and content is increasingly there. Previously this wasn’t set to anything.
We didn’t contact Google in any way (I don’t think you can?), and we didn’t do anything in phases – more of a Big Bang approach!
What has happened to our natural Search Referrals since the new site launch?
The graph below, showing, in Google Analytics, the number of non-paid visits referred by Google since the launch, tells the story best.
We had a brief honeymoon period where we got ‘double rankings’ where the old site and the corresponding page on the new site *both* ranked well on Google meaning our page coverage was outstanding. On the old site we used to get around 3,500 visits a day referred by Google natural search – here we peaked at over 4,000.
And then things went a bit quiet... It was Christmas so this is not atypical. But it hasn’t picked up since. Following are some more details broken down by time period following the relaunch.
Weeks 1-2 following site relaunch
-
Rankings / Referred Natural Search Traffic: This
went up in the first week (as you can see from graph above) as we
‘double-ranked’ for both the old site and the corresponding page on the new
site giving us great page coverage. It then fell away – not unusual given it
was Christmas / New Year which is historically very quiet for us.
- Indexation / De-indexation: The old site was still all showing in Google and in Webmaster tools. The new site was being indexed however – around 20% had been indexed by the end of the second week.
- External Page Links: We were getting
links to the new site (around 50 a day) but these weren’t visible in Google
Webmaster Tools. Legacy links still showed as pointing to the old domain.
-
PageRank: Nothing showing on the new site.
- Other e.g. Site links, Google News, Google Alerts, Images etc.: Our Sitelinks (the sub-links to site sections shown in the search results) disappeared. And our Google News Feed was broken so we contacted Google News to get this fixed. Likewise Google Alerts for our domain dried up.
Weeks 3-4 following site relaunch
-
Rankings / Referred Natural Search Traffic: No improvement
in search traffic as per graph above. And the referrals we do get are almost
all searches on our brand name or for ‘long tail’ keywords for newly created
content which has new links since the launch i.e. the legacy content doesn’t
appear to be ranking any more.
- Indexation / De-indexation: The old site now had only 7,590 pages showing in the index (using site:www.e-consultancy.com) when it had been over 40,000 before the changeover. So the de-indexing was happening very fast. The indexation of the new site was also up to over 20,000 pages, around one third of the actual number of pages (using site:econsultancy.com).
-
External Page Links: New links were
showing up in Google Webmaster Tools to the new site. Legacy links to the old
site no longer showed up as pointing to the old domain, however neither did
they show up as pointing (via 301 redirects) to the new site. They seemed to
have ‘disappeared’.
-
PageRank: Now showing on the new site. All the key
section pages (Reports, Training, Events, Jobs, Directories, Blog, Forums) maintained their PageRank, or
it improved. The Blog has the highest PageRank (due, no doubt to the large
number of inbound links) at 6. Strangely, however, our homepage’s PageRank had
dropped from 5 to 3. This might be because our new homepage, for non-logged in
visitors like the Googlebot, has less dynamic content on it so fewer links and
less frequently updated content?
- Other e.g. Site links, Google News, Google Alerts, Images etc.: Our Sitelinks reappeared in Google Webmaster Tools at week 4 and then almost immediately in the main index (search on Econsultancy to see main listing). Our Google News Feed was back up and running (see recent stories on Google News) and we were coming through in Google Alerts as before.
Weeks 5-6 (now) following site relaunch
-
Rankings / Referred Natural Search Traffic: Still no
improvement in search traffic as per graph above and the referrals we do get
are still predominantly brand searches. By now, if you search on the old ‘e-consultancy’
Google brings back results only from the new site, so it appears to know that
what was once ‘e-consultancy’ at the old domain, is now only at the new domain.
-
Indexation / De-indexation: The old site now
has only 1,490 pages showing in the index (using site:www.e-consultancy.com).
The indexation of the new site has gone up to 26,500 pages (using site:econsultancy.com
in Google). Yahoo!
shows more at 33,000+.
-
External Page Links: As per graphic
below all legacy links are now
showing in Webmaster Tools (over 90,000) as linking to the new site, so it
appears that Google has figured out that the old pages have ‘become’ their new
equivalents. However, this is not
reflected on the main index using the link:econsultancy.com
command which shows only the links acquired since launch – 563 at the time of
writing. Yahoo! is now showing 52,444 inbound links to the new domain (using Yahoo!
Site Explorer). Checking for links to the old domain on Google (link:www.e-consultancy.com)
now brings back the same results as for link:econsultancy.com so, again, Google
appears to understand they are the same thing and that ‘econsultancy.com’ is
now the primary / master domain.
-
PageRank: No change – as above.
- Other e.g. Site links, Google News, Google Alerts, Images etc.: No change – as above.
So what are we
working on now?
A number of things including:
- Looking at ‘Not found’ 404 errors reported within Webmaster Tools’ “Diagnostics > Web crawl” to 301 redirect any pages we can.
- Updating key links to our new domain – we can’t expect to update all 50,000+ but, for some, e.g. our dmoz.org listing we’re making the effort.
- Continuing to get new links for the new site. Though this is mostly done via focusing on content and marketing/PR as opposed to explicit ‘link building’ efforts. The latter includes listings sites and directories (e.g. for our training and events). Also, we’re continue to build out our feeds and distribution (e.g. jobs data to job aggregators).
- Working on getting images coming through in Google News and Alerts. Not directly an SEO thing, however. More about increasing click throughs – to then get links, to improve SEO indirectly.
- Doing more PPC to make up for shortfall in natural search traffic. No wonder Google is in no hurry to make the updates... ;)
- Bits of on page optimisation and internal link optimisation to promote the relevant pages within the site.
- Reviewing whether to update the current homepage for non-logged in visitors (including Googlebot) to include more dynamic content so it refreshes more often and has more links and keywords on it. Not convinced this will help but worth a try.
What have we learned?
Page/URL changes within a domain seem to be updated pretty fast (within a few weeks) assuming you do the 301 redirects correctly. Not so for domain name changes it would appear. Some people report 4 weeks, some 6, some 90 days, for the latter. We’re at 6 weeks now...
The indexation and de-indexation process happened over a matter of weeks too.
Webmaster Tools is useful for getting some insight into where Google is up to. However, it’s not clear what time lags there are between what you see in Webmaster Tools and how this translates to the main index.
We’re struggling to explain the drop in PageRank for our homepage which has the most inbound links, particularly when PageRank carried across fine in all other instances.
Otherwise, we’re playing a waiting game... All that’s now missing is that search traffic. Ask me again in a month whether, in hindsight, we shouldn’t have changed the domain name at all, whatever the brand police might say!
(p.s. any useful tips, advice for us going forwards greatly appreciated)
Ashley Friedlein is CEO and Co-founder of Econsultancy. Follow him on Twitter (3,800+ followers) or connect via LinkedIn (4,500+ connections) or Google+.
Recommended reports
Search Engine Optimization (SEO) Best Practice Guide
Built on the foundations of our previous, highly-renowned report, Econsultancy's SEO Best Practice Guide contains everything you need to know about search engine optimization. At more than 300 pages long, this document will help you understand search marketing like never before. Make no mistake: this guide contains lots of actionable, real world insight. It will help you immediately start to improve your performance across the search engines.
Paid Search Agencies Buyer’s Guide
The Paid Search Agencies Buyer's Guide focuses on the UK paid search (PPC) marketplace and profiles 35 leading agencies. The 200+ page report contains contains detailed information about the current trends and issues affecting this sector of digital marketing, as well as advice about how to find the right agency for your needs.
SEO at Cheapflights Media
3:25PM on 4th February 2009
Hi Ashley
Interesting post. The main issue I'd pick up is regarding the approach to 301's & 404's.
If I'm reading this correctly, this sounds like the most concerning bit. By only redirecting pages that have external links reported by Google you would stand to miss a considerable number of URL's - both pages that do infact have external links (not reported) and other URL's that are known to the engines that they may attempt to re-crawl without necessarily following links.
Yes... good way to go now. In addition, if you can do any further pattern matching on old-to-new URL's for redirecting, I'd suggest you do.
Director of Strategy at SEOptimise Ltd
3:41PM on 4th February 2009
Hi Ashley, I've taken a quick look over this and while a temporary drop is natural in any site migration, I would have expected an upward trend of gradually improving rankings/traffic since the initial reduction in traffic.
One of the reasons is because there are still a large number of URL's indexed on e-consultancy.com, so removal of these should be a main focus:
http://www.google.co.uk/search?q=site%3Ae-consultancy.com
But the most obvious reason for reduced SE traffic, in my opinion, is the move to US hosting & US geo-located webmaster central settings. This is a big risk which is likely to result in lower Google UK rankings, in addition to complete de-indexing for "pages from the UK" queries. Long-term this may prove to be a successful strategy if the rankings which were previously popular in Google.co.uk replicate across to the far larger US market on Google.com. But in the meantime it does complicate the 301 redirects, making it more difficult to maintain the original Google UK rankings from e-consultancy.com.
I would look to:
- remove all e-consultancy.com indexed pages, creating temporary links to force Google into crawling these URL's.
- ensure redirects from all old URL's now point to the most relevant version on the new domain. Looking to claim back as many long-tail rankings as possible, this may account for a large percentage of overall traffic.
- Review top-performing keywords from old domain, check the redirects from URL's which were ranking well previously, and consider re-optimising the content around these keyphrases.
- measure the difference in rankings for top keywords and review the referred traffic stats from Google UK and Google.com before and after the site migration. I wouldn't recommend anything as extreme as moving the hosting back to the UK just yet, but I would strongly consider geo-targeting the website to the UK in Google Webmaster Central. This should help towards identifying the website as UK-based in Google and hopefully steadily increase rankings back to where they were on e-consultancy.com.
Looking forward to the next update when hopefully this has all improved! ;)
CEO at Econsultancy
6:17PM on 4th February 2009
Thanks for these comments:
@Adam - sorry, we didn't *only* base our redirects on pages reported as having external links in Webmaster Tools. We did every page we could. However, Webmaster Tools showed us inbound links to pages which either no longer existed, or were there but 'hidden' e.g. not in the navigation. Basically, we tried to get all and every page migrated.
Currenly, in Webmaster Tools it's seeing around 3,500 'not found' 404s out of 92,000 inbound links, so not a high percentage. Still, we'll mop up these remaining few thousand very soon (tonight in fact though Google will need to recrawl).
@Kevin - there aren't actually that many old domain URLs left (somewhat over 1,000 relative to more like 30,000 on the new site)? And most of these links, if you click on them, redirect correctly to the new site. Are you suggesting we request (via Webmaster Tools) for these URLs to be removed? Any way of automating that?
Interesting theory about the geo-location thing. I've never been overly convinced by this for large sites with genuinely international content and customers. Around half our traffic always came from the US even when hosted in the UK. I'd assume Google might weight results according to geography a little (based on host IP and/or Webmaster tools setting) but this would be less relevant than it using the linkage data to work out how relevant a site was to a particular country market? The truth is that we are genuinely a global company (though with UK roots / heritage) so it doesn't feel accurate to lable ourselves as UK.
Director at Velocity
8:34PM on 4th February 2009
Thanks for 'opening your kimono' on this sensitive issue. It's a lesson to anyone who cares about Google performance (everyone).
Keep us informed as the traffic comes back.
Sounds like you're doing everything right. But the Google algorithm is like the Oracle at Delphi... impenetrable.
There's a class-action suit waiting to happen that will force Google to be more transparent. It's a monopoly now and needs to act more responsibly to the businesses that live and die by its decisions.
CEO at Econsultancy
8:56PM on 4th February 2009
@Doug - yes, I think there are certain duties that Google should have. Surprising that regulators haven't got more involved sooner. When you think that they're the biggest media player in the UK (in my view, certainly bigger ad revenues than Channel 4, and ITV?) I'm surprised that politicians aren't more interested. Perhaps they will be after they realise the power of the internet for the Obama campaign?
I'm surprised there isn't a simple mechanism (within, say, Google's Webmaster Tools which is an authenticated environment) to explicitly notify Google that 'what was this site, is now this site' so transitions can happen much more smoothly.
Equally, as I've said before (e.g. http://econsultancy.com/forums/other-topics/ethical-search-practices-from-leading-agency-you-be-the-judge?page=2#forum_post_4320) I think Google should have to have certain service levels, and appeal levels, for things like de-listings given they can crucify businesses. But fair enough that they should be able to charge for this.
9:11AM on 5th February 2009
In my opinion You should redirect to 301 all pagess which You don't have in database (old->new) to new main page or 404 on new site. It is waste of time to search right now each indexed pages.
I can be wrong.
Director of Strategy at SEOptimise Ltd
10:33AM on 5th February 2009
Ashley, I wouldn't recommend requesting to remove the remaining indexed URL's from Webmaster Central, it's important the weight of these pages are passed onto the relevant version on the new domain - rather than just removed. I would suggest creating a temporary webpage (which isn't highly visited by users but visible to search engines) containing links to the remaining URL's, this is to force these pages to be crawled and follow the redirects. While 1,000 URL's isn't high in comparison to the size of the website it may still be significant enough to impact rankings so it would be reassuring, if nothing else, to have these completely removed.
As for the geo-targeting of the website, if the traffic was previously widespread from global search engines it sounds like it's more of a case of waiting until the full history/link reputation of e-consultancy.com has transferred across to econsultancy.com. I agree that Econsultancy should be targeting a global audience, however the move may have had some influence over the reduced level of Google UK traffic. For example, the site is no longer ranking for pages in the UK queries and this may also be the reason for the drop in rankings in Google UK web searches too.
CEO at Econsultancy
11:12AM on 5th February 2009
@Kevin - yes, take your point about the site not ranking in pages in the UK which you would assume might well be because of the US targeting criteria we applied in Webmaster Tools. I've now removed that targeting so it doesn't have anything - I imagine this is as close to 'global' as it gets and the algo will do the rest in terms of guessin relative importance/rankings by country.
Director at Screaming Frog
11:12AM on 5th February 2009
Hi Ashley, Interesting write up. Two big changes at once with the geotargeting and change of a domain.
You can see you are not ranking well regardless of location (in .co.uk, .com /.com with &gl=us parameter) for the end part of your page title of the homepage for example - "Community of Digital Marketing and Ecommerce Professionals". So it's fairly clear all your old authority has not really transferred yet added to the geotargeting issue.
I'd agree with Kevin on this one. International filters are very strong unless you are a big authority and going for the longer tail terms.
So you are ranking in Google.co.uk for your brand -
http://www.google.co.uk/search?hl=en&q=e+consultancy&btnG=Google+Search&meta=
But not 'pages from the UK' as you switched your targeting to US -
http://www.google.co.uk/search?hl=en&q=e+consultancy&btnG=Search&meta=cr%3DcountryUK|countryGB
Which is how it should be. But quite few of your new site pages are still in the 'pages from the UK' results -
http://www.google.co.uk/search?hl=en&q=site%3Aeconsultancy.com&btnG=Search&meta=cr%3DcountryUK|countryGB
The as an example, take the reports page - http://econsultancy.com/reports. For the exact title, it's not ranking great in Google.co.uk even with your brand.
http://www.google.co.uk/search?hl=en&ie=ISO-8859-1&q=Online+Marketing+Research+Internet+Marketing+Research+E-consultancy&btnG=Google+Search&meta=
Another page is ranking in the pages from the UK page results-
http://www.google.co.uk/search?hl=en&ie=ISO-8859-1&q=Online+Marketing+Research+Internet+Marketing+Research+E-consultancy&btnG=Search&meta=cr%3DcountryUK|countryGB
In the halfway house that is Google.com from the UK you are ranking 2nd page -
http://www.google.com/search?hl=en&ie=ISO-8859-1&q=Online+Marketing+Research+Internet+Marketing+Research+E-consultancy&btnG=Search
With the &gl=us parameter (as if you are in the US on Google.com) you are nowhere.
http://www.google.com/search?hl=en&ie=ISO-8859-1&q=Online+Marketing+Research+Internet+Marketing+Research+E-consultancy&btnG=Search&gl=us
So Google looks a little confused at the moment with your switch in targeting. Plus the added fact you have just switched domain thrown on top the timeline gets longer.
CEO at Econsultancy
11:24AM on 5th February 2009
@Dan - thanks. Poor old Google, eh, can't get its head round things... ;)
As per my reply to Kevin, which presumably sneaked in as you were composing your answer, I've switched the targeting to nothing now. Reckon that's the correct thing for a global company/site?
Director at Screaming Frog
11:39AM on 5th February 2009
Yeh my comment was being composed at the time you replied!
It's difficult, from what I can see - your old site was treated as a UK site. So any organic rankings you achieved in the US were probably from brand/longer tail terms (obviously you will know this from your analytics) as it's very difficult to be truely global. That's why so many sites have targeting issues and end up having multiple tlds or subdomains etc.
As you are a .com and you switched hosting to the US, Google may still treat you as US based. Although you do have quite a few telling fingerprints onsite (address etc) and obviously your old link profile that suggests you are from the UK!
I definitely think you have made the right move taking away the US targeting. It's just whether you now should change that to the UK :) You should be able to tell how Google treats the site with your latest change and judge from there.
CEO at Econsultancy
11:48AM on 5th February 2009
@Dan - yes, we'll keep an eye and let everyone know what happens! Our overall aim, and I'm sure Google would endorse this, is not to do things for SEO reasons per se but to do the right thing for our users and our business.
Given we are global we'll just have to live with the trickiness this brings but I'm sure Google should be able to use its linkage data and semantic analysis to figure out how 'relevant' we are for different markets.
As far as I'm aware we've done everything as Google would wish. But even they caution against these big moves somewhat. We think (hope) that it's just a matter of churning the data. But no-one knows how long this takes / should take. Which makes business continuity planning very difficult.
I don't see why Google couldn't have (paid) services to manage such migrations? Though, as a tech company, I understand they're keen not to get their hands too dirty with providing services.
Director at Screaming Frog
12:30PM on 5th February 2009
Thanks Ashley - please do keep us updated with the sites progress. I agree with your comments, you have to do what makes business sense first.
I have had this discussion a few times and I definitely think it's something Google needs to get better at for sites with truely relevant global content.
These things do take time and you made two very big changes at once.
I am not sure if Google will ever offer anything like a paid migration service, but I think they could do a whole lot better providing guidelines in situations like these. WMC etc was a great start and I am positive they are on the right path improving their communication with webmasters.
CEO at Econsultancy
1:07PM on 5th February 2009
Pretty galling to see that a Google search on 'Site migration and SEO' brings up a blog that pastes in a bit of the above and links to this page, whereas the original content (this page) doesn't appear to be in even the top 300 results even though it *is* indexed.
Director at Screaming Frog
1:31PM on 5th February 2009
Yeh, shows a distinct lack of authority on the new domain at the moment. If you add 'impact' aswell, you are 7th-
http://www.google.co.uk/search?hl=en&q=Site+Migration+and+SEO+Impact&btnG=Search&meta=
Which shows you can rank for less competitive more specific queries. Like other blog posts titles -
http://www.google.co.uk/search?hl=en&q=inflexible+high+street+retailers&btnG=Search&meta=
All part of the same thing.
Director of Product Development at Econsultancy
4:14PM on 5th February 2009
I'm now seeing this page ranked 6th for that phrase, possibly due to a link or two pointing this way.
The fact that blog posts rank higher is maybe due to an appearance on Google News (Econsultancy articles are picked up by Newsbot). Certainly this used to help the speed of indexing articles on Google.com.
10:53PM on 5th February 2009
Really interesting post, it's not often we get this kind of insight into a site migration. I work on topgear.com and was involved in the migration of content with a complete site re-design to a new platform back in late September. It is tough migrating a large amount of content, and maintaining your search traffic and there are definite ups and downs, and waiting..!
We also had an international issue thrown in with separate sites for different countries using redirects.
One check I would always do, despite not using the ranking of pages as a KPI, is taking the top 100 keywords that were referring to your site before, and rank checking them now. I would safely assume that your top 100 keywords before were likely to be at least page 1, and by running it now, you could at least see if there's a trend in particular keywords dropping in ranking. You may find that there are additional problems.
One thing I've also noticed is that your 404 page is blank which could be fixed to help out and convert people to the appropriate place. You should put Google Analytics on there too so you can identify the URLs which produce the 404 errors quicker. You also might find that a majority of your search traffic is hitting this blank page, unlikely, but something to know I guess. As you said your webmaster tools list of 404s should help out sorting some of those things out.
A final question I guess, is there significantly less content linked to or available on the new site?
As always, with my addiction to stats and investigation, I'd love to dive into details and look further at it, so give me a shout if you ever wanted an additional perspective.
CEO at Econsultancy
9:03AM on 6th February 2009
@Sheppy - thanks for those comments. On the keyword rankings - haven't done a detailed analysis but it looks like we've dropped down the rankings across the board for all our key terms, apart from our actual name.
On the 404s - yes, we are tracking these. And, in fact, we don't get that many. And those we are getting mostly *should* be 404s i.e. the pages genuinely no longer exist and have no equivalent. So, really, we want them to be found as 404s and de-indexed.
On the content volume / links - yes, quite a lot of content that was on the old site is now not on the new site but a lot of it redirected to equivalent content so no huge drop in inbound links. However, the new site has a lot of new content that wasn't on the old site (e.g. our Member Directory and Supplier Directory which between them account for an additional new 20,000 pages or so).
At the moment, our only conclusion is either a) just need to wait a bit longer or b) we're being penalised for something we're not aware of.
10:54AM on 6th February 2009
I am with Kevin on this one with regards to the setting of your regional target to be the US in GWT. I'd actually class this as a bit of a blunder, and I would be very surprised if any of your close, personal SEO friends would have made such a recommendation had you asked them/us in advance!
What you haven't identified is whether the search referrers you are enjoying are more from Google.com and/or from US visitors, and whether the majority of the fall off in search visitors is from the UK and the rest of the world.
CEO at Econsultancy
11:21AM on 6th February 2009
@Steve - yes, I think the US targeting probably was a blunder. However, I only applied this about 10 days ago and removed it 2 days ago. Our problems were there long before this change. I supsect the specific consequence of this targeting would have been to ruin our UK rankings - if we'd had any to ruin.
The fall off in visitors is across the board globally so it's hard to do any sensible analysis. Of 170,000 or so visits since relaunch, 56,848 visits have come from Google via 25,204 keywords. That's around 34% of total site traffic. Which is not bad except that it was around 70% before the relaunch. The tail is very long - our top referring terms at the moment are "econsultancy" = 5,637 visits in the period, "e-consultancy" = 2,955 visits in the period and "e consultancy" with 1,745 visits in the period.
It does seem strange to me that if you search the web on the title of this blog (site migration and seo) that we don't even rank in the Top 100 search results (I gave up paging through after that) and yet plenty of sites that link to this page rank there. That can't be related to the country setting - you can do any variant you like with .com, .co.uk, pages from the web/UK etc. and we're still not there. That, to me, looks only like some kind of penalty?
Director of Strategy at SEOptimise Ltd
11:52AM on 6th February 2009
@Ashley, looking at this I think it's safe to assume the problem was caused by switching the hosting location to the US back when relaunching the Econsultancy website back in December. The move to becoming a US based website (in the eyes of Google) was then strengthened with the WMC US geo-targeting setting which followed.
I've done a few searches for econsultancy title tags/article posts and in many cases the UK hosted econsultancy.mobi is outranking econsultancy.com (despite the majority of inbound links pointing towards .com). This shows that the locational issues are certainly causing confusion, with econsultancy.com now being treated as a non-UK website and Google UK rankings dropping as a result. In the meantime, it appears that search rankings outside of the UK have not improved as quickly as expected.
Director at Screaming Frog
11:56AM on 6th February 2009
It's not a penalty but may feel like one unfortunately :)
CEO at Econsultancy
12:23PM on 6th February 2009
@Kevin. Maybe... but I'm still not convinced. The .mobi site was always called www.econsultancy.mobi so that hasn't changed *at all* in the changeover (as you'll see - it still has the old logo at the moment). It didn't change URL or IP address. If we'd changed its URL then we might be able to compare like with like better.
Does it not seem strange to you that if you search the web on the title of this blog (site migration and seo) as I suggest above that the source content (this post) doesn't rank at all whereas sites linking to it do..? That just doesn't seem right even if Google currently thinks we're a 'US' site.
BTW, what's common thinking on how Google interprets where you are and to what degree the search results (assuming you're not logged in) are tailored accordingly? e.g. If I'm in the UK but search using Google.com and don't select the 'pages from..' radio button then how different is that from doing the same thing but in, say, the US?
CEO at Econsultancy
12:39PM on 6th February 2009
Just for the record, Google's guidelines on URL and IP migration are as follows:
Google Webmaster guidelines on moving to a new IP address
Google Webmaster guidelines on moving your site to a new domain
Director of Strategy at SEOptimise Ltd
12:55PM on 6th February 2009
@Ashley, Google.com results are global, but still weighted towards the search location. Notice the difference between Google.com results (when searching from UK) and Google.com results when the US filter applied.
My point about the .mobi site is that this is ranking above econsultancy.com despite only having 19 links as opposed to 21k+ links on .com, the clear difference is the hosting location with the UK located .mobi out performing .com in many instances.
I agree that it's strange the "site migration and seo" post isn't listed for Google.com searches. But in my opinion it's unlikely to be a penatly, especially considering the circumstances of the domain and location changes, a third reason would be very unlucky! But it's possible that the location change is still in transition, caught in the middle of being labelled as non-UK and becoming US - with global search rankings weakened until this is resolved, this should happen naturally but may take time.
SEO at Cheapflights Media
12:59PM on 6th February 2009
Ashley
Did you make significant changes to your whois record?
Seems like an awful lot of changes in one go, which may have tripped some filters with Google.
CEO at Econsultancy
1:20PM on 6th February 2009
@Adam - obviously we updated the domain server details so that econsultancy.com changed from pointing at our old UK hosting location to the new US one. But nothing other than that.
Founder at Search:Johnston Google Consultants
1:37PM on 6th February 2009
Hmm, not true.
Here are 25,000 still in Google from the forum:
http://www.google.com/search?q=site:e-consultancy.com/forum/
Which don't redirect correctly:
http://budurl.com/econredir1
There is a 301 and a 302 in there, which is likely to explain the retention of these URLs in Google.
Still looking...
CEO at Econsultancy
1:54PM on 6th February 2009
@Steve - gets a bit technical (i.e. increasingly out of my depth ;) ) but the 301 and 302 are deliberate. In migrating our forum we moved from a threaded one to a more blog-style one where comments are listed below the original post i.e. on one page rather than multiple pages.
So what were actual pages on the old forum are now #comments in the new structure. And because the URL for the comments can change (e.g. when they move from page 1 to page 2 as more comments appear) we need to know the original thread/discussion URL (which is 301-ed) and then check to see where the comment now is so it is only temporarily moved. This also applies where comments are deleted for example and thus the URL of comments changes.
Founder at Search:Johnston Google Consultants
3:21PM on 6th February 2009
I understand the reason for your 301 then 302 redirect, but I'd argue that Google is sufficiently unconvinced by them that it would prefer to hold onto the old ones until it gets a more categoric response.
Given that it is also still holding onto 1200 posts to the old blog:
http://www.google.co.uk/search?&q=site:e-consultancy.com/news-blog
And encountering incorrectly configured redirections elsewhere:
http://budurl.com/econredir2
Two 301s this time.
I'd start working through the redirects that aren't being accepted by Google and make them 301 a single time, and to somewhere distinct. Many of the blog redirects appear to be redirecting to the new blog home page. This is another reason Google could be holding onto them.
As long as Google has your reputation and content split across the two domains, you can't hope to regain your search performance.
I haven't finished looking, but need to go out for the rest of the day. Good luck in the meantime.
5:57PM on 6th February 2009
Excellent overview of the process you embarked on during this change.
You are correct in stating that internal "same domain" 301s propagate quickly in most instances.
A domain change is major and I've seen things take upwards of 6 months to a year for recovery depending on the root domain.
What was the status of the current domain before the redirects?
Some additional things I would do is drop a NoArchive directive on the entire site. Get the cache out of the equation.
I'd also block Internet Archive and any other archiving services so there wasn't anything historical being referenced out there.
I did a quick review using our URIValet and it appears you've got everything in order on the surface. Nice clean URI structure too, bravo. And, no extensions, how 2009 is that? :)
2:10PM on 7th February 2009
Ah, was running some more tests and you may want to address your https issues and the recursive redirects and 302s taking place. There may be some confusion there for UAs. We usually block ALL https from being indexed via the robots.txt file. We'll serve a separate robots.txt for https requests.
CEO at Econsultancy
11:09AM on 9th February 2009
In an ironic twist we're currently ranking top of Google for a search on SEO best practice... ;)
We don't appear if you choose pages from the UK but that makes sense as the pages aren't in the UK.
On Google.com we're top if you put the quotation marks around the phrase ("SEO best practice").
We're also 4th on a search on paid search markting.
And yet, if you search the web on the title of this blog (site migration and seo) we still don't appear to be ranked anywhere despite it being indexed and having quite a few sites link to this page. Nor do we rank on something like 'econsultancy accessibility' when you'd think our Topic page on Accessibility (with PageRank 5) would surely come up...?
It may be that the above improved rankings on competitive terms represent the first green shoots of ranking recovery...? It might be that these pages are the first to churn through the system and have the reputation from the old site properly passed across to the new one?
If you look at Google's index of the old reports section of the site it shows no results. But for the new equivalent it's showing 422. So it seems that this entire section has been properly and fully 'migrated'? And it is these reports/pages which are ranking so well as above.
Conversely, Google is still showing over 24,000 pages in its index for the forum on the old site and only 2,150 for the new site despite the fact that all the old posts are on the new site and redirect from the old to new.
Founder at Search:Johnston Google Consultants
12:50PM on 9th February 2009
Ashley,
I'd advise against getting encouraged by individual results. Check your visits from natural search in total and if they aren't on the up, these improvements may be phantoms.
Get stuck into fixing the redirections and let us know how Webmater Tools reports on its crawling of the old e-consultancy.com domain once they are corrected.
Steve
CEO at Econsultancy
2:10PM on 9th February 2009
@Steve - thanks. You might well be right, but I'm still not sure. What you are saying is that our 302s on top of a 301, or our double 301s, are confusing Google? Is there any evidence for this anywhere? Surely Google would have some technical guidance explaining the limitations of their crawler if this were the case? Maybe this does exist somewhere...?
As far as we're aware we're doing what is both technically correct by the various http standards and what is right for the user e.g. in the case of the migrated forum posts, where an old page with a unique URL has become a # comment, then we could point all the posts in a thread to the same page (with a single 301) but the user would be 'lost' on that page as it would not then # link to the relevant comment. So the user experience would be broken and Google always say to prioritise this.
Can we not try and ask someone at Google to clarify, from a technical point of view, what is correct practice?
Founder at Search:Johnston Google Consultants
12:13AM on 10th February 2009
Evidence, sheesh! You don't want much do you!
While Google is a great respecter of Web Standards, their correct application across the web is pitifully low, so it has to frequently second guess what web sites actually want or mean when it experiences inconsistent signals. You may remember the 302 redirect hijack exploit, whereby temporarily redirecting a domain or a URL to the home page of another site on a different domain, caused the content from that home page - along with its search performance - to appear on the redirecting URL. You will also recall how Meta Refresh redirects failed to achieve the Google migration desired by web sites who, on hosted domains where they had little control, could not implement server-side redirection. In the latter situation now, Google largely treats it as a permanent redirect because that is likely to be the safest outcome for the domain owners, and the 302 hijack appears to have been minimized by some algorithmic tweaks that (largely) spot the patterns that suggest someone is up to no good and mitigates against its effect. Neither of these current treatments are exactly faithful to Web Standards.
The evidence in your case is very simple, the 24,000+ URLs from your old forum are still being attributed to e-consultancy.com despite the likelihood of them being recrawled many, many times since you moved domains. Faced with a 301 AND a 302, Google has decided (in my opinion) that to do nothing is safer, because a mistake may have been made.
So, it isn't a limitation of its crawler, it is the wisdom of experience being applied to situations that it is not finding convincing.Trying to do the best thing by you, the Webmaster, and by Google's users.
Tackling the specific issue of the forum posts, let's just look at what Google experiences. On the surface, you seem to have developed a neat idea for the amalgamation of the separate forum posts that were previously on unique URLs, such as:
http://www.e-consultancy.com/forum/112054-pop-up-surveys.html
By first redirecting them to a permanent location for the thread that also has a unique ID that identifies the specific post, such as:
http://econsultancy.com/forums/best-practice/pop-up-surveys/10058
And then temporarily redirecting to the following URL to maintain the relationship with the original post:
http://econsultancy.com/forums/best-practice/pop-up-surveys?page=1#forum_post_10058
Unfortunately this means Google actually gets to see just:
http://econsultancy.com/forums/best-practice/pop-up-surveys?page=1
because the anchor details (#forum_post_10058) are not passed via the header to Google.
Why is this all significant? Because what Google is seeing each time it encounters a permanent redirect for each URL of the unique content from each individual forum posts, is a subsequent temporary redirect to what turns out to be an identical page each time that carries all the forum posts (and most threads are only one page long, from what I can tell). Still with me? Many unique pages of content being redirected via URLs that don't ever really exist, to arrive on an identical page. Hell, I'd be confused. Remember Google ISN'T seeing the anchor. It just sees the same page at the end of each redirect.
What I can't tell is whether one of the many temporary redirects for each post has actually been accepted by Google and has claimed the thread page, and all the others are being held back due to a content duplication and redirection confusion, but this could easily be tested by looking at the mapping between the original post IDs and the newly indexed post number; although it seems that 83 may have made it through:
http://www.google.co.uk/search?&q=site:econsultancy.com+inurl:forums+inurl:page=1
I think your mistake here was to introduce the intermediary step; the URL like this one:
http://econsultancy.com/forums/best-practice/pop-up-surveys/10058
which presumably was created to try and defend the uniqueness of the content from that original post, and in fear of its ability to acquire visits being diminished by its addition to the single thread page. If this is true, I think these reasons are both misplaced, but we can have that debate another time.
So what to do? I recommend the following:
Shift the redirect pattern to a single 301 from the original unique post URL, such as:
http://www.e-consultancy.com/forum/112054-pop-up-surveys.html
to a location that deposits the link reputation and context of the original URL where it can be amalgamated with the other good content that is on the topic, while retaining the anchor key to defend the user experience during the transition. This URL should look like your final destination:
http://econsultancy.com/forums/best-practice/pop-up-surveys?page=1#forum_post_10058
but it should be the permanent final location.
Aha, you may say, but we wanted to retain the relationship between the post and its location in the thread, and be able to manage that over time. Well, sadly, that was always going to be an illusion, because Google was never going to be able to use the anchor location information, which means that once the migration is complete, the context of the old post's unique URL and its new location in the thread will be lost, forever. Meaning visiting users to these new pages are just going to have to go looking for the relevant information in the page for themselves.
One final, 'you may ask': You may ask would Google have ever carried the original target for the permanent redirection with its anchor encoded as a simple parameter despite it being subsequently redirected to a duplicate page? I'd suggest that if it ever did, you wouldn't find it turning up in any search results because there is nothing unique about them. They all would have the same content attributed to them which would trigger their de-duplication from any results. Google is really good a de-duplication these days.
Will fixing this solve your traffic problems? Well, I don't know about that, because you have had my divided attention, and this has hardly been the most thorough visibility audit process. But, I know that this is exactly where I'd start working.
As far as getting Google to talk to you about this, I reckon you're as likely to get an audience as anyone in my direct acquaintance.
CEO at Econsultancy
9:06AM on 10th February 2009
@Steve - that's great. Thanks for drilling into the necessary detail! We have now made the changes that you suggest so let's see what happens with GWT.
What you say makes sense and can't be a bad thing - I'd be a little surprised if it was our forum alone that was holding back the migration of our old-to-new-site reputation though there is quite a lot of content there going back over 10 years.
Today I'm seeing a search on "site migration and seo" bring back this post (finally!) in Google ranked 11 (page 2). Still not as high as it should be, in my unbiased opinion, but at least it's there.
Yesterday, Yahoo! News also started to show our stories again (though a little selectively).
Some positive signs of change perhaps but no significant increase in search referrals yet... A couple of people I've talked to have suggested that for a URL + server/IP change at once they'd expect it to take 8-10 weeks to filter through.
Chief Architect at Econsultancy
9:26AM on 10th February 2009
This is just untrue; the fragment component is, of course, "passed via the header" along with the rest of the URL, whether to Google or to any other HTTP client. If the fragment wasn't passed to the client as part of the response, the redirect wouldn't work correctly.
Any client is free to make its own decision about whether to treat URLs which differ only in their fragment component as pointing to "the same page" (web browsers do, for example), but this is unrelated to redirects.
Founder at Search:Johnston Google Consultants
10:33AM on 10th February 2009
Ashley, that's good to hear, keep us posted.
Tom, you're welcome ;-)
Yes, you are correct, I've got my terminoligy mixed up a bit. It was late when I made the post, so let me get it right now. You'll know this stuff, Tom, but in the interests of the forum readers, I'll spell it out.
When Google asks for the old URL from e-consultancy it is sent the location of a new resource (page) as a redirection. The word 'redirection' is slightly misleading because it is then necessary for Google to request the new URL before anything will happen. When it does this, it gets presented with another redirection, this time to the temporary location with the anchor (or as you correctly label it, the fragment). The problem then occurs because Google - as with any other agent of my understanding - then requests the URL WITHOUT the fragment and, in our experience, as evidenced by the lack of fragmented URLs in Google's index, it does nothing with it. (This is unlike a web browser, which will process the fragment after it has received the content from the URL, in order to locate the item identified by it.)
So, yes Google does SEE the anchor in the URL, but in our experience/knowledge, does not PROCESS it, meaning it gets presented with multiple duplicated pages at what were originally multiple unique pages. So the effect is the same as I described earlier, even if my description of what Google sees of the fragmented URL was wrong.
In the future, we suspect that Google may have to modify it's handling of URL fragments, particularly as Ajax becomes more widespread as a form of managing content at a single URL.
3:29PM on 10th February 2009
Crikey - this article - and the resulting comments - is the mother of all pages for 'advice on changing your domain name & servers for an established site'.
Well done to everyone
However, I can't help thinking that what you have actually produced is a damned fine reason for NOT changing your domain.
CEO at Econsultancy
3:42PM on 10th February 2009
@ Alan - yes, you might be right ;)
What I want to get to - using this thread as a living case study - are some 'findings' or takeaways for others to learn from e.g. around our probably blunder with country targeting within GWT, or the precise nature of redirects required.
Most importantly, though, I want to get to some kind of idea of *how long* a site owner should expect to suffer whilst reputation permeates across from one site to the new one. Not that this answer can ever be final, and will change over time, but at least to help site owners (and their agencies) manage expectations and plan their business and marketing accordingly.
So, for example, it might be that, assuming you do all the redirects etc. correctly then:
- Changing URLs but keeping the domain and server the same (e.g. an information architecture update) = 2 weeks (with Sitemaps to help)
- Changing URLs AND server/IP = 4-6 weeks
- Changing URLS AND servers AND Domain = 9-10 weeks
But the above are currently just guesses given we're now getting close to 8 weeks.
1:01PM on 12th February 2009
Hi,
Really a interesting post regarding changing domain name. I am sure if you are using 301 redirection than you will get same page rank as you had.But I have a concern regarding name changes. Why you change your domain name "econsultancy" instead of "e-consultancy"? While pronunciations of your domain name is the same.
4:20PM on 13th February 2009
Really useful post giving a lot of insight/transparency into a huge site migration.
In december I moved just our homepage from an /index.html to an /index.php. I set up 301 redirects, but even now the PR is lower than it was and google developer tools has removed all of the inbound links to the homepage (without an extension). Im guessing its a waiting game.
Will be interesting to see what comes out of this post - looks like it could make another econsultancy report!
12:57PM on 17th February 2009
@Ashley - left you a few comments on Google Support Group thread you started. Seems that some of your redirects through HTTPS are not set up correctly.
I'd also see why content that was cached uner your domain seems to be from other domains - were you using framed content at some stage?
One other thing I pick up from this post - updating you geolocation to US. If you are now hosted in SU I'd leave this with no geotarget. You'll be perceived as a US site from your hosting location and TLD, but also have the opportunity to rank better generically. No need to use the geotargeting I think, and you might find it better without.
Biggest issue I can imagine though is not managing redirects for a 90k page site. You have to do this in manageable chunks that allows Google to crawl and see all redirects gradually for best outcomes.
Hope this helps, rgds
Richard
CEO at Econsultancy
5:08PM on 17th February 2009
@Richard - thanks for comments (and in Google Support Group). We've changed https://www.e-consultancy.com/ to now use a single 301 redirect to http://econsultancy.com/ though I'm not convinced this will make much difference.
You talk about "content that was cached under your domain seems to be from other domains". Like what? We've never used frames and all content has been hosted on our site.
On the geolocation targeting - it isn't set to anything. I did change it (wrongly I now believe) but only for about a week before changing it back.
On the redirects - yes, it was a big challenge. But I'm still not convinced this can be the fundamental issue. Around 85%+ of the redirects came through very quickly and visibly in Webmaster Tools and a lot of the page not founds (404s) that remain *should be* 404s. I'm not sure how one could sensible move part of a site at a time...? I can see that we could have staggered the moving of domain and IP/host into stages.
But overall I'm still seeing an effect which looks like some kind of 'dampener' or 'sandbox' that is working at a domain level. I think 'domain authority' is a somewhat debated topic but it seems clear to me that our old site had a high domain authority (we immediately ranked well for any now blog or forum post) whereas our new one doesn't (dodgy sites scraping our content are outranking us).
12:18PM on 18th February 2009
Let me spam here with my comments...
What is still in my mind (and unfortunately at this point it's quite theoretical): how easier would it make for Google to evaluate the move of the site if you'd have been much more strict about allowing duplications of your content at least in the last year: one thing was perviously here, now it's there with proper redirection- despite the serious other changes.
As for Google there is another side of the coin: folks tend to buy more or less well-ranked sites and simply 301-redirecting the pages to their pages in the hope of better rankings. So Google cannot follow blindly even the proper redirections- have to (and obviously doing so) use many other things to evaluate a site after bigger changes:
domain: changed
Site-structure: changed
Content structure on pages: might have been changed slightly
Content: probably the pillars of the strong content haven't changed so much- but they can be found at other places unfortunately
IMO you've left only one obvious sign (beside the redirect) for Google: the inbound links. If they will not be removed (and why would they) by time then it can be strong sign to Google that yes- it's the "same" site just in better packaging- but this surely takes time.
Plus additionally changing the geotargeting together with hosting location- haven't made things easier.
So I'd put my money on that in your case the "recovery" is chances to be near to the 180 days mentioned by Google (best practices- in this case I'd consider it as a "hint" for not "simple" cases)
P.s.: sorry for being theoretical without a single usefull advice (though it's hard being "after" the changes...)- I was looking for the "restore my rankings fast" cream whole night in my wardrobe but couldn't find it :-)
All the best,
BB
CEO at Econsultancy
12:58PM on 18th February 2009
@BbDeath Thanks for those comments (and in Google Forum). Let me know if you find the "restore my rankings fast" cream ;)
I think what you say is right. It will come but it will take time. BTW, where do Google mention "180 days"?
I've said before (maybe even in this thread though its getting very long!) that I'm surprised Google don't make it easier for webmasters to explicity say "I'm moving this domain to here" to help with such migrations. Couldn't they use Google Webmaster Tools to do this? Have a 'Moving Site' feature? With this + 301 redirects + inbound links I'm sure Google are smart enough to pull it off...?!
Delving into it a bit more it seems clear to me that what we have lost is our 'domain authority'. At a page-by-page level the pagerank, and rankings, transfered really very quickly. That's how our SEO report can still top Google for a competitive phrase such as 'seo best practice'.
Where we have lost traffic and Google rankings is precisely where we used to get most of it and that's to our Blog and our Forum - those places with the most volume of new content every day. On the old site we could post anything and it would be indexed and ranking pretty much top of Google within 30 minutes of it being posted. This despite no inbound links, no history etc. It *must* have been our domain authority.
Now, the indexation is pretty much as quick but we're not ranking. We are ranking on specific pages which existed on the old site and have a direct equivalent on the new site with lots of inbound links. But new pages on the new site aren't ranking at all. And I'm sure that's because we currently have no 'domain authority' on this domain. Will our 301 redirects and new inbound links bring back our old domain authority? Probably... but in how long? Don't know.
Founder at Search:Johnston Google Consultants
8:26PM on 18th February 2009
Ashley, it is interesting that you say it is the Forum and the Blog that are under-performing the most. It is exactly these parts of the old site where the migration is so incomplete.
At today's date, 18th Feb 2009:
The Forums from the old E-Con domain have 23,000 URLs still indexed
The Forums from the new Econ domain have only 2,300 URLs indexed
Meaning that Google has not really migrated the Forum and its link reputation at all yet.
The Blog from the old E-Con domain has 1,200 URLs still indexed
The Blog from the new Econ domain has 7,500 URLs indexed
Meaning that Google has left a sizeable proportion of the content and the link reputation behind on the old domain.
The figures for these critical areas of the site are therefore nothing like the 85% successful migration you experienced for some of the other areas of the site. And therefore remain, in my humble opinion, the place to keep working.
Google is reporting a total URL count for the new domain as 32,500 (which can't be entirely trusted) which brings the scale of the above URL counts into focus.
And yes, I completely agree, that it should be rather easier to pre-notify Google of an imminent domain move. But for now, it isn't.
11:59AM on 19th February 2009
I'm emitting fumes- my FFF crashed and lost my reply so I'll be short.
Would be very beneficial some autosave feature here for these cases (or for me to stop using FFF)...
>> BTW, where do Google mention "180 days"?
"best practices moving your site"- keep control on your old domain for at least 180 days. Maybe it's a little bit twisted- but if I want I can understand it in a way that if the movving was done properly 180 days has to be anough for Google to evaluate the changes.
>> Have a 'Moving Site' feature?
Actually it has: proper permanent redirection IMO works pretty well if we are talking about simple "moving": not even touching the source code just put the site to a new domain.
But it's not your case: as you changed name, site structure, redesign, urls, targeting- if you'd have changed content (meant the "static" content) then this would be a completely new site...
Imagine how much money and time would it take to communicate these changes offline and "rebuild" your brand for a more or less different target audience: I think with Google it's still ways faster and less painful
>> Will our 301 redirects and new inbound links bring back our old domain authority? Probably... but in how long? Don't know. <<
Eh.. Domain authority... Very theoretical. Somthing surely exist (though have not even the slightest idea how it can be grabbed with algorythmic tools). Nomen est omen- offline and online too.
New links? I'm am more then a little bit skeptic about virtually any kind of link building- specially if in any way it is related (you haven't told anything about this :-) to having your content duplicated allaround the web, because of feeds or anything else. But obviously I'm not in the position the evaluate in any way your "link building" but can show a site ~#5 for keyword drawing a couple thousand searches daily (just in the UK): ~37 external links...
Beside this I think you are right: the site will recover just time, time.
All the best,
BB
P.s.: I'm dying here without my avatar...
10:51PM on 19th February 2009
Hi Steve - unlike the other guys here I am not tech at all but did a site redirect in Nov and came to this post via google support. I cant imagine your pain at moving such a big site. Well mine was a smaller site but very painful indeed. I tried moving from fashionivy.blogspot.com to my own domain thefashionivy.com and have decided to do it in phases now because my traffic plummetted from about 2000daily to virtually nothing after a month redirect. I panicked and reverted to the old blogger while i build the authority of my new site. My new domain has been live since Nov 2008 and I have been adding loads of fresh content and still no joy till about 2 weeks ago when i got spikes of traffic from serps - also a week after requesting resubmission. Since then my trafic has been yo yo ing. Most days with no traffic at all from google and some few days with spikes of traffic. I am just sitting tightly building backlinks and hopefully when the domain is more established I can redirect the blogger site again. Strange thing is my old blogger domain doesnt rank as well as it used to before i tried the redirect.
CEO at Econsultancy
6:39AM on 20th February 2009
@Steve Yes, you are correct about the migration of the Forums and Blog. I'm not quite sure *why* Google is holding onto the old domain and URLs for so long?
Admittedly for a small % of the Forum URLs we appear to be still doing a 302 on a 301 (for some of the old .asp pages) e.g. www.e-consultancy.com/forum/email.asp?message_id=102563 which, interestingly, show as follows in the SERPs on Google i.e. with it apparently indexed on the new domain but showing with an IP address rather than the domain, *and* showing on the old domain.
However, I'm pretty sure all/almost all the blog posts have always been straight 301 redirects so I don't why Google is still holding onto them.
In terms of 'keeping working on them' I'm not sure what more we can do except wait for Google to recrawl/update?
We've now got a Sitemap for the whole new site in Google Webmaster Tools (over 60,000 URLs) and what is clear is that Google is being slow to actually go through those URLs - is does 10,000 or so every 2 days at the moment. Bot needs its 20% time off clearly... ;)
@BbDeath 'Link building' for us really just means content + PR (not as in PageRank) so nothing dodgy!
On your avatar - You can show something but would need to join/register first. It then shows Gravatars automatically I think or you can add your own photo. But I suspect you're not the type that will want to go through our joining forms... ;)
Chief Architect at Econsultancy
11:18AM on 20th February 2009
You don't need to join to get an avatar -- just set one up on gravatar.com, registered against your email address, and then provide that email address when you comment here.
That being said, please do join!
Popular three-letter acronym at BbDeath
6:35AM on 23rd February 2009
Not as if I'd have found the above mentioned "cream"- but I'm back. Just for a test how my self-protrait looks here...
Specially if the previous comment by Tom was an "invitation" then thanks for it. But if I misunderstood- then consider that generally I cannot speak English.
BTW: any "good news" about rankings?
Founder at Search:Johnston Google Consultants
9:19AM on 23rd February 2009
Ashely,
Yes, you are right. Now that you have changed all the redirects from the forum to a single 301, it should just be a matter of waiting until Google recrawls them all before it migrates the forum properly.Your 'show as follows in the SERPs on Google' link is a fascinating piece of schizophrenia, where Google has attempted to follow the 301+302 link by attributing content to a new location, while leaving it at its old location at the same time. I suggested in a previous post that Google may have accepted some of the temporary redirects and this is may be such an example - of course it will no longer be an example, when Google recrawls the original URL (which makes me realise that this thread is losing all of its examples as we go along and change them).
The old 'news-blog' path items, on the other hand, are surprisingly resilient - still reporting some 1200 URLs in Google, but only showing 22 when I look - with the ones retained seemingly all redirecting to the new blog home page, which may explain why Google is holding onto them; just in case it is a mistake that they are all redirecting to the identical URL.
The other problem is that with the site's reputation now split across two domains, Google will be reducing its crawling interest in the old one. If this level of crawling continues, a Sitemap for the old URLs on the old domain may accelerate its interest in sweeping up the remnants.
CEO at Econsultancy
9:51AM on 23rd February 2009
@BbDeath - I see you've registered now and I can see your familiar skull motif on the back end but sadly not above. I suspect this because you've set your profile not to be public in your settings in which case, by default, we don't show profile photos (on the assumption people wouldn't want them shown). I'm sure we can figure out a way to get your 'picture' showing though!
@Steve - yes, we should really being pointing to screenshots rather than the live listings. So here's the "fascinating piece of schizophrenia, where Google has attempted to follow the 301+302 link by attributing content to a new location, while leaving it at its old location at the same time" for posterity.
CEO at Econsultancy
10:24AM on 23rd February 2009
@All - is the Google love coming back...?
Might be temporary, might be that BbDeath is our guardian angel, but it looks like we might have recovered our rankings all of a sudden this last Saturday. In which case it would be exactly 10 weeks after our site migration.
Have a look at this graph of natural search traffic referrred by Google since the beginning of this year. You can see a clear upturn in the last two days which is all the more significant given this was the weekend which you can see is usually much lower than during the week.
We can now see ourselves ranking on terms like:
digital marketing jobs
checkout process
SEO PR Training
...and, this post now finally ranks where it should do on 'site migration and SEO'
Why has this happened?
- Could it be that the Reconsideration Request we put in on 30th January has been 'answered' (unlikely as I'm sure we haven't done anything penalty-worthy)
- Could it be that changing our 302-on-301 redirects in the forum to plain 301s has 'fixed' a problem (probably has helped but unlikely to be the main factor?)
- Could it be that our submitting of a Sitemap to Google Webmaster Tools, which we did 5 days ago, for the new site has 'fixed' a problem? (again, no doubt helps, and is mentioned explicitly in Google's Best practices when moving your site but would be surprised if this was the deciding factor?)
- Could it be that our work over the last few days in updating internal links from old content to point directly at the new domain rather than go via the redirects has solved the problem? (no doubt helps but unlikely alone to be the deciding factor?)
- Just a matter of time and the data / changes have finally filtered through (most probably).
- Someone at Google felt sorry for us (unlikely but you never know... ;) )
What we're doing now
- We're still working on updating internal links from the old content migrated to the new site to point at the new domain rather than go via the redirects e.g. there are lots in old forum posts still to be updated.
- We're improving Page titles and automatically adding meta descriptions to all pages. Unlikely to help directly in rankings much but might help with click throughs on SERPs as the extracts will be more meaningful.
- We're going to implement "No-follow" for all links in our blog comments and our forum. Partly to avoid being seen to be linking to 'bad places' where we get spammed, but mostly just to reduce the amount of comment spam in the first place (as most are doing it for SEO reasons).
- Continuing to review what is happening in Webmaster Tools and tidy up 404s, redirects etc. where possible.
- Hoping this upturn isn't a temporary blip!
Popular three-letter acronym at BbDeath
9:02PM on 23rd February 2009
>> might be that BbDeath is our guardian angel
The next SEO myth?
Otherwise glad that things looks to be better- IMO not temporary, but wouldn't be surprised if rankings would bounce for a while.
>> Could it be that the Reconsideration Request
IMO grossly possible. It was advised several cases by Googler to file a reconsideration request even after 'would say' "tech" mistakes that heavily confused Google how they (mean the algo) see, understand and evaluate the site.
>> Could it be that changing our 302-on-301 redirects in the forum to plain 301s has 'fixed' a problem<<
Hm- the question is how "many" of them were present? If many then it's not a "small" issue (I haven't realized it previously except in the mentioned case- but I thought it's a special, not general mistake).
Otherwise IMO 302-on-301 is quite a "killer" combination: Google is very "bad" in following redirect chains (even if permanent- strongly advised for example to replace A>>B>>C with A>>C and B>>C)- personally I couldn't understand what does sy wants with a 302-on-301 chain.
The sitemap is of course helps, updating internal links (again the question is how many links are we talking about)- if this is about "many" links then it's "crucial".
"Felt sorry"- the algo not looks to be too emotional ;-)
Nofollow? For SEO? There are some funny examples how not-to-use it for 'seo'... And nofollow doesn't really help if sy is linking to bad neighbourhood. Beside this (according to it's original purpose) completely agree applying it on areas not under editorial control.
Best,
BB
p.s.: thanks for the 'assistance' re avatar. Much better (at least for me)...
CEO at Econsultancy
10:56AM on 3rd March 2009
Just as an update - seems like the upturn in our Google referrals which began on Saturday 21 Februrary, 10 weeks after the site migration, is holding.
Some more encouraging graphs:
Econsultancy Site Visitors February 2009
Econsultancy Google natural search referrals February 2009
5:38PM on 9th March 2009
Thanks for posting this case study - I decided after a year of hosting my niche site in a sub folder of my 'name' domain that I'd buy a new domain name - and changed hosts at the same time. It's a wordpress migration so it is still the same site structure. But a couple of weeks after migrating I have lost all my SE rankings that I had built up with the original site. Traffic has crashed to a mere trickle. I have 301 page for page set up so it looks like I might just have to wait it out.
CEO at Econsultancy
10:45AM on 3rd April 2009
Oh joy...
Our search referrals have just fallen off a cliff again. But this time I don't think we've done anything at all (no site migration, URL change etc.).
Here's a graph of referrals from Google natural search - you can see the uplift when our rankings came back in mid Feb. But you can also see the dramatic fall off in the last few days.
Ironically, this has happened at the same time as the PageRank update happened, and our homepage has gone from PageRank 3 up to 6 (where it always used to be). Shows how pointless PageRank is!
Google's idea of an April Fool's? I fear not.
An inexplicable drop in rankings... that's Googlewang!
11:24PM on 10th June 2009
Ashley, I seen this all around the net. People talk about this Google Slap phenomenon.
A huge site migration can be very tricky but you did a wonderful job.Google will get around like it usually does.
CEO at Econsultancy
7:10AM on 11th June 2009
@Submitter - thanks. This graph shows our weekly Google SEO referrals since December 2008 to June 2009. As you can see a roller coaster ride! Down then back up, then back down, and back up again. So on the moment we're up though not as much as once were...
Director at Screaming Frog
9:03AM on 11th June 2009
You probably could of done with this new feature a little earlier Ashley -
http://googlewebmastercentral.blogspot.com/2009/06/out-with-old-in-with-new.html
CEO at Econsultancy
9:42AM on 11th June 2009
@Dan Sharp - a Change of Address feature... Hallelujah! Only 6 months late for us... :(
Still, that has been long overdue but great that it is now there.
4:28PM on 11th June 2009
I just came back to this blog because I remembered what a difficult time you had when you were working through your re-directs. I was going to point out how Google has really started to click on that we need a way to inform it of a domain change and someone has beaten me to it!
Oh well I wrote a blog on my thoughts of the new webmasters tool overall:
http://www.4psmarketing.com/google-webmasters-tools-have-a-re-vamp-a-tell-google-youa-ve-changed-domains.html
or click my name :)
7:57PM on 24th November 2009
Hi Ashley, Great Post! Unfortunately we are sharing your pain.
a) We're in the late November now and I just wanted to know if you fully recovered your search rankings?
b) We have started a similar, but slightly more of a straightforward (at least I think) domain name change in June, 2009. We wanted to do this for a while but were very concerned with loosing our natural rankings. As soon as we saw the new option for "Change of Address" in webmaster tools, we decided that we can finally proceed with no or minimum risk since Google has finally addressed this issue...
Well, so we thought…. We went straight by the book:
1. Made an exact copy of the site on the new domain
2. 301 redirected every single page from the old domain to it's exact twin on the new domain
3. Clicked on the "Magic" Change of Address button
4. Even updated many links pointing to us, to the new domain/pages
5. Continued SEO campaigns on the new domain
We are couple of days away from the 180 days period now, and most our keywords did not recover their positions.
Just to be safe, we even sent over "site reconsideration" request to make sure we are not banned. There was no negative response so being "banned" should be out of the equation.. Plus we are seeing certain queries recovering. Very small number, and not as high though.
These are the exact words from Google for “MOVING YOUR SITE” that state the most important thing – “retain your site's ranking”
“If you're planning on moving your site to a new domain, here are some tips that will help you retain your site's ranking in Google's search results.”
Once again, we went straight by the book, and are still suffering.
Founder at Search:Johnston Google Consultants
10:25PM on 24th November 2009
Did you check on the status of the new domain before you switched to it? There was an infamous case of a publisher moving to a .com from the equivalent .co.uk without first checking on the history and the indexed presence of the new domain. Which, it turns out, had been used extensively for spamming and link farming, and hey, probably porn. If your new domain has any sort of similar problem you may be migrating into a black hole.
CEO at Econsultancy
4:38PM on 26th November 2009
@Alex
Sorry to hear your rankings suffered. Perhaps Google's new feature isn't quite as effective/seamless as it should be? Or perhaps there's something else at work (e.g. dodgy domain as Steve suggests).
You can see here the volume of referrals from Google natural search from 15 Dec 2008 to today.
You can see the first crash... then an upturn... followed by another crash. Since then our referrals have been steadily going up again. And they are now higher than ever they were before our site migration.
You can see the timings in months on the chart.
7:13PM on 26th November 2009
@Steve – Great Comment! Yes, we definitely did research on the new domain and even worked on it for 6-7 months prior to migration. We achieved PR6 and top positions on couple of keywords. Only then we were 100% sure that the domain was not banned and strong enough to move our old PR5 domain to.
@Ashley - Thank you for the analytics chart. It's good to hear that everything is back on track for you guys.
I just received a response from a Google tech... I hate how their answers are always so mysterious. I assume they simply give you small hints. Please let me know your comments on the post below, if you don’t mind. It is very interesting how """ You don't really need the "Change address""""" ... I'm not buying it 100%
--------------------------------------------------------------------------------
>>> Isn’t the most important goal is to retain your site’s ranking in Google's search results.
This is sure something you want - not necessarily something Google is after ... :D
>>> What is the exact purpose of “Change of Address” tool?
I think it's just a placeholder for the redirect in cases the redirect itself can not be installed for some technical reasons. You don't really need the "Change address" feature if you're able to correctly redirect addresses on the server (webspace) involved. Any move of addresses (technically redirected 301 on the server's side, or virtually changed using the webmastertools, or both, will take a lot of time to get reflected in Google's SERPs.
There's always a lot of delay between :
1. the change you yourself have made
2. the crawling of old pages, now redirecting
3. the crawling of new pages redirected to
4. the corresponding update of the index
5. and - regarding ranking, the whole process may trigger a re-assessment of your site
Mind that a complete change of addresses (call all this "change management") is ... ehm ... something that happens to a site once in a decade (< I'm serious here, cool URLs don't change!) perhaps, and that you really have to be patient to go through all that.
--------------------------------------------------------------------------------
10:51PM on 26th November 2009
Great to hear you have stabilised your ranking. I posted while ago on this post because I suffered the same fate like your site and its encouraging to see since april your referals have gone up
I am pinching myself because I neglected my site instead in Jan and just forgot it...too much for me (I never knew my self esteem was that closely related to the google serps)
Now I am back to where I was, in october I started working on the site again and this time it was rising steadly in the serps even though I had not touched it for months. A week after I started updating and working on the site thats about end of october and I am hit again with sandbox or whatever its called and all the traffic virtually disappeared
I then remembered this post to see how you got on and its pleasing its paid off and it looks like it took over 4 months to get stable rankings, yay only wish I had kept at it BUT I will now
Many thanks for sharing this, very useful
5:26AM on 3rd February 2010
great post! thank you!
10:19PM on 19th August 2010
Google webmaster tools are very helpful. Even if you aren't getting many visitors, it shows which keywords you are missing out on.
2:29PM on 10th June 2011
Sorry del <a href=http://google.com>.</a>
4:21PM on 10th June 2011
delete