There is an ever-urgent need for companies to both acquire and retain online users, especially in today’s economic environment. Our new Usability and User Experience Trends Briefing highlights this, but which is best for achieving reliable results - internal or external testing?

Econsultancy estimates that client-side companies spent around £214m on usability and accessibility during 2008, which demonstrated a massive year-on-year growth rate of around 20%. This is due to an ever-increasing realisation that a commitment to the user experience is a key element to online success.

A recent trend that we've noticed is the development of internal testing departments within client companies, rather than outsourcing to agencies. It sounds like a forward-thinking move, but should we be questioning the benefits of this approach?

In-house testing is less expensive than outsourcing in the long run, assuming you do enough of it, and it helps any company to understand how usability fits into their business model. But, although data can generally be collected quickly, there are many barriers to success, such as a lack of industry knowledge (which can make a thorough assessment or understanding of results extremely difficult). Equally, other restrictions such as time, internal issues and conflicts, and a lack of specific technology can hinder a company’s efforts.

By contrast, outsourcing user testing to specialist agencies can deliver the benefit of results being generated in an unbiased environment, with industry specialists who usually have far greater experience and knowledge (along with better resources and the latest testing equipment and related technology). External testing generally has no restrictions placed upon it by the client and as such, results can be gathered quickly and presented accurately.

While internal testing can save money, it sometimes inadvertently cuts corners, meaning that results can easily be affected. Inaccurate data, upon which important decisions are based, which may ultimately damage any overall online performance.

We recognise the value of outsourcing in order to maximise testing, and know that some of the more savvy companies employ usability agencies on a retainer basis to guarantee ongoing testing of their websites. But there's still some doubt as to whether bringing user testing in-house is a shrewd move. It certainly sounds like a good idea, and many e-commerce managers we know would surely drool at the prospect of their very own testing lab, but is it the smart thing to do?

Other issues surrounding usability, user-experience and accessibility discussed in the briefing include the increased focus on usability as a digital service, differentiating usability and user experience, the cross-over between usability and other digital areas, using analytics to help drive usability and embracing social media and Web 2.0 to enhance user experience. The briefing is available free to all membership levels.
Jake Hird

Published 8 January, 2009 by Jake Hird

Jake Hird is Econsultancy Australia's Director of Research and Education. Follow him on Twitter and Google+, connect with him on LinkedIn or see what he's keeping an eye on via diigo

126 more posts from this author

You might be interested in

Comments (2)

Jake Hird

Jake Hird, Director of Research and Education at Econsultancy

Thanks for the comments, Paul.

over 9 years ago


craig sullivan


I agree with Paul here on the benefits of in-house testing, even when this is 'quick and dirty' with staff, friends or family.
I've used a webcam with laptop and cheap software (total cost £80) to get very rapid feedback on prototypes and pre-production systems.  The return on investment was pretty awesome as running an external test would have cost several K.
I think that the only way to gain experience of moderating tests is to actually perform them.  I regularly give people copies of 'Don't make me think' by Steve Krug as this does a good job of explaining why some testing is better than no testing at all.
Another option for people is to go for the middle ground and get some training on moderation by an experience usability practitioner, build some practical experience through testing 'low budget' style and then hire a usability lab and run the tests yourselves.  We found that this was a good balance between the two.

over 9 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.