Usability Testing – Are the services out their bad? Or just plain Lazy
Thursday 23 June 2011 - Filed under Usability News
In recent months I’ve spoken to more and more clients on the phone. Though they of course have similar desires to get the most out of their usability testing budget, the majority also think usability Testing is one of those necessary evils.
Though some organizations swear by it as a means to an end I mostly hear stories from clients who have dabbled in it at some point and been very disappointed in the results.
Furthermore I think its strange that most individuals and organizations I advise have admitted that the value of observing people using and testing their designs is the most important factor in bringing a product to market. Still though 1 in 3 of our clients have been ‘very’ disappointed with usability services they have previously bought.
What’s gone wrong with Usability Testing services?
After talking to clients and gaining an insight into their experiences it has become more clear.
The answer is not that the current services out their are wrong or bad. Just not really ‘fit for purpose’.
Take this conversation I had last week with a product manager from a large Blue chip organization. He conceded that in the past 6 months he had employed the services of two video orientated User Testing companies. He was sorely disappointed with the results he had received. To make matters worse each video had cost a ‘mean’ average of $39. Being a large organization he of course required a large User Testing panel, furthermore he had multiple products which required testing.
Now I don’t want anyone to fall off their chair, but up until the point I spoke to the client he had commissioned more than 300 videos. Now I’m not a mathematician but that’s $11700 for a product he wasn’t satisfied with. It’s not exactly getting what you pay for is it?
The final insult was realized when he told me he had ‘lost count’ of how many man hours it had taken him and his team to analyze the often ‘random’ and ‘unstructured’ data he had received. His User Testing budget was near depletion and the man’s problems didn’t just end with the poor quality service. He had to plough through it and create a presentation discussing the findings in detail to his Director.
As someone with a background working for large corporations I felt the clients pain and it wasn’t my money or time. He wanted the full package, the opinions, revelations, perspectives, motivations and of course the real user experience that makes user testing as effective as possible. Why is it too much to ask for all these crucial elements to be made up into a good looking, well structured and most importantly ‘presentation ready’ document?
Heres what the Testing Users client had to say.
” The cost/reward ratio of these video sites to test my users is not what I expected. Though we did get some comprehensive feedback, the money and the time spent analyzing the data with my team could have produced far better and economical results using the Testing User model. The Testing User analysis I have received is very impressive and enough for me and my CFO to award all future usability testing work to TestingUsers.com. The service definitely stands out as a new and improved way to conduct Usability Testing research. Instead of writing a blanque cheque for what are effectively home movies I have in my hand a great value, professional document, ready for me to take in the board room. I only wish Testing Users had started 4 years ago.”
As I realized the extent of the problem with the current User Testing services out there I affirmed my belief that the majority of clients who came to us were not getting the results they wanted or deserved.
Its not their fault though. It’s my industry’s, which has experienced an apparent ‘laziness’ and complete ‘lack of imagination’ with respect to bringing the basic model of Testing Users up to date.
I’d even go as far as to say there was more than a hint of ‘collusion’ amongst some video testing providers to keep the service as bland as possible.
The frustrations don’t end with larger corporations. Many of our small business clients have issues of their own (see my small business blog).
Before the Testing User Analysis was developed the industry trend seemed to be that most of the time it was just not economically for a small business to conduct Usability Testing on their web presence and that Large corporations no matter how much money they threw at video testing were not able to achieve their desired results.
The Testing User Analysis addresses these concerns by changing the way usability data is both analyzed and presented. We ask the questions;
Why should comprehensive and professional User Analysis be out of reach for small businesses?
Why should good value, well analyzed and presentation ready User Analysis be ‘off limits’ to large corporations?
Read our Why should small businesses be interested in Usability Testing?
2011-06-23 » Sam
