Performance Metrics Explained

Wednesday 31 August 2011 - Filed under Usability News

These days most technology that we use in everyday life requires some type of interface for users to interact with. We know websites have links for us to click on, dvd players use remote controls. The list is endless.

Interaction almost always has an effect on our behaviour. These behaviours are pretty crucial to performance metrics because they are almost always measurable in some way.

You can measure ease of use by say how long it took a customer to find a specific product on your website and take it to the checkout, or in the case of a dvd remote control, how many mistakes were made before the user could navigate to a certain task.
Without these measures usability testing or more specifically performance metrics isn’t possible. These performance metrics are the key to establishing the effectiveness and efficiency of your product. They identify areas for improvement.

Performance metrics can also be a very useful tool in establishing the magnitude of a particular usability issue or problem. If you know ‘somethings up’ chances are you probably want to know a little more information like how may people is it likely to effect and to what extent it can be improved.  So by for example measuring how long it takes a group of users to complete a task you can establish whether you have a usability issue. I mean if only 50% can complete the task in your desired or expected time it’s obvious there’s a performance usability issue.

For company managers, shareholders and usability professionals, usability and performance metrics are usually areas which you need to sit up and pay attention to, especially when the data is presented professionally . Most of the time organisations set aside large usability budgets with the goal of gaining success for this.
When focussing on a company’s web presence organisations often see the data from these performance metrics as a reliable indicator of any potential revenue increases or any potential cost savings that can be made.

Performance metrics when applied to most technologies need a sufficient sample size of of participants. The statistics and data will give an indication most of the time weather you have 2 or 3 participants all the way to usability testing panels of 100+ people. Put simply though a study involving just a few people though will only indicate the worst of the worst issues. To gain reliable performance metrics you need a panel of ideally 50+ people.

Performance Metrics though are only one part of the Usability Metrics picture. They will confirm the ‘what’ but not the ‘why’. These metrics focus on issues like length of time to complete a task and success rates in task completion, emphasising areas of the product interface that may be problematic. This is great but most organisations when looking for problems and their solutions would want to support this with other self-observed and surveyed data.

The most common performance metrics include task success, time on task, errors made, efficiency and learnability.

2011-08-31  »  Sam