Scenarios in usability testing
One of the most important aspects of running a successful usability test is getting the scenarios right. Making a mess of scenarios will, more than anything else, result in a usability test that is worthless or highly biased.
Good usability test scenarios can be hard to write – they have to be realistic, have enough detail to be complete, be jargon-free and should not bias towards a particular action.
Most importantly, and I cannot stress this enough, they have to motivate the participant to work as they would in a normal situation. I have seen and read results from tests where I could tell that the participants were just following the script – picking up a couple of words out of the scenario and looking for those in the interface, or worse, typing them straight into the search engines. These aren’t bad behaviours in themselves but in a usability test they indicate to me that the participant has not connected with the scenario enough for it to be representative of their normal situation. It is like painting by numbers. In order to replicate a real-life situation, the participant has to make a connection with the scenario and be motivated to complete it.
I’ve been doing a lot of reading into memory lately. One interesting aspect (amongst many) is that stories that contain content that triggers an emotional response, or that contain very vivid details, are remembered and recalled better than those that aren’t. Details are often not recalled well, but the essence of the situation is recalled well. This occurs even for stories that are unrelated to our personal experiences.
Linking the two
I’ve been leveraging this aspect of memory in usability tests recently. Instead of minimising the amount of information in a scenario, I’ve been enriching them with vivid detail and emotional aspects. I include a real names, products and places and describe a situation in a lot of detail. The scenarios may be long, but the richness of detail means that the test participant visualises and connects to the entire scenario. When they approach the task, they aren’t trying to recall the detail, they are feeling the situation. They remember the essence of the scenario and work through that rather than hunting for keywords to put into a search box. This means that they are working more realistically and we can put more trust in the outcome of the test.
This method works best for usability tests that contain a small number of scenarios. In order for it to work effectively, the participant must be given enough time to read slowly through the scenario and think on it a little before starting.
Try it. I promise you more realistic test results.