DonnaM » Blog Archive » Usability testing for findability

Usability testing for findability

A few days ago I wrote about exploratory information seeking and commented that I thought that the usability community hadn’t really come to grips with it.

I’ve been thinking more about this, and also about the differences between usability testing for informational and for interactive environments.

The ‘standard’ usability testing approach is to provide a set of scenarios to participants and ask them to work through the scenarios. The scenarios are usually in the form of ‘find something’. Even if they are written in a more descriptive form (such as a short story to provide some context), they are about finding information within a site. So, given that participants are told what they should be finding, the usability test is purely testing known-item information seeking. Even if the testing is on a site that has been designed to meet an exploratory need, this style of usability test leads only to testing known item. The very act of writing a scenario and asking someone to work it does this.

The other thing that these type of usability tests focus on is about ‘findability’. The finding of information is seen as the end of the process. Rarely have I heard about a test that explores whether people can understand the content once they have found it, whether they can use it to make good decisions or gain meaning from it.

I found this a bit scary in itself, realising that this type of usability testing is missing a major part of the information gathering journey.

But then I was also thinking about the various usability gurus, where they get their learnings from and how they create their guidelines etc. And I thought about a couple of gurus in particular, who many of you will know, but this is relevant to a reasonably broad group. I realised that most of the information that is published, and the guidelines that are created and the rhetoric that is spouted primarily comes from undertaking usability tests of sites. The same types of usability testing that I mentioned above – give someone a scenario, ask them to work it through.

So this scared me even more. A very, very large proportion of our body of knowledge about how people approach sites, and about how we should design sites, is based on a very narrow activity of looking for known information. And in most cases finding the information is seen as the end result.

Provided that I’m seeing the world as it really is, this has significant implications for our profession and the way that we design information environments. We may have already gone a significant way along the wrong path, where findability is king, where we spend more time on designing navigation than on designing answers and where we may be missing a major part of information seeking.

17 Responses to “Usability testing for findability”

  1. Dey Alexander Says:

    A few days ago, Donna? I think you’ve been working too hard!

    I couldn’t agree more about the usability/usefulness of content often being overlooked. But I think clients have to take some of the responsibility here. All too often the focus (and development/maintenance/testing budget) is on solely on UI design or technical issues.

    And it’s a common story to hear about staff being charged with writing and maintaining content, regardless of whether anyone’s checked to see whether they can write, or whether they know how to write for the web. It goes without saying that they don’t have the time to do this, let alone the time to have content reviewed/edited.

    However, I have to say that in my experience, testing an information site for known items isn’t always the standard fare, nor is focusing on whether the user found the information the extent of where the analysis lies. All of the decision points along the way, and the interaction elements that users encounter, are important to observe. But on this point, I’m probably reading your post too literally.

    We frequently ask people to find things they’re interested in, when we don’t know for sure whether or not this content exists (maybe we’re just horrible people?). This happens a lot when we test competitor sites, but I’m also interested in seeing if the site I’m testing gives enough clues early enough to stop people from wasting their time.

    And I recall a usability testing workshop with Rolf Molich a few years back, where he sort of advocated (in the context of writing hint-free task scenarios) asking users to find something that didn’t exist.

    We also ask users about the readability, usefulness and actionability of the information they have found. We’re lucky to have a few enlightened clients who care about such things.

    BTW, congrats to you and James on the new intranet design kit.

  2. Donna Maurer Says:

    Holy cow, it was longer than a few days ago!

    I’m not suggesting that everyone who usability tests only considers known item and findability, but certainly a lot of high-profile usability testing material is about this. And I do think that a lot of the guideline development/guru statements spring from this type of testing.

    I never ask people to do impossible tasks. By coming to the test and following my instructions, participants assume that the tasks can be done, and I have seen people go way beyond the ‘answer’in order to address an obscure point of the scenario.

    And congrats to you on your UX resources – that is an amazing set of stuff!

  3. Michael Says:

    The question really comes down to what you are building the system to do. If you are building an operating system, for example, you definitely need to test for any way people might possible want to manipulate the system. But if you are building a website to buy socks, you’ve hopefully thought beforehand about all the ways people might be able to find socks, and all the different shades of information people will need to make that decision (price, color, materials, workmanship, etc). You’ve got a business strategy. You’ve done focus groups. You’ve created user personas/scenarios.

    If you’ve done that work up front, you’ve already made certain decisions about what information is necessary to people to make their decisions. These questions aren’t ones of usability – which is about how usable the solution is. The questions you bring up are ones of strategy and user needs, and you shouldn’t start building without thinking about these things. They should inform every decision you make when building the system. If your usability testing has to test those basic assumptions, you’ve gone too far in the process.

    The way I’ve come to define it user testing/research is what you do at the beginning of the process to determine what the system should look like. Usability is simply testing whether the system you put together is the best way to facilitate the solution you’ve put together based on the initital user research.

    I think they are two very different things, but I definitely agree the web hasn’t really caught on to the best way of figuring out what users really want. While their doesn’t seem to be much research of this type for the web (what a surprise) I definitely feel like I’ve seen some. Plus, having just finished my Master’s degree, I can tell you there is tons of more generic information seeking behavior research in the LIS and HCI literature.

  4. donna maurer Says:

    Michael

    whether or not there is tons of stuff in the LIS literature is irrelevant. When someone is new to site creation and wants to know about how to make it good, they turn to the public web and find the usability gurus. They don’t turn to the LIS literature. All of that good stuff is completely hidden from the people who could benefit from it most, and the stuff that is easily available may have this enormous, known-item flaw.

  5. Alexander Says:

    I’m not sure it’s a flaw per se than say a somewhat focused approach. Often I think to myself; let’s solve one problem at a time, and one of the first big ones is indeed to make things findable. Once we all at least find *something* we can perhaps move on to make that information understandable and usable, and I think most gurus approach it this way even if they don’t explicitly say so. Maybe they should, and maybe that they don’t *is* the flaw. Or they’re stupid. One or the other.

    Anyway, I assume that as we find stuff more and more, the shift in focus for usability folks should indeed be “is it useful’, and Google keeps pushing that point for us. Usability folks need to re-focus to keep up with technology. :)

  6. Column Two Says:

    Usability testing for findability

    Donna has blogged an item on usability testing for findability. To quote: A very, very large proportion of our body of knowledge about how people approach sites, and about how we should design sites, is based on a very narrow…

  7. Michael Says:

    There is 50+ years of HCI literature about the way people interact with computers. There is 100+ years of cognitive psychology and information science literature about how people gather information for decision-making purposes.

    It may be true that people “new to site creation” may not have access these materials, or be able to fully understand them, but I thought we were talking about user experience professionals. A professional who chooses to ignore decades of research because it doesn’t apply directly to the web or is more difficult to find then jumping on Google doesn’t speak to a problem about lack of research. Rather, that is a problem with user experience designers failing to capitalize on a wealth of information that already exists.

  8. blog | bookslope Says:

    Findability

    「Usability」の前提になるのでしょうか。米国でこの手の情報を探していると「User Experience」と「Findability」という表現がしきりによく出てきます。 DonnaM: Usability testing for findability というこ…

  9. chris Says:

    Just a quick note, I couldnt read this article in IE6, SP2
    The text seemed cropped, in firefox it’s fine!

    I didn’t want to make this negative note public, but when I went to look for your contact details on this page….

    I couldn’t find them.

    ‘usability testing for findability’
    It’s unusable, you obviously didn’t test your site… and I can’t find what I need!

    Ironic don’t you think?

  10. chris Says:

    my bad, works fine at home, winxp sp2…

    doesn’t at work, winxp sp1

  11. Tony Says:

    I think the point here is that by talking about it online, we’re making this issue (and any potential solutions) more findable. You search, you should pull stuff up.

    Having 50-100 years worth of research in a library doesn’t do me any good if I can’t find it easy. I can’t go to the library in the middle of my work-day, and I need some questions/answers for a presentation.

    The fact that there is all this information already developed is great. The fact that this stuff has been studied before is fantastic. I’m not knocking that at all. We just need to pull/relate that information/studies/theories to what we’re doing here. And make it easy to find and use.

    Okay. Whew. Sorry for the long post. I’m done now.

  12. vanderwal Says:

    There is a huge problem of usability beyond just finding information. As more users are going mobile and have many devices they interact with moving information that is helpful from one device to another is problematic. There are many things that can be done early on to help the user in the IA framework to identify content objects and content types and from this point develop information delivery methods that ease the users use and reuse of the information. Thinking on about getting the information infront of the user fails the user in many ways.

    This is an area I have been working on for a couple years with the Model of Attraction and Personal InfoCloud frameworks. Much of the HCI work in this area is lacking as it focusses on one device and acquiring, using, and resuing information across devices. A person finds infomation they believe is valuable and want to keep that information close to themselves, in a Personal InfoCloud.

    One example is information in PDF, which is not a format that works well with out modification on mobile devices. The best practice when linking to a PDF is to let the user know the information is not an HTML file (mobile devices do not have status bars). Being careless with this linking practice can cause a mobile device to not access the page, or as I have seen many times in meetings the devices just lock-up. The framework for many UCD and IAs has been the desktop or laptop environment. But the rest of the world is moving beyond that, some parts of the world have most of the internet uses accessing information by something other than desktops and laptops.

    There are also information standard formats for some information types. Take address information, many sites still have this information in a graphic. This requires the person interrested in this information to write or type the information again rather than copy and paste (which will decrease error rates). Even better is using a stand such as a vCard to offer the information as well as offering the info in text/HTML. The vCard can be used on many devices and provides a consistant and accurate format for sharing and storing contact information. IA has failed to recognize this easy approach as IA has focussed on just getting the info infront of the user, but not how they user will use and reuse the information. There has not been a framework to work from that helps designers and developers think about this informaiton use and reuse properly.

    Findability is extremely important as it the rest of the IA framework, but where it stops helping is where many users need the help to use the information more easily and accurately.

  13. chase Says:

    If a website has a “rate this page” link on every page… then, I’m wondering how many people actually provide feedback through that method?

  14. Donna Maurer Says:

    Chris – thanks. That would be the IE invisible text bug (or whatever it’s called). I might not have fixed it in all of my templates.

    Thomas – I’ve been following your Model of attraction and personal infocloud thoughts. You’re way ahead of me thinking about this ;)

  15. IM @ XIST: The IM Blog Says:

    Usability testing for findability

    A posting on the difference between testing in informational and interactive environments. From DonnaM a new (for me) IA/usability blog….

  16. mediajunk Says:

    Survival Of The Easiest

    It’s nice, for a change, to hear Jakob Nielsen sounding positive. In an interview with CNN today, the man who…

  17. IM @ XIST: The IM Blog Says:

    Usability testing for findability

    This is a blog entry that never made the leap to XIST’s (no longer) new site. It is still worth a read. Here’s a posting entitled Usability testing for findability on the difference between testing in informational and interactive environments…