DonnaM » Blog Archive » Making decisions about user research

Making decisions about user research

Note: I’m going to polish this later, but wanted to get the basics down quickly.

We know that we should do user research for projects. All the user-centred design material says so, we talk about it at conferences, we put it in proposals. We just know that it is a good thing to do.

But when I talk to people about their actual projects, I find that very few people actually do user research. There are many many reasons (no time, no money, already know what users need etc etc etc).

I think that part of the reason it doesn’t happen is also that we don’t have good tools to tell us just how much research to do, and even when it isn’t necessary at all to do research.

In preparing for my Edge of the Web talk, I spent time thinking about that issue, reflecting on some of the projects I’ve worked on in the past and thinking about the factors that led me to push to do research, or to go ahead without.

The factors I came up with are:

  • Importance to the business: Just how important is the project/application in meeting organisational/business goals?
  • Importance to users: What will happen to users if you mess up. Will they be harmed, or will they just go elsewhere?
  • $$: How much is the project going to cost? (i.e. how much will be wasted if you mess up)
  • Profile/politics: What sort of profile does your project have? Is there a political implication? (e.g. is the Minister going to get hauled up in Parliament if you mess up. Will your work reflect badly on your industry?)
  • Convincing others: How much work will you need to do to convince other people that your ideas are good?
  • Existing knowledge: How much (real) knowledge do you have about your users?
  • Ability to iterate: Can you make changes quickly if you make a mistake, or is it a one-shot deal?
  • Feedback: How easy is it to collect feedback from your users?

Given each of these is a continuum, we can do this:

Each of the above factors, plotted with low and high ends

And then we can think about our projects, and plot where we fall on the dimensions…

Example 1: A personal blog

Example 2: A conference website

Example 3: An e-commerce shopping cart

Example 4: An enterprise-wide core business application

10 Responses to “Making decisions about user research”

  1. gordon Says:

    Hi Donna: I really like the tool you’ve started here. I think a key will be keeping the tool simple. Maybe just add some explanations and guidelines to quantify, and then leave the decision making up to the IA (or whoever is using the tool). Otherwise, if it gets at all cumbersome, people will just not do it, I suspect….


  2. Patrick Kennedy Says:

    Good thinking, I like this.

    One factor I’ve come up against lately is “how easily we can confabulate user needs”. It’s easy to dismiss research when you can just make stuff up and convince yourself that’s good practice by using different terminology (like giving your self-fulfilling-prophecy-use-cases the name “user stories” and “validating” them by getting the business to “sign-off on them”).

  3. Peter Van Dijck’s Guide to Ease » Blog Archive Says:

    [...] all say we should do lots of user research, and then nobody does. Why is that? Donna has some good thoughts. I do think there’s more to it than that though, it’s just one of those things [...]

  4. Steve Baty Says:


    I like this set of dimensions for modelling when to undertake user research. I think you need an additional dimension for internal — external for whether our users are from within the organization or outside. The further ‘away’ the users are from us, the more we need the research.


  5. Max Design - standards based web design, development and training » Some links for light reading (11/11/08) Says:

    [...] Making decisions about user research [...]

  6. Jen Says:

    Surely this needs to be in a honeycomb diagram? How can this be a valid model if it isn’t in a honeycomb diagram?

  7. Donna Spencer Says:

    @Gordon – thanks. I have no intention of ever making something more complex than it needs to be ;)

    @Pat – that is so dangerous, isn’t it. Looks like user research, smells like user research, but completely made up…

    @Steve – I think I’ll put the internal/external idea in Existing knowledge when I pad this out. Just because people are internal doesn’t mean you know them, and vice-versa.

    @Jen – damn, I knew I was missing something important!

  8. Web Design - Says:

    [...] Spencer has put together a handy tool and thought process for working out how much user research to do for a given [...]

  9. DonnaM » Blog Archive » Conference season Says:

    [...] Web standards groupĀ Canberra (4 Feb): Talking about user research, why we don’t do enough of it and my new user research decision diagram [...]

  10. 10-minute talks « UX Australia Conference 2009 Says:

    [...] year (at Edge of the Web) Donna Spencer presented her UX Equaliser as a conceptual tool. 10 months later from the mindmeld of the Perth UX community, after testing, [...]

Leave a Reply