DonnaM » Blog Archive » Card sorting on the surface

Card sorting on the surface

I recently came to the conclusion that I don’t like card sorting as a technique for determining an information structure. I’m only now starting to realise why.

In my last few big card sorts, I have noticed that participants don’t really look at the cards and try to form sensible groups based on how the information should be grouped to help them achieve a task. Instead, they try to get rid of most of the cards as quickly as possible. Usually this involves looking at a card, thinking that it is not something they are interested in and trying to group it with similar things they aren’t interested in. If they find something they are interested in, it is usually because it is in their own information domain.

What this results in is a bunch of cards grouped by surface characteristics. Surface groupings I have recently seen include:

  • big corporate documents
  • newsletters
  • articles
  • pages that belong to ‘xxx’ team

This doesn’t help. There is no ‘aboutness’ in these groups. This doesn’t give me an insight into why content should go together. How is putting all of the newsletters together going to achieve someone’s goal.

I need deeper thought and groupings on deeper characteristics – not just surface characteristics and “may I go home now”.

6 Responses to “Card sorting on the surface”

  1. eric scheid Says:

    Maybe combine it with a heads-up that they’ll be tested on their groupings with some goal-directed scenarios. Effective groupings get rewarded with bonus incentives.

    Give a couple of scenarios as examples, hint strongly that the actual scenarios may differ widely in goal/purpose/audience (so as to avoid them “cheating” by grouping to suit the limited goals suggested). Design the scenarios such that they require bits of information from many different groupings.

    Just a thought.

  2. Melanie Kendell Says:

    I’m presuming the people doing the card sort are representative of your audience and the purpose is to elicit what is useful to them rather than getting them to think of what might be useful to themself and others.

    How about doing a two stage process:

    1. Pick out all the things that you would want to look at.

    2. Now sort them into groupings.

    That way they can legitimately discard all the stuff they don’t care about and you get a real feel for how *they* want to access *their* information.

    If you get representatives from several areas you should find you cover a fair percentage of the material and then it is up to you to see the patterns in the various groupings.

    It might also highlight stuff that really shouldn’t be on the site at all.

  3. Donna Maurer Says:

    Good tips – I have a whole bunch of things I want to try out over the next few projects.

  4. James Bullock Says:

    You might try throwing them into scenario / narrative space with the cards in hand. “So you decide you need to WHATEVER, then you what with these cards?” I suspect you’ll find patterns where the same cards are often associated together as people work.

    I suspect that most information rich processes benefit from organizing information these two ways:

    - Information that provides context for a process step, where the goal is performing work,
    - “Types” of information, that are in some sense similar, where the goal is selecting or rejecting some candidate object, or maybe filling in a missing bit of context.

    Given that, the other thing to jiggle the card sorters a bit is have them do some search / lookup kinds of things.

    You might end up with some different “same-ness” groupings depending on people’s jobs. For the guy who runs the printer, the big corporate docs that jam the printers are “the same” in a meaningful way. I suspect that the strange sortings you mention may have more to do with an underlying idea about how hard it is, or isn’t to work with the items on some cards, than with the meaning of the stuff on the cards.

    Perhaps either processes or searches with the grouped cards could be a goal for the card sorting exercise. Once a sort has been achieved, walk them through some processes and searches. “Good-ness” goes to card groupings that make sense in the process walks, and searches.

    Would you consider posting the results of whatever you try?

    Thanks for posting this.

    - Jim

  5. Peter Says:

    Great comments guys. Peter FDA

  6. Daniel Engelberg Says:

    I have tried card sorting once using EZCalc. Interpreting the results was a nightmare. And that’s the best software that exists right now for analysis. There were also unreasonable limitations on hierarchy depth for data entry, but that might be specific to EZSort.

    Supporting some of Kevin’s comments, there is a wide body of psychology research that concludes that users do not have reliable access to their own expert knowledge. So I am suspicious about the validity of card-sorting tests.

    Furthermore, it is becoming increasingly clear to me that this technique is poorly adapted to sites with large quantities of categories.

    I have decided that in the future, to avoid the problems associated with analysing data, I will either get all my users to collaborate on the exercise in the same room, or I will skip the step altogether and instead perform testing on a reasoned hypothesis. The collaborative approach will at least ensure consensus and avoid analysis headaches.