Supporting some of Kevin’s comments, there is a wide body of psychology research that concludes that users do not have reliable access to their own expert knowledge. So I am suspicious about the validity of card-sorting tests.

Furthermore, it is becoming increasingly clear to me that this technique is poorly adapted to sites with large quantities of categories.

I have decided that in the future, to avoid the problems associated with analysing data, I will either get all my users to collaborate on the exercise in the same room, or I will skip the step altogether and instead perform testing on a reasoned hypothesis. The collaborative approach will at least ensure consensus and avoid analysis headaches.

I suspect that most information rich processes benefit from organizing information these two ways:

- Information that provides context for a process step, where the goal is performing work,
- “Types” of information, that are in some sense similar, where the goal is selecting or rejecting some candidate object, or maybe filling in a missing bit of context.

Given that, the other thing to jiggle the card sorters a bit is have them do some search / lookup kinds of things.

You might end up with some different “same-ness” groupings depending on people’s jobs. For the guy who runs the printer, the big corporate docs that jam the printers are “the same” in a meaningful way. I suspect that the strange sortings you mention may have more to do with an underlying idea about how hard it is, or isn’t to work with the items on some cards, than with the meaning of the stuff on the cards.

Perhaps either processes or searches with the grouped cards could be a goal for the card sorting exercise. Once a sort has been achieved, walk them through some processes and searches. “Good-ness” goes to card groupings that make sense in the process walks, and searches.

Would you consider posting the results of whatever you try?

Thanks for posting this.

- Jim

How about doing a two stage process:

1. Pick out all the things that you would want to look at.

2. Now sort them into groupings.

That way they can legitimately discard all the stuff they don’t care about and you get a real feel for how *they* want to access *their* information.

If you get representatives from several areas you should find you cover a fair percentage of the material and then it is up to you to see the patterns in the various groupings.

It might also highlight stuff that really shouldn’t be on the site at all.

Give a couple of scenarios as examples, hint strongly that the actual scenarios may differ widely in goal/purpose/audience (so as to avoid them “cheating” by grouping to suit the limited goals suggested). Design the scenarios such that they require bits of information from many different groupings.

Just a thought.