IA - card sorting

I mentioned last week that there is a new card sorting tool available called OptimalSort.

I’ve set up a card sort and would love you to participate - give you a chance to play with the tool. The card sort is for the IA Summit and the content is conference presentations. I used this content to test other software tools last year.

If you can spare some time (15 minutes or so), please go here and do the card sort:

• IA Summit card sort using OptimalSort

The fine print:

My friends at Optimal Usability have released their brand-spanking new card sorting tool into beta release (it’s free until August).

Their tool, called OptimalSort, caters for open and closed card sorts. The sorting interface is a spatial drag-and-drop.

The thing I like best (and I will fully disclose that I was involved in this, so I should like it) is the analysis options. No stinkin dendrograms here, but lists of category names, detailed participant results and lots of ways to explore the data.

Go try OptimalSort.

I have just uploaded something I’ve been working on for many years: the spreadsheet I use to analyse card sort data. My spreadsheet is great for analysing open sorts - it manages participant results, helps you explore the data and does some basic statistical analysis.

It’s free. Check it out!

Recently I talked with Gerry Gaffney, from Information & Design & UXPod about card sorting. The interview is now ready and I am pleased with it - it was fun to do and fun to listen to.

There are a lot of other great interviews & discussions about UX topics, so go listen to those as well.

I’m very excited to let you know that, between now and the end of the year, I will be writing a book on card sorting. Specifically, it will be about how to use card sorting in information architecture and similar projects. I think it will fill an important gap – there is some material about how to run a card sort, but little on how to actually use the outputs.

My book will be one of the first published by Rosenfeld Media, a new publishing house focused on producing short and practical books on user experience design. RM will be applying UX methods to determine what books to publish and how its books should be designed. RM will also be incorporating users’ input as much as possible throughout the writing process, primarily via a blog and dedicated site for each book. You can find mine at the Card sorting book website and monitor it via an RSS feed:

I’d love to hear from you about your experiences conducting card sorts and have set up a survey to help collect this. It is short – 10 questions – and should take no more than 10 minutes:

Card sorting - your experiences (survey)

I’m really looking forward to working on the book, and hope you’ll consider participating by sharing your suggestions. Send comments and ideas via the site or to me at (cardsorting -at- maadmob -dot- net).

For years (literally) I have been trying to figure out what closed card sorting is good for, and today finally had an idea.

Now, the common theory is that a closed card sort is valuable for evaluating a classification or the results from an open sort. Using this approach, you give participants a set of cards with content and ask them where, from a set of pre-selected headings, they would place that content. The idea, though not always explained, is that this corresponds to where people would look for the content.

No! This is completely invalid. Categorising information and finding it are two entirely different tasks, with entirely different cognitive processes. The only way to test whether a classification will allow people to find information, is to ask them to find information (or at least ask where they would look). You don’t learn it by asking them to place information in the classification.

But, if you are testing a classification where the key task is to categorise content, closed card sorting is completely appropriate. This may be a great way to learn where intranet or website authors would place their page and what metadata they’d choose; or what categories people select when entering data into a business application. Give them some content examples, and ask what categories it belongs in. Then separately test whether users can find that same information.

And if you have an idea for a valid use of this technique, please let me know - I am interested in discovering good uses.

Following yesterday’s post about IBM’s card sorting tools being removed from the site, I recieved the following answer from IBM:

“The EZSort and EZCalc applications have been archived. The applications are no longer supported and will no longer be developed. Therefore, the applications are no longer available to the general public. Our apologies for any inconvenience.

Thank you for your interest in ease of use….”

Looks like that’s it then!

Look, a new card sorting tool - Card Sword.

It’s an 0.1 alpha and doesn’t work for me. Will let you know when I have anything to report.

A reader pointed out that the link I provided to IBM’s USort/EZCalc was not working. I investigated and found out that the pages are no longer on the IBM Ease of Use site, and have been archived!

I have contacted them to find out what has happened, and will let you know.

I posted a short summary of card sorting tools last week. At the time, I had looked at all of the tools and entered cards into the two that I thought would allow me to go further. I didn’t intend on getting real users to use the tools - I ran a sort as part of a workshop and entered the outcomes into the tools myself.

I used two tools - IBM’s USort/EzCalc and CardZort. A summary of how they went for results entry and analysis:

USort

• USort was as annoying as it always was (well, it hasn’t changed since I first used it years ago, so it isn’t any less annoying). I didn’t read the instructions properly first time and forgot to ‘promote’ all of my groups to top level categories, meaning that I had to start over.
• The sorting involves dragging list items from one side of the screen to the other, making sure that they end up in the right group (this is buggy).
• The program forces a very step-by-step approach (sort, group into higher groups, label) and there is no ability to go backwards
• The cluster diagram is OK, with colour used to separate the main clusters. The only way to look at the labels is to look at the individual results

CardZort

• CardZort is nice. As I mentioned before, there is a label limit of 20 characters, making it impractical for my use at the moment, but that is the primary limitation
• The sorting method is spatial - the cards look like cards and can be dragged around the screen, then piled on top of one another. I do like this - it feels nice and natural to me.
• The groups can be labelled at any point, re-labelled and the groups can be changed. Almost as nice as working with paper, provided you have a good screen and high resolution
• The cluster analysis is similar to that from EZSort. It doesn’t colour code the main groups, but does allow you to see what labels have been applied for each group, which is good.

Other

I also had a chat with relevant people about two of the other tools:

• uzCardSort is currently not being developed further as the main developer is unable. It is available for further open-source development. If anyone is interested in picking this up, I’m willing to help out on functionality, interaction and interface design.
• In WebSort, you don’t have to name the categories before you create groups, but it gets very difficult to manage a bunch of unnamed categories that keep collapsing. You’ll probably have to look at it to see what I mean. Otherwise, it is very nice - attractive, easy to use and runs across the web.

I was preparing a bundle of cards for a workshop today, and reminded by the recent discussion on a discussion list and peterme about cluster analysis, thought I’d have another look at computer-based card sorting tools. I was vaguely thinking of dropping the results into a couple of tools, comparing them and seeing if my reservations about cluster analysis were still valid.

So here’s a summary of the current tools in no particuar order. I have put card details into those that work, but haven’t put any results in yet. Will report on that in a few days:

• WebCAT - Online tool. I can’t follow the install instructions, so have never tried it. Oops!
• uzCardSort - a Mozilla extension. Needs a lot more work, clunky interface and somewhat buggy.
• Card Zort - Windows application. Card titles display as 20 characters maximum (but can be stored as longer), which isn’t terribly useful. I got around it by putting some stuff in the ‘description’ field, but this isn’t sustainable for a real-life situation. Spatial sorting metaphor - dropping cards into piles seems more sensible than moving text around. It has its own clustering tool. Some potential, pending results and analysis.
• WebSort - this is a commercial service, runs online and has an online demonstration. It is pretty easy to set up a sort, but the model is backwards - you have to create categories then add content to them. It might have had some promise otherwise.
• CardSort - Windows application. Card titles are limited to 18 characters. Not useful at all.
• IBM’s EZSort and EZCalc - Windows applications. Easy to enter cards into, slightly clunky sorting interface (but it at least groups then labels), cluster analysis tool available. Reported to flunk out with over 100 cards.
• Classified - Windows application. It uses categories and content and participants have to determine which category the content goes into. If the participant doesn’t select the ‘correct’ category, they get an ‘incorrect, please try again’ error message. For this reason alone, I wouldn’t use it.

Bit of a disaster all ’round, methinks!

[I have written a follow up to this - Card sorting tools - final summary and also fixed some mistakes in this post. Thanks for the feedback and questions]

You could hardly expect me not to comment on Jakob Nielsen’s alertbox on card sorting. I get the feeling that people expect me to disagree just on principle
There is some good stuff in here - the same points that I return to frequently:

• card sorting is a generative method, not an evaluation method
• much of the value comes from listening to people’s comments
• it is important not to design an information architecture on similarity scores

Also good is that he points out that the findings were based on one research study. I have done card sorts on enough information domains to know that some domains cluster easily, some don’t. I can think of a couple of homogeneous domains where high correlation would be possible with many fewer participants.

Actually, there are only a couple of things I disagree with:

• a card sort is not a ‘test’ so the language of ‘testing’ users confuses the issues of generative and evaluative activities
• I truly don’t believe that users have a mental model of an information space. A card sort doesn’t elicit this model - it just gets some ideas of what things people think are similar.
• I can’t tell whether the number of users is in reference to individual sorts or group sorts. I expect it must refer to individual sorts (so you are after 15 sets of results, not 15 sorts). However, if this is the case, then the reference to ‘listening to users comments’ is irrelevant as people don’t tend to talk to themselves when they sort!

If you would like to know more, read my definitive guide to card sorting over on Boxes and Arrows or have a look at previous musings in my card sorting category.

Yay! the card sorting article that I wrote with Todd Warfel has been published on Boxes and Arrows.

So, someone’s sure to ask why I, after carrying on about card sorting here and on mailing lists, would write an article about it. I don’t think it’s a bad technique, but am concerned when I see people using it as their main technique to ‘create a new IA’. Two main reasons:

1. it’s too content-centric, which may result in an IA not suited when users attempt real tasks
2. no technique should ever be used in isolation

Anyway, go read it, and leave your comments and variations over on Boxes and Arrows…

I’ve written before about how I have reservations about how people use card sorting as a technique to develop an IA, but I’ve always talked about open sorting. Today I had a great experience that shows how useless closed card sorting is.

Just for background, in a closed sort, participants are asked to place content into pre-defined categories. Some people use it as a further exploratory method after an open sort, some as an evaluation method (can you tell that I use neither
Today I was working with a team, exploring a new IA for a site and discussing where we would put various pieces of content. We’d talk about some content and one of us would suggest a place where it would fit (like you’d do in a closed sort). I’d turn it around and describe a scenario with someone looking for the content as part of a task.

So many times, after thinking about the scenario, we’d all say “no, I wouldn’t look there, I’d look in xx”.

I wish I could give you examples to clarify this, but I can’t. Believe me, it is amazing the amount of difference there is approaching content placement from a task-based perspective, rather than a content-based perspective. Try it!

I recently came to the conclusion that I don’t like card sorting as a technique for determining an information structure. I’m only now starting to realise why.

In my last few big card sorts, I have noticed that participants don’t really look at the cards and try to form sensible groups based on how the information should be grouped to help them achieve a task. Instead, they try to get rid of most of the cards as quickly as possible. Usually this involves looking at a card, thinking that it is not something they are interested in and trying to group it with similar things they aren’t interested in. If they find something they are interested in, it is usually because it is in their own information domain.

What this results in is a bunch of cards grouped by surface characteristics. Surface groupings I have recently seen include:

• big corporate documents
• newsletters
• articles
• pages that belong to ‘xxx’ team

This doesn’t help. There is no ‘aboutness’ in these groups. This doesn’t give me an insight into why content should go together. How is putting all of the newsletters together going to achieve someone’s goal.

I need deeper thought and groupings on deeper characteristics - not just surface characteristics and “may I go home now”.