DonnaM » Blog Archive » Regular folks searching

Regular folks searching

Last week I ran a usability test on a very neat system that allows people to search or browse a fairly large set of content. It’s been a while since I ran a usability test with regular folks (part of the hazard of working for too long on intranets, and one reason I’m glad I’m not specialising any more). The interesting thing was seeing whether people’s expectations of search had changed over the last few years (you know, the whole google phenomenon thing).

One of the things that came strongly out of this test is people’s mental model of search, and what they expect to happen. Here’s how I interpreted their expectations of search:

  • It is better to put more than one word in as one word gives too much stuff
  • Adding an extra word gives fewer results (although most search engines give more results with more than one word, people strongly thought it would give fewer – I even probed on this)
  • The first word in the search box is more important than the other words
  • If the words make a sensible phrase (one that humans would recognise as a phrase), the search engine should do so too and return results for the phrase
  • If the words do not make a sensible phrase, the search engine shouldn’t look for the phrase (yes, this contradicts the previous point, but no-one ever said people are logical)

How interesting!

8 Responses to “Regular folks searching”

  1. Lou Says:

    Sounds really interesting Donna. Do you think the issue of an extra term providing fewer results is due to the Google effect?

  2. donna Says:

    Yeah, I think the extra term/s is a result of getting so many results back in public search engines. It was interesting seeing that people were really thinking of the extra words as a ‘filter’. i.e. adding ‘Australia’ would *only* give Australian content…

  3. Ben Kraal Says:

    In my moments acting as a moderator on a discussion forum with a lot of “regular people” as users, I’d say that those findings are reasonable.

    People have been spoilt by Google or conditioned by Google to think that search works in a particular way.

    Often a user will say “I’ve searched for X, but I can’t find anything” or “I’ve searched for Y but I get too many results”. The problem is, lilke you say, they expect the search to be as smart as they are and know what they want (…because that’s how Google seems to work? I wonder how people think Google works?)

  4. bj Says:

    Very interesting indeed.

    I would think that people don’t necessary expect fewer results if they use more words in their search but better results further at the top of the result list. Like if I add the search term “Australia” to my search (which I actually do sometimes…), the total number of results might be higher than with just the single term, but to find what I’m looking for I’d only have to look at the first page. Now without the additional “Australia”, I probably would have to weed through a few result pages until I find what I was looking for (if I find it at all).

    So what I really want to say is, that I think people actually get it right if they think more words provide less results – after all, who really looks at all these 267,385 results anyway? ;) So it’s more like: more search terms provide less results that I have to weed through until I have found what I was looking for.

  5. donna Says:

    Bjorn – that’s the whole point of my post. They expected that more words would give fewer results. I observed, questioned and probed, and that is exactly what they expected to happen.

    Your explanation is the system model, mine is the user mental model. There is a mismatch, which was what is interesting.

    The system wasn’t a public search engine with millions of results and they knew it. It had a reasonable set of results and it would have been possible to check throught the entire result set. More words >> fewer results!

  6. bj Says:

    hmm, actually I tried to explain both the system and the user’s mental model…

    Anyway, the core message from me would be: I think the user’s are right – who cares for the system model? :)

    It’s still interesting though…

  7. Marian Steinbach Says:

    I think that it’s a good thing that users adapt to the Google model of search. The fact that entering more words lead to less results (or for the boolean folks: make “AND” the default operator) is the only simple way to make a result more specific that I can imagine.

    The opposite way, having more terms yield more result entries (OR being the default operator), makes it very difficult to get highly relevant results.

  8. Xavier Says:

    Hi donna, is very interesting your research, I am doing a research about the usability in forums, specifically in virtual learning enviroments, please if you can give me any help in my research would be great. Thanks.

Leave a Reply