Should you test in a lab or in the field?

I haven’t been in a usability test lab for about a year. Ironically, since I was writing a book about usability testing, much of my work was field research to learn about particular audiences and their tasks.

And, though my usual position about labs is that exploratory usability testing is probably better done in the user’s environment, I’m excited about getting back into the lab.

Good reasons to test in a lab
I’m doing these upcoming tests in a lab facility because

• The testing is quantitative and summative. That is, I’m doing very specific counts of errors and failures that are strictly defined, so I want to control other aspects of the test such as the computer setup.

• I don’t want to interact much with the participants. I only want to direct participants when to start their tasks. Otherwise, I will intervene in the session only at prescribed points, so I will direct the session from a different room from where the participants are working. 
• I may have observers, but I won’t know until the last minute. Though I prefer it if observers arrive before the session starts and stay through a whole session, at a facility they can come and go because they can observe from a separate room.

Good reasons to test in the field
I recently did a usability study in the field. Why?

• I wanted to learn about the user’s environment (rather than controlling it). In the exploratory study I’m thinking of, I got the best of both worlds: usability testing data in a realistic situation. I learned about lighting levels, surrounding noise, and what the participant’s desk setup was like. But I also got to observe relationships and interactions the participant had with others, typical interruptions (and recovery from those), and how the thing I was testing fit into the person’s work.

• It was convenient for the participants. They don’t have travel to the testing site. The interruption of their typical day is minimized. 
• The sessions were informal enough that observers could be present in the room (after they had been properly trained). In fact, people from neighboring cubes often chimed in comments or questions because they’d overheard what we were talking about. I took this to be a good thing because I learned about that communication dynamic, but those eavesdroppers often contributed information that was useful to me in my study.

In a future post, I’ll talk about what to look for in a lab facility if you’re renting one and how to find one.