The true costs of no-shows

One of the first things people say when they call up looking for help with recruiting is that they want to recruit “12 for 8” or “20 for 15”. They know what they want to end up with. They’ve got to get data. Managers are showing up to observe. They’ve gone through a lot to get a study to happen at all. They don’t want to risk putting a study together only to get less data than they need. So, compensating for a show rate of between 60% and 80% means over-recruiting.

Even though a recruiting agency probably won’t charge for no-shows, those no-shows can be costly in lots of ways.


Stress. First, there’s the anguish of the moments before you know whether the participant will show. It’s not only the researcher who’s nervous. Nobody knows what to do. Time is ticking away. The session is running late late. Observers get antsy. The careful planning and scheduling of putting on a study feels like it is falling apart. That can be agonizing. When the participant arrives, it’s difficult to center yourself and get on with the session.

Embarrassment. There is nothing like the humiliation of having participants not show up after nudging, cajoling, and begging a team to show up to observe their users. The campaign of marketing user research disintegrates as the promise of enlightenment fades. It’s like promising fireworks but the fuses won’t light. The negative energy can be palpable and extremely difficult to recover from. Claiming that this is out of you control will not help the credibility of the study.

Loss of productivity. When you find out there is a longer-than-planned gap between sessions, conducting shadow puppet shows or showing cat videos to fill the time will not win the day. The researcher has got to get the team engaged in something productive that will help everyone feel like they’re still getting value out of being there. Without a participant to observe, it’s tough to keep the team from thinking you’re not wasting their time. That’s difficult to pull off for even the most experienced of researchers.

Loss of trust. Let’s not forget the pain of losing momentum as observations are interrupted. Say you got a few sessions done. Everyone is charged up with the revelations they’ve witnessed. Epiphany is everywhere. And then, a no-show. It can suck the air out of the room. Clients and teams can become disenchanted and impatient. They may lose confidence in the process and the results.

By comparison, the amount of money going for lab time when you’re not running sessions — or paying for even more lab time and recruiting to make up sessions — seems small. The damage of no-shows is long lasting. Yet, most of us treat no-shows as a cost of doing studies. They just happen. It’s the recruiting agency’s fault. We can’t control the participants.

What if you had a near-perfect participant show rate? The first time it happens, it’s surprising. The next few times, it’s refreshing — a relief. Teams that do great user research start with the recruiting process, and they come to expect near perfect attendance. In the next post, I’ll share their secrets for getting great participants who show up.