5 Types of User Testing Participants
Every designer is constantly involved in user testing. Without user testing, we wouldn’t know if our design is easy to use and helps users to get their job done.
User testing is actually not the right term. It’s not about testing the user, but the usability of a product. That’s why usability testing is the originating term. However, user testing can reveal insights that go beyond usability, which is why I’d like to use this term.
Rule of 5
If you’re practicing user testing, you’re probably familiar with the rule of 5. It suggests to not test with more than 5 users, because after this point, the amount of usability issues found will not increase with the amount of users. Instead you’re observing repeatedly the same issues, thus testing with more than 5 users is a waste of time.
Clearly, there are situations when you want to test with more than 5 users, e.g. if your user group comprises different roles or if you’re looking to investigate the attractiveness of a design which tends to be very subjective. However, when you’re mainly concerned with usability, you should start testing with no more than 5 users, knowing that you can always test again with an improved version of your design.
Since I usually test with only 5 participants, I observed quite different reactions among the participants of the same test. However, having conducted user tests over the last 5 years, I saw the same psychological and behavioural pattens again and again, independent of the product or concept being tested.
This inspired me to come up with a typology of user testing participants.
Don’t take this differentiation too seriously. Everyone is individual and probably falls into more than one group. This typology is not meant to tell people apart, but rather to make us aware of the different personalities we encounter during user tests – and how to best deal with them.
5 Types of Participants
- The Tester
- The Passive
- The Clueless
- The Expert
- The Fan
The tester is the least tricky participant to conduct a user test with. She is familiar with user testing, knows the rules and is therefore easy to work with. But this makes her also a risky participant, because her awareness of the test scenario might cause her to act unnaturally.
For instance, she tends to spend a lot of time on each screen, carefully reading and assessing every bit of copy, although she probably wouldn’t do it in a real life scenario when she just wants to get something done.
She also might raise questions or make comments, not because this is what she is actually wondering, but because she thinks the interviewer might be interested in it.
- focused on the given task
- great at thinking aloud
- pays attention to every detail
- uses knowledge from other user tests
- tries to anticipate the hypotheses of the test
How to treat them
During the test, everything will go so smoothly, that you won’t have trouble with this kind of participant. Still you’re mainly looking for issues in your design, thus you don’t want too many of these experienced participants in the same test.
If you recruit from your user data base, try to avoid testing with the same people again and again by maintaining a list of users you tested with. If you recruit from a panel, tighten your screening criteria or ask the panel provider if they can prioritise testers who only test every few months.
This type of participant is particularly tricky because you always have to remind him that he can interact with the design and not just comment on it. His behavior can have two reasons: First, the participant is afraid of doing something wrong. Second the design doesn’t feel interactive, e.g. because it’s a low-fidelity paper prototype.
- listens carefully to the instructions
- doesn’t try to show off own expertise
- provides insights beyond the tasks being tested
- prefers talking over acting
- tends to go off script
How to treat them
First, you should prepare tasks the user is asked to accomplish during the test. Make sure you introduce these tasks in a way that makes clear that you expect the user to actually perform these tasks using the design. Instead of giving the user one very long task, rather split it into a couple of small ones to give the participant a chance to pause in-between.
Second, create a comfortable test environment. Make sure the participant can see the screen and has access to the keyboard or mouse. If testing on a mobile phone, let the participant hold it in his hands rather than placing it on a table. Even consider testing on the participant’s phone to avoid any technical barriers.
Last but not least, it never hurts to remind the participant to interact with the design. Repeat that you’re testing the design and not the ability of the participant to make him feel at ease. If the look and feel of the design is very low-fi, explain in your introduction that you’re testing it at an early stage and that you’re here to learn how the user interacts with it.
Similar to passive participants, the clueless participant needs a lot of help during the test. The difference is that she doesn’t have problems interacting with the design, but she doesn’t know what she is doing.
Often she rushes through the design, looking for the next best action to click. She often doesn’t even notice the tiny details you spent so much time designing. And if she does notice them, she doesn’t take the time to mention or comment them.
- focused on completing the given task
- not afraid of interacting with the design
- skips important information
- poor at thinking aloud
How to treat them
The fact that this type of participant seems to blindly go through the design is not necessarily bad. On the one hand, it’s a great opportunity to learn if you got the visual hierarchy right. On the other hand, you might still feel uncomfortable, because you want the user to not just use, but also understand your design.
To clarify the latter, include some probing questions at the end of your testing script. If you’re able to answer these questions during the test, just leave them out. If you’re still not sure, go through these questions after the participant completed the task. As always, make sure the questions are non-leading and open-ended. You could ask ‘Can you please describe the steps you just went through?’ or ‘What do you think happens next?’
This one is my least favourite. Due to their high self-esteem, it’s very hard to stop the expert once he’s in the flow of showcasing his knowledge. This type of participant easily goes off script and ignores the instructions given to him at the beginning. I had tests I was unable to conduct because the participant wasn’t willing to engage with the design at all.
The expert often doesn’t complete the task but stops in-between and thoroughly assesses every detail. As opposed to the The Tester, he steps out of the user role and conducts an expert analysis, comparing the design to other products and using commonsense knowledge about design and usability.
- great at communication
- shares knowledge from his field
- gives suggestions for improvement
- not focused on the given task or design
- sees the design through an expert lens
- points out solutions rather than pain points
How to treat them
First of all, you should make sure to exclude any design and product experts from your sample. This is best achieved by a screening survey sent to the participants during the recruiting process, in which you ask them basic demographic questions including their profession. If you recruit with the help of a research institute, they usually exclude industry experts by default, but it doesn’t hurt do double-check.
However, there is always the chance that you encounter an expert in your user test. When this happens, it’s best to treat him as such and learn as much of his knowledge as you can. Shift your user test towards an expert interview or peer review and talk about the design at eye level. Ask him directly about ways to improve it. If necessary, you can still use his answer to identify the problem behind the proposed solution.
A fan of your product, company, or brand is probably very positive and enthusiastic about anything you show to her. She usually asks a lot of questions which are not related to the user test, making it hard for you to stay on track. She doesn’t want to miss an opportunity telling you how great your company is and how much she knows about it. This is because she often brings in interests which have nothing to do with the test, e.g. getting insider information or a job.
- interested in the product
- tries to understand the whole picture
- easy to distract
- prefers talking about the company
- has an own agenda
How to treat them
You can filter fans out by avoiding incentives that are related to your product, e.g. a voucher or discount. Excluding experts from your field (see above) will prevent having potential job candidates in your user tests.
But overall, this kind off participant is fairly easy to deal with. Even though they like to chat about unrelated topics before and after the test, they are usually quite concentrated on the task during the test. Just don’t get too excited if this participant is very positive about your design. Pay even more attention to things they struggle with and dig deeper into those.
We’re all humans
Some unwanted variables can be easily avoided by screening participants before inviting them for a user test. However, you can’t avoid situations in which a tricky user is about to ruin your test.
So come prepared with a script that can be adapted to different kinds of personalities. But most importantly, just be human: Be empathetic with the person in front of you and don’t push them through your test at all costs.