Tips for Creating Effective Usability Tests

Tips for Creating Effective Usability Tests

Feb 15, 2016 by Lance Johnson | Filed under Design, Development

In usability testing, a product or service is evaluated by testing it on an end user. The advantage is obvious: it gives direct and specific input on how people use the product. Don’t let usability testing be mistaken for opinion gathering; that is market research. Instead usability testing measures the capacity of the product to fulfill its purpose. A number of products are evaluated through usability tests such as consumer products, foods, devices, computer interfaces and more. Web sites are no exception and should certainly be subject to usability testing prior to launch.

If you want to get accurate results, it is important to control the conditions under which, the tests are administered. Here are a few tips to create effective usability tests.

Test script should answer specific questions

When you begin a usability test for a website, do not assume that you just have to show specific areas of the website to the participants and then record what they do. You may get some insights by adopting that method, but when you report the results back to the project stakeholders what will you do if they hit you with questions that you do not have answers for? So always add specific questions to your test script.

It is a good idea to ask the project stakeholders what specific questions they want answered. If there are many questions, prioritize the ones you need answers for. Sometimes, a question may be vague; in this case you can ask the project stakeholders for clarifications.

Allow participants to behave naturally

When the test participants come to the usability tests, they may not know what to expect. They can get nervous about facing the camera and noticing that someone will be looking at them when they use the product or service. Do not be too controlling at this point because then the test participants will feel that they have to get permission from you before they do anything, which is not the objective of the test. You can get the participants to relax by allowing them to visit any page of the website they want, initially.

Allow the participants to approach tasks the way they want

Sometimes, participants may stray off the test script and begin to do other things on the website. At this point, you may be tempted to correct them and get them back on track. You may not realize it, but it will make you lose your rapport with the test participants. You will also lose a chance to see user behavior that you may have witnessed otherwise.

The best thing to do is to allow the users leeway, to stray off the test script and explore other areas of the website. If they persist for a long time and do not go back to the task, gently remind them of the test script. It is possible that you may feel that you are losing control of the test or that the participant has not understood the task, but resist doing it for some time. Pull the participant back to the test script only if you are sure that they are not going to come back to it.

Give up control for a while and see how the task unfolds


It is easy to be controlling, rigid and to focus on how you want the test to go. However, what if the user does something unexpected or interesting. At this point, you can ask users what they were thinking. However if you do this too early in the test or too late, you will miss out on the participant's natural behavior. Note that the more interruption there is, the less likely it is that the participant will complete the tasks without your aid. Do not ask them questions every 30 seconds because that will interrupt their flow and cause them to deviate from their natural behavior. You can always ask the participant what happened at a point, later.

The task should be specific to the participant

You cannot ask a sixteen year old to think like an experienced professional. So do not ask test participants to imagine scenarios. Many usability testers make the mistake of writing out exact scenarios for a test, but they soon find that the participants do not engage with them as much.

You can set an overall task for your participants, but try to keep it generic and tailor the scene as per the participant. Note that it is not possible always, but there is also value in spending time to find out who the participant is and what products or services he/she is currently using. If you can do this before the test, you will learn more from the participant than from someone who has to 'pretend' to be in the scene. Hand picking testers makes your testing more effective.

Include tasks on competitor or peer websites

Spending an entire hour on just one website may bore the participant. Moreover, all your observations and findings will be dependent on that isolated case. For example, you will not know if the person would actually go to the search box or just did it because he or she was confused by the outlay of your website. So just one website cannot give you a good picture of how your participants use the internet.

Include other websites (your competitors and peers) in the test. Do not tell them which of the websites you are testing, so that you can see their natural reactions. It may be difficult to sidestep, but if it is possible, do it. The reason is, it is very difficult for someone to be totally honest about how they would use a website when you tell them that you are testing it. Anyway, your test participants will deduce the website you are testing for, sooner or later because your test script will be built around the website. By the time they get this, you would have already collected a lot of interesting data.

The number one way to improve the reliability of your usability tests is to conduct more tests. Note that you should be careful about the test design and your interaction with participants because it can affect the research outcome.

Discuss this Article