The beginner’s guide to usability testing [+ Sample Questions]

In virtually any discipline, letting others evaluate your work with fresh eyes is a good idea, and this is especially true when it comes to user experience and web design. Otherwise, your preference for your own work may distort your perception of it. Learning directly from the people you work for – your users – can help you create the best possible user experience.

UX and design professionals use usability tests to get user feedback on the user experience of their product or website at any time. In this post you will learn:

What is usability test?

What are usability tests?

Usability testing is a method of assessing the user experience of a product or website. By testing the usability of their product or website with a representative group of their users or customers, UX researchers can determine whether their actual users can use their product or website easily and intuitively.

UX researchers typically conduct usability studies on every iteration of their product from early development to release.

During a usability study, the moderator asks participants in their individual user session to complete a series of tasks while the rest of the team observes and takes notes. By watching their actual users navigate through their product or website and listening to their praises and concerns, they can see when attendees can quickly and successfully complete tasks, where they enjoy the user experience, run into problems, and create confusion.

After the study has been carried out, they analyze the results and report interesting findings to the project manager.

What is the purpose of usability testing?

Usability testing allows researchers to uncover problems with the user experience of their product, decide how to fix those problems, and ultimately determine whether the product is sufficiently usable.

Identifying and fixing these early issues saves the company both time and money: developers will not have to rewrite the code of a poorly designed product that has already been built, and the product team will be more likely to release it on schedule.

Advantages of usability tests

Usability tests have five major advantages over other methods of examining the user experience of a product (e.g. questionnaires or surveys):

  • Usability testing provides an unbiased, accurate, and direct examination of the user experience of your product or website. By testing the usability on a sample of actual users, disconnected from your team’s emotional investment in creating and designing the product or website, their feedback can resolve most of your team’s internal debates.
  • Usability tests are useful. To do your studies, all you need to do is find a quiet room and bring a portable recording device. If you don’t have a recording device, someone on your team can just take notes.
  • Usability tests can tell you what your users are doing on your website or product and why they are doing those actions.
  • Usability testing can help you fix the issues with your product or website before you spend a ton of money creating something that ends up badly designed.
  • For your business, intuitive design increases customer usage and results, and increases demand for your product.

Examples of usability test scenarios

Usability testing sounds good in theory, but what is its value in practice? Here’s what it can do to actually make a difference to your product:

1. Identify friction points in the usability of your product.

As Brian Halligan said at INBOUND 2019, “Dollars flow where there is little friction.” This applies to UX as well as to sales or customer service. The more friction your product has, the more reasons your users will have to find something that is easier to use.

Usability tests can uncover points of friction from customer feedback.

Example: “My process starts in Google Drive. I’m constantly switching between windows and making several clicks just to copy and paste them from Drive into this user interface. “

While the product team may have had this task in mind when developing the tool, seeing a use case the tool couldn’t compensate for when they saw it in action and heard the frustration of the user. This could lead the team to resolve this issue by creating a simple import feature or a way to access Drive through the UI to reduce the number of clicks the user has to make to complete their task.

2. Stress test in many environments and use cases.

Our products do not exist in a vacuum, and sometimes development environments cannot compensate for all variables. Bringing the product out and testing it by users can reveal bugs that you may not have noticed during internal testing.

Example: “The checkboxes disappear when I click on them.”

Let’s say the team is investigating why this might be the case and finds that the user is using a browser that is not used often (or an outdated version of the browser).

If the developers only tested the browsers used internally, they may have missed this bug and it could have caused frustration among customers.

3. Offer different perspectives from your user base.

While the individuals in our customer base have a lot in common (especially the things that made them need and use our products), each individual is unique and brings a different perspective. These perspectives are invaluable in uncovering issues that your team may not have noticed.

For example, “I can’t find where to click.”

Upon further investigation, it is possible that this feedback was from a color-blind user, causing your team to realize that the color choices didn’t create enough contrast for that user to navigate properly.

Insights from different perspectives can lead to improvements in design, architecture, text and accessibility.

4. Give clear insights into the strengths and weaknesses of your product.

You likely have competitors in your industry whose products are better than yours in some areas and worse than yours in others. These market fluctuations lead to competitive differences and opportunities. User feedback can help you bridge the gap on critical issues and find out which positioning works.

For example: “This interface is so much easier to use and more attractive than [competitor product]. I just wish I could do it too [task] in order to.”

Based on this feedback, two scenarios are possible:

  1. Your product can already do the job requested by the user. All you have to do is make it clear that the feature is there by improving the copy or navigation.
  2. You have a really good opportunity to incorporate such a feature into future iterations of the product.

5. Inspire you with possible future additions or improvements.

Speaking of future iterations, the next example of how usability testing can make a difference for your product: The feedback you gather can inspire future improvements to your tool.

It’s not just about rooting out problems, it’s also about imagining where you can go next to make the most difference for your customers. And who is the best one to ask other than your potential and existing customers themselves?

Examples of usability tests and case studies

Now that you have an idea of ​​the scenarios in which usability tests can be helpful, here are some practical examples:

1. User fountain + satchel

Satchel is an educational software developer and their goal was to improve the website experience for their users. Consulting agency User Fountain conducted a usability test that focused on one question: “If you are interested in the Satchel product, how would you go to get more information about the product and its prices?”

During the test, User Fountain experienced significant frustration when users tried to get the job done, especially when it came to finding pricing information. Only 80% of the users were successful.

Usability test example: user fountain + satchel

Image source

As a result, User Fountain hypothesized that a “Get Pricing” link would make the process clearer to users. From there, they tested a new variant with such a link against a control version. The variant won, leading to a 34% increase in demo requests.

By testing a hypothesis based on real feedback, friction losses were eliminated for the user, which brought real added value to Satchel.

2. Kylie.Design + Digi-Key

E-commerce site Digi-Key reached out to consultant Kylie.Design to find out which site interactions had the highest success rates and what similarities those interactions had in common.

They performed more than 120 tests and recorded:

  • Click paths from each user
  • Which actions were the most common
  • The success rates for everyone

Usability test example: Kylie.Design + Digi-Key

Image source

This as well as the written and oral feedback from the participants were incorporated into the new design, which led to an increase in the buyer success rate from 68.2% to 83.3%.

In essence, Digi-Key was able to identify their most successful features and duplicate them to improve the experience and bottom line.

3. Sparkbox + an academic medical center

An academic medical center in the Midwest partnered with consulting agency Sparkbox to improve the patient experience on their home page, where some features suffered from low engagement.

Sparkbox conducted a usability study to find out what users expect from the homepage and what doesn’t meet their expectations. From there, they could suggest solutions to increase engagement.

Usability test example: Sparkbox + Medical Center

Image source

One important measure, for example, was the ability to access electronic medical records. The new design, based on user feedback, increased the success rate from 45% to 94%.

This is a great example of how to put the user’s pain and desires at the center of a design.

The 9 phases of a usability study

1. Decide which part of your product or website you want to test.

Do you have urgent questions about how your users interact with certain parts of your design, such as: B. a certain interaction or a certain workflow? Or are you wondering what the first thing users will do when they land on your product page? Gather your thoughts on the pros, cons, and areas for improvement of your product or website so you can build a solid hypothesis for your study.

2. Select the tasks for your study.

Your attendees’ tasks should be the most common goals your users make when they interact with your product or website; B. Make a purchase.

3. Set a standard for success.

Once you know what to test and how to test it, make sure you set clear criteria to determine success for each task. For example, when I was doing a usability study for HubSpot’s content strategy tool, I had to add a blog post to a cluster and report exactly what I did. By setting a threshold of success and failure for each task, you can determine whether or not the user experience of your product is intuitive enough.

4. Write a study plan and script.

At the beginning of your script you should include the purpose of the study, if you want to include background information about the product or website, questions about the participants’ current knowledge of the product or website, and finally their roles. To keep your study consistent, unbiased, and scientific, moderators should follow the same script in every user session.

5. Delegate roles.

During your usability study, the moderator must remain neutral and carefully guide the participants through the tasks and strictly adhere to the script. The one on your team who is best at staying neutral, not indulging in social pressure, and making participants feel comfortable while getting them to get the tasks done should be your facilitator

It is equally important to take notes while studying. If there is no recorded data, there will be no evidence to prove or disprove your hypothesis. The most attentive listener on your team should take your notes as you study.

6. Find your attendees.

Screening and recruiting the right participants is the hardest part of the usability test. Most usability experts suggest that you should only test five participants during each study, but your participants should also be very similar to your actual user base. With such a small sample size, it’s difficult to replicate your actual user base in your study.

To recruit the ideal participants for your study, create the most detailed and specific persona possible and motivate them to participate with a gift certificate or other monetary reward.

Recruiting colleagues from other departments who would potentially use your product is another option. However, you don’t want any of your team members to know the participants as their personal relationship can create bias – since they want to be nice to each other, the researcher can help a user complete a task or the user may not want the product design criticize the researcher constructively.

7. Perform the study.

During the actual study, you should ask your participants to do one task at a time without your help or guidance. If the participant asks you how to do something, don’t say anything. You want to see how long it takes users to understand your user interface.

Asking attendees to think out loud is also an effective tactic – you know what goes through a user when they are interacting with your product or website.

After completing each task, ask for their feedback, such as: For example, whether they expected to see what they just saw, whether they would have completed the task if it weren’t for a test, whether they would recommend your product to a friend and what they would change with it. This qualitative data can reveal other advantages and disadvantages of your design.

8. Analyze your data.

You will collect a lot of qualitative data after your study. The analysis helps you to identify problem patterns, assess the severity of each usability problem and give the engineering team design recommendations.

As you analyze your data, pay attention to both user performance and how they feel about the product. It is not uncommon for a participant to achieve their goal quickly and successfully, but still have a negative product experience.

9. Report your results.

After you have gained insights from your data, report on the key insights and determine the next steps to improve the design of your product or website, as well as the improvements you can expect in the next round of testing.

The 3 most common types of usability tests

1. Hallway / guerrilla usability tests

Here you set up your study somewhere with a lot of pedestrian traffic. It allows you to randomly ask people who most likely have never heard of your product or website, such as passers-by, to rate the user experience.

2. Remote / unmoderated usability tests

Remote / unmoderated usability testing has two main advantages: You use third-party software to recruit target participants for your study, so you can spend less time recruiting and more time researching. It also allows your participants to interact with your user interface themselves and in their natural environment.

Letting participants interact with your design in their natural environment without anyone sitting on their backs allows you to get more realistic, objective feedback. Being in the same room with your attendees can make them put more effort into getting your assignments done, as they don’t want to appear incompetent around an expert. Their perceived expertise can also lead you to like them instead of being honest when asking for their opinion, and skew the reactions and feedback of your user experience.

3. Moderated usability tests

Moderated usability tests also have two main advantages: interacting with participants in person or through a video or phone call, you can ask them to explain their comments if you don’t understand them, which is impossible in an unmoderated usability study. You can also help your users understand the task and keep them updated if your instructions are not initially registered with them.

Script and questions about the usability test

It wouldn’t make sense to follow a script or even a template of questions for each of your usability studies – the subject of each study is different. You need to tailor your questions to the things you want to learn, but most of all you need to know how to ask good questions.

1. If you [action]what do you do first [goal]?

Questions like these shed light on how users interact with the tool and what their natural behavior is.

Julie Fischer, a senior UX researcher at HubSpot, offers this advice: “Don’t ask key questions that bring your own bias or opinion into the minds of participants. They will end up doing what you ask them to do, rather than what they do. “Would do it by itself.”

Example: “Find [x]”is better than” can you easily find? [x]? “The latter inserts a connotation that may affect how the product is used or how the question is answered.

2. How satisfied are you with the [attribute] from [feature]?

Avoid asking participants like “Is this feature too complicated?” respectively. Instead, measure their satisfaction on a Likert scale, which ranges from very dissatisfied to very satisfied. This will result in a less biased outcome than if they lead to a negative answer that they might not otherwise have received.

3. How do you use [feature]?

There can be several ways to achieve the same goal or use the same function. This question helps to find out how users interact with a certain aspect of the product and what they find valuable.

4. What parts of [the product] do you use the most? Why?

This question is intended to help you understand the strengths of the product and what makes enthusiastic fans about it. This indicates what is important to keep, and may even provide insight into what you can improve on for other functions.

5. What parts of [the product] do you use the least? Why?

This question aims to reveal the weaknesses of the product or the friction in its application. That way, you can fix any issues or plan future improvements to bridge the gap between user expectations and reality.

6. If you could change something [feature] what would it be?

Since it’s so similar to # 5, you might get some of the same answers. However, you’d be surprised at the ambitious things your users could say here.

7. What do you expect? [action/feature] make?

Here is another tip from Julie Fischer:

“When participants ask, ‘What will this do?’ The best answer is ‘What do you expect from it?’ instead of telling them the answer. ”

This can reveal user expectations and clarity issues with the copy.

Your work could always use a new perspective

It takes courage to let another person review and possibly criticize your work – no one wants an injured ego. But most of the time, if you allow others to constructively criticize or even tear up your item or product design, especially if your work is to help these people, your bottom line will be better than you could ever have imagined.

Editor’s note: This post was originally published in August 2018 and has been updated for completeness.

ux templates

Leave a Reply

Your email address will not be published. Required fields are marked *