Applying the Scientific Method to User Testing

The scientific method: observation → question → hypothesis → experiment → analysis → conclusion. It’s a formula you probably learned and used in school as a child.

You may have used it to prove the freezing point of water or test how baking soda reacts with vinegar. But did you ever consider how this formula might be utilized outside of the realm of traditional science? 

Design – especially digital product design – is both an art and a science, and user testing is an important step in the process. Applying scientific curiosity, rigor, and analysis to user testing will enable you to generate clear, valid conclusions that inform sharper, more impactful design decisions. Here’s how.

1.  Observation

First, observe the current state of the product you are working on. You can do this by conducting one or more of a variety of different types of audits (eg.: a content audit, a UX Heuristics audit, etc.). You can also do a competitive analysis at this stage and see what other products in the space are doing.

2.  Question

Once you’ve completed your observation, consider what you want to know. Come up with questions and objectives that are important to get answered. For example, an objective could be, “We strive to understand the ease of enrolling in insurance.”

3.  Hypothesis/Hypotheses

Creating the Hypotheses

From there, turn each objective into a hypothesis, which is a specific statement that asserts an assumption that needs to be tested and validated and will have a true or false outcome. In traditional science experiments, only one hypothesis is used at a time; however, in user testing, four or five can be appropriate depending on the scope of the experiment. An example of the aforementioned objective turned into a hypothesis could be, “Users will find it easy to enroll in insurance.” This turns the unbiased objective into a statement that can be understood as true or false.

Creating the Discussion Guide

After creating all of your hypotheses, create a discussion guide with open-ended questions such as, “How easy or difficult was that process?” or, “Tell me about the content you are seeing on this screen,” that each tie into your objectives and hypotheses. All of this should help shine light on the validity of your hypotheses. Remember to stay objective. Whether your hypotheses turn out true or false, you must accept the results and relay them without judgment to your clients or stakeholders.

4.  Experiment

Your next step is then to conduct your “experiment,” the user testing itself. At Method, we typically build prototypes as a way to test before building. We design our experiments and our prototypes in tandem and walk through them, question by question, with representative users, gaining valuable insights as we go. A prototype is a valuable way to gain insights before putting in all of the development work needed in order to actually change or create the product. Conducting this experiment with a prototype before launch allows us to control multiple variables and thus have more confidence in our conclusions. 

5.  Analysis

Once you conclude your user testing, analyze the data you have gathered to determine the validity of each hypothesis. You can analyze the data both qualitatively and quantitatively. For example, you can analyze the time it took users to complete a specific task using your prototype, then cross-reference that with their verbal assessment of how quickly they were able to complete it.  While some of your hypotheses may be deemed only “partially true” or “mostly false,” you should be able to validate or invalidate all of your hypotheses to a confidence level that allows you to create a much more valuable product for your users. 

6.  Conclusion

Based on your analysis, you should be able to articulate insights and recommendations for changes to make to your product. These insights and recommendations are your conclusion. Armed with your conclusion, your team will be able to confidently and precisely improve the product based on clear user data.

Applying the scientific method to user testing is effective not only at illuminating clear data insights for your team, but also in giving you a structure in which to present your findings to your stakeholders in a clear, familiar way. It provides a thoroughly established process by which to conduct your research, as well as a simple and familiar framework for communicating results and recommendations. The next time you need to prepare for and conduct user testing, consider the scientific method.