This article is about my experience conducting two kinds of user research: generative studies, and evaluative studies—and how, regardless of the methodology you choose to carry out the critical process of understanding your users, you should be involving the whole team in your efforts.
User research plays a critical role in how we make product decisions at Clio (a legal technology company).
We conduct continuous customer discovery research to get to know our users and uncover their biggest pain points.
Moreover, we don’t conduct our research in silos. We treat it as a team sport. The product manager, designer, and engineers of the project need to pass the ball to each other strategically to score the goal. Support and other customer-facing roles are also involved as the players who talk to customers on a daily basis. Leveraging everyone’s knowledge helps guide the effort of the product team to the right places.
We conduct two types of studies: generative studies to identify problems that need solving, and evaluative studies to test our solutions for the problems identified.
Generative studies
Generative studies help us identify user problems that we can solve. With this kind of study, we get to know our users, what they value, and what they need. Often, we conduct generative studies at the beginning of a new project or a new product offering.
Our customers are a diverse group of people. They are from all over the world like the US, Australia, Ireland, and Ghana. They have different practice areas in law as well as different firm sizes.
Recently, we became a multi-product company which adds to the ways our users differ from each other. This variation increases our need to run generative studies and get to know our users better.
By flying to different cities and embedding ourselves with law firms, we learn more about our users in their own daily life. These visits give us a chance to interview our customers, observe them in their day-to-day work, design with them collaboratively on the spot, or go through the exercise of building a journey map together.
Our focus in these studies is to identify pain points that we are not addressing with our current solutions.
When we come back to our office, we continue to talk to our users remotely, making sure the problems we focus on aligns with their real-world needs. This is either a phone conversation or a video conference that lasts 30 minutes to an hour.
Evaluative studies
Evaluative studies help us test whether the solutions that we have designed solve real problems and are easy to use. Typically, we run these studies remotely and have people join via UberConference or Lookback so they can share their screens with us. We ask participants to think aloud about what they expect to see in a workflow or where they get confused.
We organize evening meetups in our city for our users where they can meet the product team. These meetups give users the opportunity to provide us feedback, get help on our products from experts, and participate in usability testing to ensure the solutions that we have designed align with their needs.
The research process
User research at Clio is a collaborative effort between designers, product managers, and engineers.
It is important to involve as many members of the product team as possible to build empathy first hand and avoid losing insights in translation.
Every person is aligned on the customer pain points they need to solve, and can speak about the user’s journey. The engineer and the designer are each able to put a face on the user and know about their pain points.
The decisions made on the roadmap have the direct goal of solving these user problems. By observing these problems first-hand during research, the team gains a collective understanding of the context around the pains they are solving.
Writing up a research plan aligns the team.
As the first step of any research endeavor, the designer or product manager who wants to lead the initiative writes up a research plan to make sure everyone is aligned. This is the team’s strategy for this study. This document includes what questions they want to answer by conducting a study at this time: the “research questions”. They then identify a strategy to best answer these questions by thinking about the following:
What research methods should we use?
Who do we involve as participants?
Customer-facing members make recruiting the right users easy.
Customer-facing members such as Support, Success, Enablement, and Sales have a closer relationship with users. They talk to customers daily about their needs, their problems, and what they are trying to accomplish. It’s helpful to collaborate with the customer-facing team members while gathering the right list of target users for research. They know people who are having certain problems. They are aware of criteria such as tech-savvy or non-tech-savvy users. They recognize the various types of people that call them every day.
One other way they can contribute is by participating in internal preliminary usability testing as they are internal advocates for the user. This does not eliminate the need for regular usability testing. But it does remove low-hanging fruits in a cost-efficient manner.
Clions conducting research. One person is taking notes while another is speaking to the user.
Involve the team by assigning different roles.
During the majority of remote studies, it’s not costly to have the designer, the product manager, and the engineering manager present in the study. Engineers are also often involved. The different roles are involved by taking on various research roles. So we not only build first-hand empathy, but we also have more people to help with the study. Engineers are often involved in the study by taking notes. That way, the lead interviewer, often the designer or product manager, can focus on asking the right follow-up questions.
Analyze data collaboratively to bring in different perspectives.
After the interview, the people involved get together to debrief and analyze data collaboratively. This is an important step that is often overlooked. Having multiple people present from different roles allows them to view the data from various perspectives. It’s always good to involve multiple people in qualitative studies because it’s easy to overlook seemingly small insights.
Share insights with the entire organization.
After data is turned into insights, they share their findings internally with the entire organization. Designers and product managers talk about their findings in their own meetings with their functional team.
With a product design team of 13 people, and still growing, it is difficult to keep up to date on the different problems other designers are working on. We, therefore, format our presentations in the following order:
The ProblemThe InsightsThe Current Solution
Similarly, our product management team of 13 must portray the problem they are working on using insights from the discovery research they conducted, before moving onto describing their vision of the product.
A team sport played by the entire organization
Involving the product team in research is a requirement for the team sport to be successful. Other than them, our customer-facing team members are also involved in gathering the insights we need to build a product that is both usable and drives value to our customers. They record their insights on UserVoice, a tool to keep track of feature requests. They also record things like the why behind what people are asking for in each request, and how they have implemented a workaround for that problem in order to measure impact.
For example, a user might ask for a feature to dislike a post on Facebook. The support agent continues the conversation at this point to ask what lead the user to want the dislike button and finds out that they wanted to express their real feelings such as feeling sympathies for a sad post. In this case, providing a sad face as well as a few options to express your feelings might be a better solution.
This is an insight that you could have found out by throwing a few researchers or designers at the feature request. Having the customer-facing members have a researcher mindset allows the product team to have a deeper understanding of the whys of a request and provides a guide to focus their efforts.
Research helps us foster empathy and, with that, build the right solutions for our users.
We can only accomplish this goal if we are all aligned on what that really means to customers. Everyone works together to do their part in learning about our users and doing what’s best for them. And that’s exactly why user research is a team sport!