One question often leads to a significant fork in the road: “Are we validating our releases, or are we evaluating our hunches?” This insightful query, posed by Kendall Avery , strikes at the heart of a common dilemma faced by product teams.
Validation:
“Let’s do some UXR to validate this” = Tell us this works.
Evaluation:
“Let’s do some UXR to evaluate this” = What’s working well, and what needs to be improved?
—Kendall Avery, Research Manager, Uber
The allure of validation is strong; it’s human nature to seek affirmation that our ideas are on the right track. Yet, as we’ll explore, the true path to innovation and user satisfaction lies not in seeking affirmation but in the rigorous evaluation of our ideas through User Experience Research (UXR).
Understanding UXR: The Compass of Product Design
UXR is not just a tool; it’s a compass that guides product teams through the wilderness of innovation. The term encompasses a variety of practices: from UX Testing and Usability Studies to UX Insights and UX Analysis. Each practice serves as a beacon, shedding light on the user’s interaction with a product. UXR encompasses any activity that gets you closer to understanding your users. By leveraging these techniques, we can uncover what users do and why they do it.
The Validation Trap: A Myopic View
The validation trap is enticing. It promises a quick affirmation that we’ve done our job well. “Let’s do some user research to validate this,” the team says, hoping to hear that their solution is ready for the market. But this approach often overlooks the true potential of UXR.
There’s no rigid way to learn about users, and there are more answers in between a checkmark (yes, it works) …and the world is chaotic (here’s how people think, feel, and behave)
Jakob Nielsen, a leading voice in the field of usability, warns, “To design an easy-to-use interface, pay attention to what users do, not what they say.” When we focus on validation, we risk ignoring the silent signals and subtle cues that reveal the deeper needs of our users.
To design an easy-to-use interface, pay attention to what users do, not what they say.
—Jakob Nielsen, Founder, UX Tigers
Evaluation Over Validation: Embracing a Culture of Inquiry
Evaluating hunches requires embracing a culture of inquiry and continuous learning. “Let’s do some UXR to evaluate this,” should be the mantra, indicating a quest to uncover the layers of user experience—what delights, frustrates, and can be improved. This perspective shift is supported by Jared Spool, who advocates for a focus on “building the right thing” rather than “building the thing right.”
How to Evaluate Your Hunches: A Strategic Approach
Evaluating hunches in UXR is a strategic endeavor. It’s about asking the right questions, selecting appropriate methods, and interpreting data with a keen eye.
- Set Clear Objectives: Define what success looks like. Are you aiming to enhance the user interface, improve accessibility, or streamline the user flow? Be specific.
- Choose the Right Methods: Match your objectives with suitable UXR techniques. For instance, if your goal is to understand the emotional response to your product, qualitative methods like user interviews might be your best bet.
- Interpret the Data: Look for patterns and themes in the data. Consider employing tools like affinity diagramming to make sense of qualitative feedback or statistical analysis for quantitative data.
UXR Methods and Evaluation Techniques
User Experience Research (UXR), your toolbox is an indispensable resource, brimming with evaluation methods to unravel the complex layers of user behavior. It’s not just about observing what users do but delving into the intricacies of their interactions and experiences. Here are a few ways to bring in evaluation techniques, highlighted with examples from skincare company SkinSavvy in their own customer advocacy program:
Interviews and Surveys
Interviews and surveys are another vital part of the UXR toolkit. These methods allow you to hear directly from users, capturing their candid thoughts and feelings. You can tap into their experiences, preferences, and pain points through well-crafted product survey questions. This direct feedback is invaluable in understanding the user’s perspective, which might not always be evident through behavioral observations alone.
SkinSavvy started their customer advocacy program with simple surveys used to gauge the present state of their customer audience and how they react to current products:
After initial surveys of their customers, the SkinSavvy team moved into validation of their product ideas and concepts:
This particular survey revealed that most beauty providers would only initially purchase one of this new SkinSavvy product, with no one indicating they would make a bulk purchase of more than 5 at a time.
UXR Ethnographic Studies
Ethnographic studies take you a step further, immersing you in the natural environment of your users. This technique lets you observe how they interact with your product in everyday contexts. Such immersive experiences provide a rich, nuanced understanding of user behaviors and needs, revealing insights that controlled environments may miss.
The SkinSavvy team also tapped into Helio’s advocate experts to conduct secret shopping studies immediately following the release of their new provider app.
The Helio team assembled a group of 5 participants to put the new SkinSavvy app to the test by scheduling appointments through the new platform, following through with in-person treatments, and observing interactions around the app between providers and customers.After reaching out to 30 beauty providers each through the app, and following through with a treatment, Helio advocates conducted interviews with each participant to understand their user experiences:
The most impactful insight in this ethnographic study came in the form of hard data, with only 25% of beauty providers contacted through the app actually providing a response.
Interviews with the participants also revealed that the process of completing customer transactions through the app was anything but smooth, with 4 out of 5 providers handling loyalty discounts outside of the platform after treatment.
Card Sorting and Tree Testing for UXR
UXR methods like card sorting and tree testing are essential for optimizing information architecture. These techniques help understand how users categorize information and navigate through a system. You can significantly enhance usability and findability by aligning your product’s structure with the user’s mental model.
To understand whether the SkinSavvy team mindset matches with their customer perspective, they ran a card sort test asking participants to indicate what categories of skincare they see different skin problems fit into:
The majority of participants typically sorted each skin problem into the ideal category that SkinSavvy was hoping to match, with the purple boxes in the graph indicating the highest percentage of answers for each card that was sorted.
Validating Ideas through Usability Testing
For comparison, validating our ideas requires different methods. A fundamental element in idea validation is usability testing. This technique transcends simple task completion assessments. It delves into understanding the reasons behind user actions when interacting with a product. During this process, users often face challenges and obstacles. Gaining insights into these challenges is crucial for idea validation.
Reflecting on Ernest Hemingway’s insight about trust – “The best way to find out if you can trust somebody is to trust them” – also applies here. Placing trust in your users’ behaviors and feedback opens a path to significant discoveries.
Observing where they struggle, get confused, or succeed provides a glimpse into their cognitive processes. This understanding is invaluable in determining whether an idea resonates with users, allowing for refinement and optimization of the user experience to better align with their needs and expectations.
Idea Validation with A/B Testing: Leveraging Empirical Data
In validating ideas, A/B testing offers a rigorous, data-driven approach. This method involves presenting users with two versions of a product or feature and analyzing their responses. The insights gained from this empirical data are instrumental in making evidence-based decisions.
For SkinSavvy, much of their product decisions come down to testing different design iterations against each other before pushing forward with a direction. Rather than launching multiple versions live and then waiting for answers, SkinSavvy uses Helio testing to get interaction feedback across design variations before sinking in real development time:
A/B testing shines in its ability to fine-tune various aspects of a product, such as its user interface design, feature sets, and content strategies. By directly comparing how users respond to different iterations, it becomes possible to discern which elements are more effective in meeting user needs and preferences. This empirical approach to idea validation helps accurately determine the most impactful and user-friendly solutions.
Case Studies: UXR in Action
Take, for example, the case of an e-commerce platform struggling with cart abandonment. A series of A/B tests might reveal that simplifying the checkout process significantly reduces abandonment rates. Or consider a mobile app aiming to increase user engagement. Ethnographic studies could uncover that users are looking for more personalized content, leading to a revamped recommendation algorithm that boosts user activity.
UXR is fueled by curiosity, balanced by business need
Evaluating hunches with UXR is an indispensable part of creating products that meet and exceed user expectations. It’s a testament to the power of curiosity and the relentless pursuit of user-centered design. User experience cannot be manufactured; it must be cultivated with care and empathy.
When it comes to UX research, one of the trickiest parts can be navigating the diverse landscape of stakeholder expectations, especially when balancing the needs for validation and evaluation. Stakeholders, ranging from product managers to developers, and even marketing teams, often come with their own set of priorities and concerns. For example, while a product manager might be focused on feature functionality (validation), a developer might be more concerned with the technical feasibility and user interaction (evaluation). Read more about how to kickstart your research with a continuous discovery framework for your organization.
The key here is effective communication. It’s crucial to articulate the value of UX research in a way that resonates with each stakeholder group. This means not only highlighting how the research aligns with the broader business goals but also how it caters to the specific interests of each group. For instance, when discussing usability testing results, it’s beneficial to frame findings in a way that addresses a product manager’s focus on market readiness. It also touches on aspects relevant to developers, like user interaction patterns.
UXR needs a continuous, collaborative approach
Developing a collaborative research approach involves setting clear, shared goals and choosing research methods that align with project needs and stakeholder expectations. It’s about finding that sweet spot where validation meets evaluation in a way that satisfies all parties involved. And as the research progresses, maintaining an iterative approach with regular updates and feedback loops can help manage and balance these expectations. After all, keeping stakeholders in the loop builds trust and ensures the research stays aligned with their evolving needs and perspectives.
Navigating stakeholders in UX research is a delicate dance of communication, collaboration, and continuous adaptation. If you need guidance or support, simply reach out to speak with an expert to learn more.
UXR Frequently Asked Questions
UXR encompasses practices like UX Testing, Usability Studies, and UX Analysis to understand user interactions with products.
UXR aims to foster user-centered design, driven by curiosity and empathy to meet and exceed user expectations.
UXR evaluates ideas through rigorous examination rather than just seeking affirmation that a solution is market-ready.
A culture of inquiry in UXR helps continuously learn and uncover deeper layers of user experience.
Evaluate hunches by setting clear objectives, choosing the right methods, and interpreting data to understand user needs.
Interviews, surveys, ethnographic studies, card sorting, and tree testing are key methods for evaluating ideas in UXR.
Usability testing goes beyond task completion to understand the reasons behind user actions and obstacles faced.
A/B testing provides empirical data to make informed decisions by comparing user responses to different product versions.
