Hello, digital innovators and product development nerds! 👨💻🚀
Diving into the digital product testing pool can be a thrill and a challenge. Whether you’re fine-tuning an app or launching the next big software update, validation is your lifeline. That’s where Itamar Gilad‘s AFTER framework comes into play, giving structure to the sometimes chaotic world of digital product testing. And guess what? We can turbocharge this framework with Helio to achieve testing brilliance.
The AFTER Approach
The AFTER framework is a systematic approach to digital product testing covering every angle, from cost-effective assessment methods to the high-stakes launch and release analysis game. This graphic presents the AFTER framework, developed by Itamar Gilad, a systematic approach for validating product development at various stages.
By thinking within this framework, product teams can systematically approach testing, ensuring that each feature and iteration is validated against real user data and feedback. This leads to a more user-centric product, increases the chances of product-market fit, and maximizes the efficiency of the development process.
There’s a wide gamut of validation methods at our disposal. I like to group them into five categories, from the cheapest and least accurate, to the most costly and rigorous: Assessment, Fact-Finding, Tests, Experiments, and Release results (AFTER).
Itamar Gilad, Author, Keynote speaker, Coach, Product Front
Applying the AFTER Framework with Helio
Let’s break the AFTER framework down and discuss its benefits:
Assessment:
Here’s the groundwork where we set our sights on what we’re trying to achieve and how the market looks.At this initial level, we see foundational activities like “Goals Alignment,” “Business Modeling,” and “ICE Analysis,” which involve setting clear objectives, understanding the business context, and prioritizing features or changes. This stage ensures you’re building the right thing before investing heavily.
Online eCommerce companies need to keep their finger on the pulse of their target audience throughout the year, especially during seasonal holiday shopping. Getup, an online formalwear company, starts with product discovery and before moving into UX feedback and interaction experiments.
For the formalwear eCommerce company Getup, we started by gauging user needs and reactions to early concepts before putting any effort into production.
The simplest feature ideas can be expressed through writing, which gives us early opportunities for gut reactions to Getup’s ideas. For instance, the foundation of their online professional stylist concept didn’t receive as much interest as they expected, showing the need for more discovery before moving down that pathway.
Fact-Finding:
Think of this as your digital product’s background check — understanding your users, competition, and market fit.
This phase involves gathering qualitative and quantitative data about your product and its users. Techniques like “Surveys,” “Competitive Analysis,” and “User Interviews” help collect insights that inform the direction of product development.
Getup can be sure that their online apparel consumers have strong opinions and decision-making habits, since their audience is targeted towards people with experience purchasing formalwear before:
List the most important things you consider when purchasing a suit for yourself:
“1. Weight of the suit (I don’t like real heavy ones), 2. Fit (do I feel constricted or able to move around and do what I need to), 3. Price (can I afford it), 4. Color / Style, 5. Brand name (for me it is all about the fit, the name does not matter.)”
— Helio Participant, Online Clothing Consumer
Tests:
The real deal of digital product testing starts here with prototypes and usability trials.
This is where prototypes and early versions are put to the test with methods like “Smoke Tests,” “Wizard of Oz,” and “Concierge Test.” These low-fidelity tests ensure the core idea has merit before further development.
For Getup’s concepts that make it to the build phase, lo-fi wireframe testing is in order to ensure usability before spending time on more refinement. Actions like changing stores and selecting dates for their Push to Store feature were tested for usability success.
As a primary action on the page, this directive to change stores was expected to show between 80-90% success, and the resulting 86% of successful clicks means the team can move on with confidence.
Experiments:
This is where we ramp things up with rigorous A/B testing and more to refine our digital offerings.
The product is subjected to rigorous testing with real or nearly real customers at this more advanced stage. Methods like “A/B/n tests,” “Multivariate,” and “Beta” versions allow for testing variations and collecting feedback on what works best.
Using real prospective customers provided by Helio’s ready-made audiences, concept testing between feature variations helps validate your pathways before sinking time into production and development. Helio is great for multivariate testing, like the 2 versions of Getup’s Look Suggestion feature tested for user emotions and impressions. Once the 2 surveys are run with separate participant pools, the data can be compared in a framework such as this:
Differences of 10% or larger are considered significant when it comes to impression testing, so Getup’s V1 is a clear winner. Impressions of helpfulness were more significant for V1, and it did a great job of avoiding the confusion and some doubt produced by the other variation. View the data comparison framework.
Release results:
Finally, the launch pad where we measure success and learn from real-world applications.
This final phase involves analyzing the success of the product post-launch with metrics such as “Post-launch,” “% Launch,” and “Holdback,” providing data on actual usage and satisfaction.
Net Positive Alignment, produced by taking the sum of positive emotions around an experience and subtracting the sum of the negative, is a great data point for comparing across variations and establishing benchmarks for future iterations.
As Getup builds off the success of their winning variations, they can compare new versions against the results of past attempts to ensure that their user experience improves over time.
Benefits of Using the AFTER Framework for Digital Product Testing
The Helio platform, as highlighted across all these stages, can be used with the AFTER framework to enhance each stage with tools like targeted surveys, audience segmentation, and in-depth analytics, making the process more efficient and effective. Before jumping into Helio, let’s outline more of the benefits of using the AFTER framework for digital product testing.
- Cost-Efficiency: The AFTER framework spans from low-cost to high-investment techniques, allowing teams to choose the most cost-effective method at each stage of development.
- Risk Mitigation: By employing progressively more rigorous validation methods, AFTER helps mitigate the risk of building something users don’t need or want.
- User-Centered Design: This framework emphasizes user feedback at every stage, keeping the product development process aligned with user needs and preferences.
- Evidence-Based Decisions**: The AFTER framework encourages decisions based on data and user research rather than assumptions, leading to more successful outcomes.
- Progressive Validation: It acknowledges that product development is a journey. Instead of a single test, it recommends a series of validations that grow in complexity and depth.
Supercharge Your Digital Product Testing with Helio
Helio isn’t just another tool in your digital product testing kit—it’s the nitrous boost your testing strategy needs.
- Next-Level Testing Creation With Helio, your digital product testing isn’t just about answering “what” users like; it’s about uncovering the “why” behind their choices with interactive tasks like click testing and prototype feedback.
Helio empowers interactive testing, such as the click tests used to gauge participant actions in the Tests phase of the AFTER framework, explained above. You can also upload a website link, Figma file, Figma prototype, audio, and visuals to get real-time interactive data from your creative assets. - Precision Audience Targeting Targeted audience selection is key in digital product testing. Helio’s massive panel ensures you’re not shouting into the void but talking to a crowd tailor-made for your digital product.
Use Helio’s ready-made audiences to quickly tap into your target customers. Search through over 1,000 pre-made participant groups, such as the Online Formalwear Consumers (US) used to put Getup’s eCommerce site to the test. - Dive Deep with Research CRM Get to the heart of your digital product testing by engaging with your users directly. Helio’s research CRM goes beyond mere numbers; it’s about meaningful interactions.
Gather data from a prospective audience group of Helio’s participants, or connect to your current customers using CRM lists or site Intercepts. Pose questions to visitors of your site as they complete specific actions, or simply load a list of your customers into Helio for rapid testing and tracking of results across design iterations.
More Than Data: It’s About Engagement
Helio elevates digital product testing from collecting data to engaging with it. It enables follow-up questions and deep dives into the user experience, turning surveys into conversations.
Fast-Track Your Learning Curve
Digital product testing is a cycle of learning and iterating. Helio, combined with the AFTER framework, propels you through this cycle at warp speed, ensuring your product evolves with genuine user input at every step.
From AFTER to Evidence-Guided Mastery
Itamar’s wisdom isn’t confined to the AFTER framework. Dive into his book “Evidence-Guided” at EvidenceGuided.com to saturate yourself in evidence-backed digital product development.
Turning Helio User Feedback into Your Product’s Greatest Strength
Getup used audience feedback to master product discovery, measure feature variations against each other, and deliver rapid results for their team. Each of their product decisions are based off of user data and validation.
And there you go! Your path to mastering digital product testing is clear. Embrace the AFTER framework, empower it with Helio, and join the ranks of those shaping the future of digital experiences one test at a time. Get out there and transform user feedback into your digital product’s superpower. 🌐✨
Until our next digital rendezvous, keep testing, keep innovating, and above all, keep creating those exceptional digital experiences the world eagerly anticipates! If you want to learn more about how to use the AFTER framework with Helio, speak with an expert today.
FAQ on Digital Product Testing with the AFTER Framework
The AFTER framework, developed by Itamar Gilad, is a systematic approach for validating digital product development at various stages, ensuring a more user-centric product and increasing the chances of product-market fit.
The AFTER framework consists of five stages: Assessment, Fact-Finding, Tests, Experiments, and Release results. Each stage progressively increases in validation rigor and investment.
- In the Assessment stage, foundational activities like Goals Alignment, Business Modeling, and ICE Analysis are conducted to set clear objectives, understand the business context, and prioritize features or changes.
- The Fact-Finding stage involves gathering qualitative and quantitative data about the product and its users through Surveys, Competitive Analysis, and User Interviews.
- During the Tests stage, prototypes and early product versions are tested using methods like Smoke Tests, Wizard of Oz, and Concierge Tests to ensure the core idea has merit before further development.
- The Experiment stage involves rigorous testing like A/B/n tests, multivariate tests, and beta versions with real or nearly real customers to refine the digital offerings based on feedback.
- The Release results stage analyzes the success of the product post-launch using metrics like Post-launch analysis and Holdback data to assess actual usage and satisfaction.
Helio enhances the AFTER framework by providing tools like targeted surveys, audience segmentation, and in-depth analytics, making the digital product testing process more efficient and effective.
The AFTER framework offers cost-efficiency, risk mitigation, user-centered design, evidence-based decisions, and progressive validation, enhancing the product development process.
Itamar Gilad’s book “Evidence-Guided” provides further insights into evidence-backed digital product development and can be a valuable resource for mastering the AFTER framework.
