Helio Guides
  • Progressive Design
  • Gravity Score
  • Custom Lists
  • Heart Framework
  • Landing Page Optimization
  • Sick Design
  • Video Testing
  • Consumer Psychology
  • Interaction Matrix
  • Brand Experience
    • Search
    Generic filters
    Help Center

    Heart Framework Evaluations and Test Results

    Once your Helio surveys have collected all their responses, evaluate your metrics across the 5 HEART categories.

    The easiest way to set up a HEART framework evaluation is to load the data directly into your data comparison framework, which will give you a big-picture view of which product experience performed better.

    Heart Framework Evaluation Data For…

    Happiness: The Net Positive Alignment and NPS scores gauge user satisfaction. 

    • AI Tools Example: Google Bard has a higher NPS (36.63) compared to Chat GPT (21.57), and its Net Positive Alignment is 189% versus Chat GPT’s 142%. These figures indicate that users are generally happier with Google Bard.
    Heart Framework Evaluation and Test Results for Happiness

    Engagement: Engagement is measured by the first-click interaction with the tool. 

    • AI Tools Example: While interaction with the command bar is nearly equal across the two tools (about 60%), Bard’s layout drives over 3x more users towards command suggestions.
    Heart Framework Evaluation and Test Results for Evaluation

    Adoption: The actions of users after initial use gives insights into adoption of the tool. 

    • AI Tools Example: Chat GPT users are more likely to try a different AI tool or type in a new command compared to Google Bard users. This could suggest that Chat GPT users are exploring the platform more, but it might also imply they aren’t fully satisfied with the initial results.
    HEART Framework Evaluation and Test Results for Adoption

    Retention: The likelihood for participants to interact with the tool again after first use.

    • AI Tools Example: Google Bard appears to have a higher retention rate, as only 3% of its users are unlikely to use it again for recipes, compared to 10% for Chat GPT. Also, 52% of Google Bard users are very likely to use it again, a figure higher than Chat GPT’s 44%.
    HEART Framework Evaluation and Test Results for Retention

    Task Success: Primary actions should have more than 80% of participants successfully complete. Secondary actions should be at least 75%, and tertiary actions on the edge of the experience should reach 55%.

    • Users’ ability to successfully utilize the command bar and suggestions determines task success. Google Bard boasts an 81% success rate, noticeably higher than Chat GPT’s 65%.
    HEART Framework Evaluation and Test Results for Task Success

    Load data into the comparison framework

    Using the data framework provided in the previous Template section, grab the data from your Helio surveys and load into the appropriate sections. Helio’s data copy tool will grab percentages and answer options from quantitative questions, such as multiple-choice or numerical scales. 

    You can then paste the data into the table in the spreadsheet you’re looking at, such as under the Net Positive Alignment section.

    For click test data in Helio, click and drag over a cluster of clicks to see what percentage of participants interacted in that area. For task success, we want to measure the amount of participants who clicked on the correct action on the page, while engagement looks at the top 3 places on the page where participants made their first click.

    Heart Framework Evaluation: Compare iterations against each

    After loading the survey data into the spreadsheet framework, you can compare the reactions to each experience side by side.

    In our example case study, based on the Heart framework evaluation metrics, Google Bard provides a better user experience than Chat GPT. This is evident across most HEART components, especially with Happiness, Adoption, and Task Success.