Setting Up Your Video Test
Here is a checklist of items you’ll need when creating your own video testing session:
- Find your audience
- Gather hunches & questions
- Create Helio survey(s)
- Segment participants across surveys
- Prepare data comparison framework
Find your audience
The most important part of your testing always comes first: understanding who you’re talking to. Helio makes it easy with over 1,000 ready-made audiences at your fingertips. If you can’t find the group you’re looking for, reach out to our team and we can create a custom audience for you!
For our study on Leadpages, we targeted an audience of Small Business Owners in the United States who may want to quickly set up their own websites.
Gather hunches & questions
Start with how you think the world works. This means imagining how people will react to your video ad; both the good and the bad. Then think about how you want people to react to your videos. These predictions you have about how people can and may react to your content represent the hunches you have about your work.
Hunches form the basis of the questions we have about our videos. If you think “people like energetic background music more than calming background music”, this hunch can be turned into a question in your test.
HUNCH
The background music is too loud in our video
QUESTIONs
What type of background music do people expect in this type of video?
How do they want the music to make them feel?
We don’t recommend asking direct “yes/no” questions to participants. Instead, think about how you can prove your hunch is right or wrong. For our video test, we thought Variation A might be perceived as more vague than Variation B, so we asked participants if that was one of the impressions they felt while watching the video.
Create Helio survey(s)
Helio’s different question types can be combined to get a variety of feedback on your videos.
Of course, if you’re looking for a quick comparison between two videos, a classic A/B test is always a good choice. Our Preference Question type gives you the ability to upload up to 3 variations of a video, and requires participants to watch each video before choosing their favorite. The preference question can be asked in a variety of ways beyond direct preference, like which of the videos does the best job of communicating an idea.
Aside from direct preference tests, a common question type that we like to use for getting emotional reactions to a video is the brand impressions method. This can take several hunches you have about a video (“people will think it’s too loud and too long”) and test them at the same time. Turn each of your hunches into an impression you think will be produced by the video, and give participants a selection of positive and negative impressions to choose from.
Our hunch about the Leadpages video being too vague was tested alongside other impressions, such as Clear, Engaging, and Confusing. We give participants the option to choose any and all of those impressions, and then compare the emotional reactions to each video.
Once you’ve set up your survey(s), it’s just a button click away from collecting feedback on your video ads. Once the data is pouring in, you can use our data comparison framework to make sense of all the numbers!
Segment participants across surveys
If you have multiple videos already developed, we suggest crafting a single test that can be used to measure reactions for each video. Copying the test and sending 1 for each video will provide you with consistent data that can be compared across variations.
Helio’s Audience Exclusion feature can be used to ensure that no participants who see your first variation will be included in the tests to give feedback on the other videos. When you set up your second survey, just exclude any participants who take the first survey. If you’re setting up a third survey, exclude the participants of the first two tests, and so on.
This gives you consistent data points that can be compared between your videos, and is even more easily synthesized using our data comparison framework below.
Prep data comparison framework
Use our data comparison method to quickly make sense of the data in your surveys. The example in our Template section can be copied and customized for your own data set.
In the DCF, each separate survey is indicated by the teal header at the top of the column. The rows underneath that column layout the type of question that was asked, the answer options for that question, and the data results for each answer option.
It’s not as intimidating as it looks! Helio has some cool data copying features that allow you to easily fill out your data comparison framework in a matter of minutes. Check out our section on interpreting the test results to find out more.
