创新｜How to Have A Better“Storyboard Test”? （英文版）

注:本文是由我在medium上写的文章,原文请见 https://medium.com/@1273546760/how-to-have-a-better-storyboard-test-4785fd4ffe88

｜全文预计需要 10min 阅读时间

Last month, I took the team including me and 5 freelancers (who were hired by our team) to carry out the storyboard test for one project in China. Even though the storyboard test is not a high-skill-needed approach, unexpected things happening made me gain a deeper understanding of what is the storyboard and what we should be aware of. In this article, instead of thoroughly introducing the approach and how to set it up, I am going to share my key learnings on the test combined with my recent working experience.

I was conducting a storyboard test with a corn farmer in Heilongjiang Province, China

What is a storyboard test and why do we need it?

Generally speaking, the “storyboard test” approach is often used in the innovation process and when we have generated some ideas and want to get first-hand feedback from our users.

The Storyboard test, where you will embed your ideas/solutions, is a perfect approach to getting real feedback in a quick and cheap way. Usually, it is hard to express your ideas in your mind to others clearly and people may understand the same words in a different way. In this case, visual showing is definitely better than words only. Besides, because of the magic power of the story itself, participants are more easily put themselves in the story which enables them to give more true feedback, which makes the storyboard better than just visual showing. Also, it is easily made without much money and time spent.

The content and structure of the storyboard may vary according to different objectives. Commonly, the story should include six boards:

1. First, introduce the main character who is also the tested, and some features of the main character. 

2. Second, show what kind of problems she/he meets. 

3. Third, show the current way of solving this problem and why it doesn’t work anymore. 

4. Fourth, introduce the new idea/solution. 
5. Fifth, describe the details of the storyboard and how it works to solve the problem. 
6. Sixth, show the result of using the new idea

What is the process of the “storyboard test”?

The process of conducting this is quite simple. After getting some necessary demographic info of the participants, we will show the storyboard to the participants, and ask them to give marks (usually from 1–5) on the ideas after that. then we will ask some follow-up questions.

One page of the storyboard test material

After finishing collecting all data from the participants, we need to analyze the data including quantitative and qualitative data. Usually, we will calculate the average marks given by participants for each solution and the percentage of people who give higher than 4/5.

the data was calculated and put in Excel

What should we be aware of before, during, and after the storyboard test?

Before:

1. Storyboard can only test the desirability of the ideas, not the feasibility and viability. Before implementing a new idea, we need to know if it is desirable (by users), feasible, and viable. Usually, when we present this to our stakeholders or the testees, they often question whether it is feasible or viable in business. In storyboard testing, the main focus is to test whether the ideas itself liked by our users. If the answer is no, there is no room to consider the feasibility and viability.

2. Internal stakeholders may think your idea needs to be more specific to test in the field. For example, they may say you haven’t defined clearly user segments and service content, and so on. Actually, based on the agile process, we shouldn’t work too much on a solution if we are not sure whether it is liked by our users. More importantly, the information that we come up with in the office is from our perspective, not the real users, and even worse, doing this work (which may be useless in the end) will cost you a lot more time.

During:

1. Be open to gaining more information from testees in addition to understanding their opinions on ideas. When we create the storyboard, we need to express our main ideas clearly. We may put some details but not too much because at that stage we are not able and not necessary to add so much info and details.
   During the test, participants may ask some very detailed questions. Congratulations, it is the moment we should seize. Asking questions means the things behind the questions are very important to them. For example, they may ask one thing “Whether can they be there when you spray herbicide?” In this case, what you should react to is “Why do you ask this? ” Then they may express their reasons. To dig deeper after that, you should ask “What do you think this case should be?” Through this, we can know their underlying needs and their preferences to them which will help us a lot in developing the solutions.

2. Make sure other interviewers understand what you really want to test through a storyboard. Usually, through storyboard tests, we can get quantitative data (marks given by testees) and qualitative data (answers to the follow-up questions), thus, to make the quantitative data more convincing, we usually need to test with multiple persons. In our projects, each solution we test with more than 20 participants. In this case, we need other interviewers to do the interview along with you. In the best situation, you have enough colleagues who participate in the project from the beginning and know the storyboard well. However, if unluckily, you have to ask colleagues who haven’t been involved in the project before to help, hire an external agency, or even recruit freelancers to do the interview work on your behalf (this is how we conducted the test: I was in charge of one province, the two teams were in charge of other two provinces).

   In the beginning, I thought it was enough to walk them through the storyboard materials and they would know how to convey the content of the storyboard to participants and what part they should pay more attention to. However, the freelancers don’t know exactly how to tell a storyboard and don’t know what kind of questions they should ask or continue asking according to the participants’ answers. The root cause is that I didn’t really tell them what the hypothesis was and what the pain points and ideas were behind the storyboard. For example, there is one solution, what I really want to test is whether they are willing to adopt the digital management tool and what kind of worry they have for it. But in the end, interviewers put a lot of time on the price. It is not the fault of the freelancers because they don’t participate from the start but if we can show them what we want to test, they will be more oriented.

After:

1. Look at the data from different dimensions. Usually, even looking at the average marks, it can be from different dimensions. For example, for a certain solution, all people together rate 3.3, then we can check it average marks from different dimensions, such as different locations, different age ranges, and different education levels. Different dimensions will make you find out more insights hidden in massive data. There is one example from our project. One solution is liked by the majority of farmers, but by comparing medium and mega farmers' reactions to it, we find out the reason why they like it is really different even though they all give high marks to it.
2. Commonly, if we continue checking data, we will find some unusual data, for example, one data in one group of people is much higher/lower than other groups. When we find out that what we should do is look into this to explain why it happens like this, which usually will help us find more interesting insights. In our project, we found out there is one province’s data is much higher than the data of the other two provinces. After understanding why it happens like this, we found out the unusual province might be a promising market for our solution.

Here are all the learnings from this time of work. Hope it can help you carry out a better storyboard test the next time.

往期文章：