Predicting advertising effectiveness

!
Pre-­‐ and post-­‐tes-ng of adver-sing effec-veness Prof. dr. Victor A.F. Lamme, Head of science Neurensics Dr. H. Steven Scholte, Head of R&D Neurensics May 2013
!
!
!
TV commercial effec-veness predicted by func-onal MRI !
What makes for a good TV commercial? The ques-on itself can already be understood in very different ways. What is ‘good’? A commercial that wins prizes for crea-vity or originality? Or does a good commercial simply increase sales or brand value? Obviously it is best when the two go hand in hand. However, while adver-sing agencies oUen know very well what makes for an original or crea-ve design, there is less agreement on what the ingredients are of a TV commercial that pays off in the more literal sense. !
Here, we use brain scanning techniques to get a grip on that more elusive feature of TV commercial making: how to get consumers off their couches and run towards shops. We have looked at the ‘neural signature’ of effec-ve ads, using func-onal MRI, a technique that allows for the recording of deep brain structures involved in (partly unconscious) emo-onal valua-on and decision making. We find a dis-nc-ve pa[ern of neural ac-vity that is exclusive for effec-ve TV commercials. Moreover, using this pa[ern as a template, we are able to predict whether a TV commercial will be effec-ve or not. Using this technique, we even predicted one of this year’s Effie awards. !
The elusive ingredient of effec-veness In the Netherlands, prizes are awarded for TV commercials on the basis of effec-veness, popularity, and annoyance. To win an effec-veness award (the ‘Effie’) adver-sing agencies send in their TV commercials together with evidence substan-a-ng the claim that the commercial led to an increase in sales, recogni-on, or valua-on of the brand or product the commercial was for. A professional jury looks at each case and decides who wins. Popularity awards (the ‘Gouden Loekie’) are voted for by the general public, and winners are announced in a fes-ve program on na-onal TV. Awards for the most annoying TV commercial (the ‘Loden Leeuw’) are also voted for by the general public, and are announced in a consumer interest TV program. !
Although each of these awards are based on a variety of aspects, they also form dis-nct categories. Gouden Loekies are typically won by TV commercials with high GRP that are funny, cute or otherwise likeable. Loden Leeuwen typically are high GRP ads that annoy, for all sorts of reasons. The Effie winners are a much more difficult to describe category. Their dis-nc-ve feature obviously is their capability of increasing sales or brand loyalty, but it is not so easy to get a grip on how this is achieved. They are oUen not par-cularly funny, some-mes they are aesthe-cally pleasing or technically well made, but Effie awards have been won by very non-­‐outspoken TV commercials. !
Effec-ve TV commercials have a unique ‘neural signature’ For this study we combined brain ac-vity recorded during 6 mul-client and internal research studies in each of which we recorded from 16 to 27 subjects. Subjects were exposed to 8 to 16 TV commercials, in addi-on to 4 to 6 benchmark commercials (Effies, Gouden Loekies, Loden Leeuwen). Furthermore, we performed one study (16 subjects) in which we only used Effies, Loden Leeuwen en Gouden loekies (8 each). Results from the la[er study were used for the whole brain analysis presented in figure 1. In total, we thus have 17 recordings of Effies (many of the same ad but in different pools of subjects, 8 unique ads), 20 ‘Gouden Loekie’ recordings (8 unique ads), and 20 ‘Loden Leeuwen’ recordings (9 unique ads). !
!
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 1
!
!
!
The whole brain analysis (fig. 1) clearly shows how an effec-ve TV commercial is different from other award winning commercials. Effies stand out on the basis of their ability to ac-vate brain regions such as the orbitofrontal cortex and the striatum, known to be involved on the valua-on of reward, expecta-on and desire, and found to predict consumer conversion (Plassmann et al., 2008; Berns & Moore 2012, Falk et al., 2012) !
!
Figure 1: The neural signatures of effec-ve (blue), popular (red), and annoying (green) TV commercials. Note how effec-ve commercials (blue) ac-vate prefrontal and deeper subcor-cal regions associated with emo-onal valua-ons such as expecta-on, reward, and desire. Popular commercials (red) ac-vate visual associa-on areas and parietal regions involved in a[en-on. Annoying commercials (green) are characterized by a strong ac-va-on of early visual and auditory areas, indica-ng that they almost literally strain the eyes and ears (whole brain analysis of 8 TVC’s of each class, 16 subjects, recorded on a 3T Achieva TX, 32 channel headcoil, Philips). !
To quan-fy the differences between Effies and other award winning commercials, we analyzed the full set of 57 benchmark recordings using our standardized Brain Ra-ng procedure (fig. 2). In this analysis, we first map, in each subject, the regions of the brain involved in 13 different emo-ons and valua-ons either using standardized s-muli or proprietary s-muli and tasks developed in our lab. That the reverse inference of emo-ons from brain ac-vity is possible using this technique was verified using classifica-on and correla-on analyses. For a descrip-on of the Brain Ra-ng methodology and the emo-ons characterized see www.neurensics.com. Each ‘emo-on’ is given a standardized value (a number between 60 and 140, comparable to IQ). !
Effies stand out from the other types of award winning TV commercials because they evoke higher levels of posi-ve emo-ons like desire, lust, expecta-on, and trust, while at the same -me evoking lower levels of nega-ve emo-ons like fear, anger, disgust and danger. Another striking feature is that they do not a[ract a lot of a[en-on and do not evoke much ac-vity in neural networks that mediate personal involvement (which appears to be the defining feature of popular commercials like Gouden Loekies). What do we learn from this? Most remarkable is that effec-ve commercials do not need to draw a lot of a[en-on or personal interest. That explains why they are oUen quite inconspicuous. Nega-ve emo-ons are typically avoided. Elici-ng desire and expecta-on seems most important, together with evoking a sense of trust. In our sample, we have seen these emo-ons for example being brought about by portraying craUsmen or people in a family semng, and by using easy listening type music. Filming is generally of high quality and the cumng is of medium to slow pace, all contribu-ng to a composed rather than exhilara-ng quality. !
Effies stand out from the other types of award winning TV commercials because they evoke higher levels of posi-ve emo-ons like desire, lust, expecta-on, and trust, while at the same -me evoking lower levels of nega-ve emo-ons like fear, anger, disgust and danger. Another striking feature is that they do not a[ract a lot of a[en-on and do not evoke Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 2
!
!
!
much ac-vity in neural networks that mediate personal involvement (which appears to be the defining feature of popular commercials like Gouden Loekies). !
What do we learn from this? Most remarkable is that effec-ve commercials do not need to draw a lot of a[en-on or personal interest. That explains why they are oUen quite inconspicuous. Nega-ve emo-ons are typically avoided. Elici-ng desire and expecta-on seems most important, together with evoking a sense of trust. In our sample, we have seen these emo-ons for example being brought about by portraying craUsmen or people in a family semng, and by using easy listening type music. Filming is generally of high quality and the cumng is of medium to slow pace, all contribu-ng to a composed rather than exhilara-ng quality. !
!
!
!
Figure 2: Brain Ra-ng of effec-ve (blue), popular (red), and annoying (green) TV commercials (weighted average of 17-­‐20 TVC recordings of each class). For a descrip-on of the Brain Ra-ng methodology and the emo-ons characterized see www.neurensics.com). !
!
!
Can effec-veness be predicted? Now that the neural and emo-onal signatures of effec-ve TV commercials are known, can this be used to predict effec-veness in advance? Tes-ng commercials prior to airing is fairly common-­‐prac-ce, which makes sense because the costs of airing usually outweigh the costs of produc-on by a factor of 10 or so. We analyzed whether pre-­‐tes-ng for effec-veness is possible. For each class of commercials, we constructed an average (template), leaving one of the commercials out of this average. The pa[ern of neural ac-vity of the leU out commercial was then correlated with each the three templates. This was repeated for each commercial (fig. 3). Correla-on values were fed into a classifica-on algorithm to categorize each commercial as either effec-ve, popular, or annoying (leave-­‐one-­‐out classifica-on, e.g. Ponse- et al., 2012). !
Categoriza-on was par-cularly successful for the Effie commercials: 15 of the 17 Effie commercials were indeed classified as being effec-ve (88% hit rate), while 35 of the 40 non-­‐Effie commercials were classified as not effec-ve (87% correct rejec-ons). This included correla-ons between tests of the same commercial but recorded in different pools of subjects (usually ~22 ‘average’ consumers). Such test – re-­‐test classifica-ons are typically iden-cal (92%). But even if these commercials were leU out, 6 of the 7 Effies (86%) and 10 of the 13 non-­‐Effies (77%) were s-ll correctly classified. Classifica-on of popularity or annoyance, however, was more or less at chance level in that case (33% and 43% respec-vely), indica-ng that Gouden Loekies and Loden Leeuwen form a more heterogeneous type of commercials than Effies. !
!
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 3
!
!
!
Figure 3: Leave-­‐one-­‐out correla-ons for Effies (blue), Gouden Loekies (red) and Loden Leeuwen (green) show that effec-ve commercials (blue circles) stand out as a dis-nct category from non-­‐effec-ve commercials (red and green). Effies (blue) have high internal correla-on and are nega-vely correlated with the other two types of award winning commercials (red and green). Note that the graph depicts both test – re-­‐test correla-ons of the same commercials tested in different groups of subjects, as well as correla-ons between different commercials of a par-cular category, tested in either the same or different groups of subjects. !
Pushing the right buHons In sum, we show that effec-ve TV commercials evoke a specific neural signature and stand out as a dis-nct category when rated via an fMRI based neural analysis. This allows for the successful pre-­‐tes-ng of the effec-veness of a TV commercial. Two conclusions can be drawn that are of par-cular importance to the neuromarke-ng field. The 92% correct repeat classifica-on tells us that recording from ~22 subjects yields sufficiently stable results to reliably classify a TV commercial. That classifica-on of different Effie commercials is almost as good (82%) tells us that – from a neural perspec-ve – effec-ve commercials form a very homogenous category, all pushing the same ‘bu[ons’ in our brains. We now know which bu[ons those are. !
A final proof of this pudding came recently when the 2012 Effie awards were announced. One of the awarded commercials (Eneco ‘Dutch wind’) was tested by us about 6 months earlier, where we classified it as effec-ve. It won bronze medal (gold and silver medals were not tested by us). Similarly, this year’s Gouden Loekie (Telfort ‘Calling as long as you like’) was indeed previously classified by us as a popular TV commercial. !
!
!
!
!
!
!
!
!
!
!
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 4
!
!
!
Predic-ng TV Commercial Effec-veness from Story Boards !
The pre-­‐tes-ng of TV commercials is common prac-ce, given the huge costs of airing, and the need to allocate marke-ng budget specifically to the most effec-ve ways of influencing consumers’ decisions. Func-onal MRI based neuromarke-ng techniques have proven a par-cularly useful tool, given its capability to predict effec-veness in a highly reliable way. For example, it has been shown that the ability of an--­‐smoking campaigns to induce a change in smoking behavior is be[er predicted by brainscans than by the opinions of smokers or marke-ng experts (Falk et al 2012). In the previous paragraphs we have shown that the effec-veness of TV commercials (measured by Effie award winning) can be predicted with over 80% accuracy using fMRI based evalua-on techniques. !
Huge sums of money can be saved in airing costs by this kind of pre-­‐tes-ng. Even so, it remains problema-c having to conclude that a TV commercial is be[er not aired (or with limited budget) aUer the costs of produc-on have been made. Some TV commercials take hundreds of thousands to make. It is possible to alter a TV commercial aUer produc-on, by re-­‐edi-ng, using different music, altering voice overs, or changing the pack-­‐shot. If guided by proper neuromarke-ng research and exper-se, this approach can some-mes improve the commercial considerably. But even more helpful would be a tool that allows for the tes-ng of effec-veness prior to produc-on. !
We have conducted a study to inves-gate whether this is a viable op-on. Already prior to the computer age, ideas for TV commercials were oUen pitched in the form of a story board. A sequence of images was drawn, accompanied by a brief story explaining the narra-ve and general idea or concept. Many major studios s-ll use this technique in some form or another to compare ideas and present them to clients. We tested to what extent these kinds of story boards could predict the impact on the brain – and hence the effec-veness -­‐ of the actual full-­‐produc-on TV commercial. !
Reverse engineering of Story Boards from TV commercials Twelve actual TV commercials were reverse engineered to story boards in a systema-c fashion: key shots of the TV commercial were transformed into hand made drawings, typically 10 or so per commercial. A (male) voice over was generated, explaining the ac-on, and reading the final pack-­‐shot message (we also used another method, where the original soundtrack of the TV commercial was combined with the drawings, but this yielded worse results, probably because of the oUen irrita-ng or confusing mismatch between the soundtrack and the s-ll images). Drawings and voice over were compiled into a slide show, of equal length as the original commercial. Subjects, while lying in the MRI scanner -­‐ first viewed the 12 story boards versions, followed by the 12 actual TV commercials. Note that subjects did not know these commercials, as they were foreign (mostly UK and US) commercials not aired in the Netherlands. !
To evaluate TV commercials, we use a proprietary method where we record ac-vity from 13 neural networks, that each represent a typical (largely unconscious) emo-on or valua-on given by the brain. These emo-ons / valua-ons (called mappers) can be grouped into four clusters: posi-ve emo-ons, nega-ve emo-ons, personal appeal, and general impact. Our previous research has shown that the balance between posi-ve and nega-ve emo-ons is most indica-ve of TV commercial effec-veness. We correlate the mapper values obtained for story boards with those obtained for the actual TV commercial, to test to what extent story boards have any predic-ve value for the neural impact of TV commercials. !
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 5
!
!
!
The example shown in figure 1 illustrates the principle of analysis. Both the TV commercial evoke fairly similar (but not iden-cal) mapper values. Note for example how in both, nega-ve emo-ons are generally larger than posi-ve emo-ons. This similarity can be expressed in the correla-on value Rbetween the two series, which is this case is 0.76. A value of 1.0 would indicate perfect similarity, a value of -­‐1.0 perfect dissimilarity (so that values are their exact opposite). !
Figure 1: Spider plot of 13 neural emo-ons / valua-ons (mappers) for an example TV commercial (red) and its Story Board version (gray). Values are not iden-cal, but fairly similar (note how in both nega-ve emo-ons are higher that posi-ve emo-ons) resul-ng in a correla-on value R of 0.76. !
TV commercials are predicted from their Story Boards In the sample of 12 TV commercials and story boards we tested we found cases of fairly high correla-on and cases of nearly zero correla-on, but never nega-ve correla-ons (fig 2, leU). On average the correla-on value is 0.35. This value is comparable to the correla-on value of 0.37 that is obtained when each individual mapper value of the TV commercial is plo[ed against the mapper value obtained for the story board (fig 2 sca[er plot on the right). Sta-s-cally, the correla-on is highly significant, with a p-­‐value of 0.0004. Figure 2: Correla-on values between the spider plots of 12 TV commercials and their corresponding story boards. Correla-ons are always posi-ve (leU). Sca[er plot of the individual TV commercial mapper values (13 mappers x 12 commercials) against the mapper values obtained for the story boards. Correla-on value is highly significant (right) Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 6
!
!
!
These results indicate something very important: indeed the rela-vely simple images and narra-ve of story boards have a reliable predic-ve value for the neural impact that TV commercials will have that are made according to these story boards. !
Avoiding mistakes: nega-ve emo-ons and impact are predicted best In evalua-ng our results, we found that not all mappers are predicted equally well. For some mappers (such as A[en-on) there is a very high correla-on between what is measured in the story board and the TV commercial. In others, however, the correla-on is low (figure 3). Figure 3: Sca[er plot of the individual TV commercial values of the mapper ‘A[en-on’ against the mapper values obtained for the story boards. Correla-on is 0.72 (leU). Correla-on values for each mapper vary considerably in our sample: nega-ve emo-ons and impact mappers are predicted best (right) !
On the whole, we find that nega-ve emo-ons (Fear, Anger, Disgust, Danger) and impact valua-ons (A[en-on, Novelty) are predicted best. It can be that this is due to the sample of TV commercials we tested. We used foreign commercials, and maybe nega-ve emo-ons come across most clearly in a foreign language. Or it could be that nega-ve emo-ons and the impact of a commercial best survive the transforma-on into a story board. !
Either, way, this has important implica-ons for what story board tes-ng can bring to the industry. The tool is apparently best suited to avoid mistakes: What can be predicted best from a story board is to what extent a TV commercial will annoy (anger), or evoke nega-ve emo-ons like fear, danger or disgust. These happen to be all emo-ons of which we know they need to be avoided to create a neurally effec-ve message, that can convert consumers into buying the product that is adver-sed for (see previous paragraphs). Also the amount of a[en-on that will be drawn can be predicted very well. !
Improving predic-ve value: main axes correla-ons are higher For some individual mappers, the predic-ve value of story boards is high, for others it is lower. Can this be improved? An obvious way of improving predic-ve value is to collapse emo-ons to their main axes (posi-ve emo-ons, nega-ve emo-ons, personal appeal and general impact). At the level of these main axes, story board values correlate more strongly, and also very significantly with TV commercial values (R = 0.48, p = 0.002). The same phenomenon is observed as with the individual mappers, that nega-ve emo-ons and general impact are predicted best. However, also posi-ve emo-ons and personal appeal have significant correla-ons, and hence predic-ve value. !
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 7
!
!
!
Figure 4: Sca[er plot of the Main Axes TV commercial values (4 x 12) against the main axes values obtained for the story boards. Correla-on is 0.48, and highly significant (leU). Of the four main axes, values are predicted best for general impact, followed by nega-ve emo-ons, personal appeal, and posi-ve emo-ons (right). !
This implies that if we restrict our scope to predic-ng the neural impact at the level of main axes of emo-ons, predic-on is much more reliable than at the level of individual emo-ons. All main axes show posi-ve predic-ve values, indica-ng that the story board carries informa-on about all emo-onal aspects of the final TV commercial. !
!
This makes story board tes-ng a viable tool for the evalua-on of a TV commercial prior to produc-on. Nearly perfect predic-on in the dimension that maHers: pos-­‐neg balance Earlier research has shown that the most important parameter for the effec-veness of a TV commercial is the balance between posi-ve and nega-ve emo-ons (see previous paragraphs). Much more than the absolute values of these emo-ons, this balance predicts effec-veness: will consumers buy the product (or ‘buy’ the message) and show behavioral conversion? !
Why is that the case? Posi-ve emo-ons can be high, but if these are offset by equally high nega-ve emo-ons, the net result is a message that leaves mixed feelings. Therefore, consumer behavior is best predicted by the net balance of these two opposing forces (posi-ve main axis value – nega-ve main axis value). !
Figure 5: Sca[er plot of the TV commercial Pos-­‐Neg values (12) against the Pos-­‐Neg values obtained for the story boards. Correla-on is 0.64, and significant at p = 0.03. !
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 8
!
!
!
We find that story boards are very well suited to evaluate the future pos-­‐neg balance of a TV commercial. Correla-on values increase to 0.64 (p = 0.03) when story boards are compared to TV commercials along this dimension. !
This has very important implica-ons. The future effec-veness of a TV commercial (indexed by its pos-­‐neg balance) turns out to be the value that is best predicted by its story board. This makes story board tes-ng a most useful tool if TV commercials are targeted at real conversion of consumers. It is the power to convince, to induce a change in behavior, to move consumers towards shops that is best evaluated using fMRI based story board tes-ng. This adds to the growing evidence that the real power of fMRI based neuromarke-ng lies indeed in its capability to predict consumer behavior, instead of consumers’ opinions (Falk et al 2012, Berns & Moore, 2012). !
AVerthoughts: towards a new way of TV commercial produc-on We find that even a very simple story board, consis-ng of s-ll images and a spoken narra-ve that explains the ac-on and reads the pack-­‐shot, can predict the neural impact of a TV commercial. Predic-on at the level of individual emo-ons is moderate (r = 0.37), higher at the level of main axes of emo-ons (r = 0.48) and highest at the level of the balance between posi-ve and nega-ve emo-ons (r =0.64). The la[er is of most value when it comes to predic-ng consumer behavior and conversion. !
This means that the tool of fMRI based story board tes-ng is most suited to compare different TV ad proposals at the level of storyboards when a decision needs to be made about which of these proposals will most likely result in consumer conversion. This may require a change from common prac-ce, where story boards are compiled rela-vely late in the decision process: typically to brief produc-on aUer a concept has been selected and fleshed out. !
We propose that story boards should be used rela-vely early. Already in the conceptual stage, when several ideas are discussed, story boards should be made, and tested for their effec-veness in convincing consumers. !
For the first -me a tool is available that, when properly used, can change the industry of TV commercial produc-on. No longer do we need to wait for a commercial to work. We can already know it in advance. !
!
!
!
!
!
!
!
!
!
!
!
!
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 9
!
!
!
References Berns GS & Moore SE (2012) A neural predictor of cultural popularity, Journal of Consumer Psychology 22, 154–160 !
Plassmann, H, O’Doherty J, Shiv B & Rangel A (2008) Marke-ng ac-ons can modulate neural representa-ons of experienced pleasantness. Proceedings of the Na-onal Academy of Sciences USA, 105, 1050-­‐1054 !
Falk EB, Berkman ET & Lieberman MD (2012) From Neural Responses to Popula-on Behavior. Neural Focus Group Predicts Popula-on-­‐Level Media Effects. Psychological Science, April 17, 2012 0956797611434964 !
Ponse- J, Granert O, Jansen O, Wolff S, Beier K, Neutze J, Deuschl G, Mehdorn H, Siebner H, Bosinski H. (2012) Assessment of pedophilia using hemodynamic brain response to sexual s-muli. Arch Gen Psychiatry, 69, 187-­‐194 !
!
!
!
Neurensics I Predic-ng adver-sing effec-veness I May 2013 I 10