Cautionary Advice: Big Game Advertsing and the "Polls"

Okay all my friends.

Most of America and a fair amount of the rest of the world will be sitting down on Sunday night to watch the "Big Game" (can't use the actual words in print/press for fear of lawsuit by the NFL) and inevitably what will be most discussed is the commercials, or adverts that run during the game.

As a seasoned Market Research Professional, here's why the news and all of you should put little to no weight behind those morning after polls or meters on what was the "best ad" during the game.

First of all, although there will be more polls than ever this year, many of the new ones will claim that they gathered their data via Facebook and Twitter which on the face of it makes it seem like it's the opinion of MILLIONS of people and therefore "more valid".  In reality, and from a sampling statistics point of view, the sample is inherently biased in that it's only of people who are willing to give an opinion and who do so via social media.  There will be no controls to ensure that, for example, Mississippi represents the .95% of the US that it does.  So if, for some reason, 3% of the responses are from MS and plenty of other variations across the States it erodes the confidence level of those results.  Additionally, there's no balance based on age, gender, income, etc., which are important things when it comes to the validity of research and that's why specifically the USA Today poll is not scientific one bit.

Secondly, in almost all advertising testing that we do, we randomize the order of the ads we show consumers.  The ads during the game will play in a prescribed order and all viewers will see them in the same order.  If, for some reason, the second or third ad is really really funny, and very well could be the best, then all ads after it are evaluated to the standard set by that ad and maybe an ad that ends up in 3rd or 4th place in these "polls", if it had been seen first in order, would have been the winner.  This is a basic tenet of good research that each and every year is ignored.

Thirdly, let's be honest, people do not pay attention to every ad.  There are bathroom breaks, there's the euphoria after a big score or big play that might be being debated at your viewing party or celebrated as the network goes to commercial and if, for some reason, it's a blow-out, then people turn back to the social aspect of their parties and pay less attention to the ads running in the 3rd or 4th quarter unfortunately penalizing those ads.

Lastly, and this is the thing that has irked me the most in the last couple of weeks, several of the advertisers (Honda, Acura to name a few) have truly blown up their own "spot" by running the ads, either in protracted version or in edited version on YouTube or their own websites far in advance of the big game.  This pre-exposure taints the ability for any true immediate impression of the ad to be measured.  Why a company would pay $3.5M for their :30 to run and then run it on the Internet prior to that is beyond me, but I would argue that by the time of the game, they may already be experiencing some wear-out of the ad.

That's the argument from the Market Research background on why those polls really aren't a true measure of what ads are the "best".

This year will also have commercials dominated by attempts to integrate social media into their big spend. Watch for some of these horribly used social media tools by almost every advertiser as most are still struggling with how to "play in the social media space".  Although I have to say my last bullet point has the biggest potential to be the best one used during the day, and is the one attempt I've seen, being used the most properly.

• The hashtag # will likely be in almost every ad.  You've seen this in many ads already but if you're not a regular user of Twitter, then it's probably meaningless.  Additionally, I'm not sure I want to be following the hashtag on my Twitter feed because frankly, it's going to be a sea of opinions from many that won't mean much
• 2D QR codes.  GoDaddy has already said that they will have a 2D code in their ads so it can take you right to their website.  You can do this if a) you even know what it is, b) you either pause your TV (with your DVR) or go back to it so you can take a picture of it with your mobile phone, c) you have a piece of free software on your phone to interpret the code and be able to use it's functionality.   This all is fine and well but if it just takes you to a website, then the advertiser is missing a key opportunity with the consumer.  Give us something for their effort (and for most who are technically challenged it IS an effort) and it'll be successful but I bet you 95% of any use of this tool just takes consumers to the advertiser's website (boring)
• Shazam logo.  I really like this one.  There is a app on iPhones, Android devices and others that "listens" to the media being played via the microphone on the phone and then the app will direct you to some additional enhanced content or offer.  The primary use of this app is so when you hear a song on the radio (or on tv or in a movie or anywhere) and you don't know who sings it, you can "tag" it and then you can buy it and download it or look it's details up later.  Some retailers and advertisers as well as some cable programs (Covert Affairs for one) are now using this app in genius ways to get consumers to specialized, app-only content.  If that happens with any ads during the game I'll be really pleased.  If it just takes you to a website, again it's a #fail.  UPDATE: IT WILL HAPPEN DURING GAME

UPDATE 2:  I got this email from Shazam (as an app user).  Very excited to see them proactively doing this, it's going to be huge for them although I can't say I'm excited about them getting into the "poll" business either.  The universe of app users can't be a balanced sample either!!

Okay, so I've probably bored most of you and I don't say all this to take any fun out of the game but I think as with political polls, Americans need to have more open eyes and transparency into what goes into quality-based market research and what happens when you lack the controls around sampling when doing it.

There are going to be some really good advertising and some real duds, but that's what we talk about every year and we all have fun re-living for days after the game.

For the record, I don't have a preference one way or another on who wins the game and since I have many friends who support both teams, I'll just say "Go Team"!

Thanks for listening.  It's how I make a living and I've been doing it for 20+ years.  Views expressed here are mine alone and not those of my employer(s).