Barbie vs. Benchmarks: Are Those Old Metrics Stuck in the Dollhouse?
Hope you're seated, because Hollywood is still processing a shock bigger than flat feet and rain at the beach. Barbie’s initial box office projections for opening weekend came in at $70 million based on age-old research benchmarks. It blew past those predictions to $160 million, and it’s already done over $1.34 BILLION globally. By contrast, another summer big budget movie - The Flash - projected around $80 million on opening weekend but came in a lot slower at $55 million. Ouch!
Both films were audience tested by professional research firms. So what were their metrics missing?
Barbie isn’t alone. A LOT of old metrics are increasingly less reliable – partly because of poor response quality (aka, survey cheating) and partly because of a lot of these metrics don’t work as well anymore (or never did).
Even the patriarch of metrics - Net Promotor Score - is getting less reliable. LinkedIn presented some groundbreaking research at Quirks NYC in July on why they are dropping NPS. Net Promoter Score just wasn’t predicting success or failure for their new features, so they spent millions of dollars on a year-long project (with 20,000 surveys and 600 live interviews) to find an alternative. Where did they net out? They built a composite metric from two simple scores: feature value and ease of use. Now they’re replacing NPS across the company. Other companies are moving away from closed-ended metrics altogether, some even replacing them with incredibly predictive AI models from open-ended data. Curious? Ask us!
Barbie vs. Batman: Beyond Pink and Black
As if Barbie beating The Flash wasn't enough of a plot twist, "Barbie" went on to claim the title of the highest-grossing domestic movie in Warner Bros. Discovery's century-long history, eclipsing Christopher Nolan’s 2008 blockbuster, “The Dark Knight.” While superhero movies have been the heavyweights of the cinema world for years, the triumph of "Barbie" confirms that preferences are evolving.
But shouldn't market researchers have predicted this?
As it turns out, we did. Sort of... In August 2022, a Morning Consult poll showed superhero interest dropping from a peak of 41% to 36%, but that's not really a massive shift. We've seen stronger data movements in search trends (like the chart below, by StatSignificant). But even at the 2023 low, interest was at peak 2016 levels.
Clearly, there is SIGNAL in the data, but it's still missing quite a bit. Surveys (like the one above) still rely on 5 point scales that bunch up at "Interested" or "Very Interested" and don't capture true underlying enthusiasm or commitment. Google trends show us awareness, but not intent - and by the time we see the trend shift the movie's already been made.
​
There's a tremendous need in our industry for new metrics that capture things the old metrics miss - using new types of data and even completely new types of analysis. (And yes, we have some great new technology to do this, but this isn't a sales pitch.) The big point is that just as consumer preferences and needs are changing, market research needs to change too!
​
OK, that's enough heavy stuff for one day! How about some fun?
Barbie vs. Wednesday Addams: What kind of researcher are you?
Only three months ago, teens were dressing in black because of the new Wednesday Addams show on Netflix. Now it's pink (thanks Barbie)! That just goes to show how fast consumer interests can change.
But with Barbie and Wednesday Addams as inspiration, we decided to do our own little segmentation quiz.
What type of researcher are you? Are you more like Barbie? Or perhaps you resonate with Wednesday Addams?
Take our fun one-minute survey below!