In the past year, Facebook has been hit with a series of faux pas related to metrics, but one analyst has discovered what could be the most eye-popping of all.

According to Pivotal Research Group's Brian Wieser, the audience reach estimates Facebook frequently displays to advertisers vastly exceed those of the number of people who are actually in those groups.

CNBC explained:

Facebook's Ads Manager claims a potential reach of 41 million 18- to 24-year olds and 60 million 25- to 34-year olds in the United States, whereas US census data shows that last year there were a total of 31 million people between the ages of 18 and 24, and 45 million in the 25-34 age group, the analyst said.

This raises a huge question: how can advertisers trust the reach estimates if they indicate there are far more users on Facebook in a particular group than there are living, breathing people in that group?

One possible answer: non-human users, or bots.

As one commenter suggested on Hacker News, "I doubt that Facebook is purposefully lying about their numbers, but the fact that they estimate their reach to be greater than the census results means there must be a lot of bot accounts on Facebook."

He added:

Assuming that Facebook isn't lying, and they actually see as many accounts as they claim to reach this data would suggest that at least 25% of the accounts on Facebook are alt accounts or bot accounts. And that is assuming that everyone in the target demographic who was in the census is on Facebook. Facebook must be greater than 25% bots.

While bots are one possible explanation for the discrepancy between Facebook's reach estimates and census data, other possiblities include users with multiple Facebook accounts and underage users who Facebook's algorithms have lumped into older age groups.

Technically, Facebook, like most social platforms, isn't open to users under the age of 13, but that doesn't mean they don't use the social network. According to one survey, the majority of 10- to 12-year-olds use social platforms despite the rules and 49% of those surveyed said they use Facebook.

For its part, Facebook says its reach estimates "are designed to estimate how many people in a given area are eligible to see an ad a business might run. They are not designed to match population or census estimates." Facebook's estimates are based on "user behavior, user demographics and location data from devices."

The company revealed that its reach estimates include people who visit a geographic area but don't live there, but given the significant gaps that appear to exist between Facebook's estimates and census figures, it's hard to know just how much this accounts for the discrepancies.

Which highlights the real problem advertisers face: Facebook's reach estimates, like many of its other metrics, are often generated by black boxes that advertisers have little visibility into the workings of.

According to Pivotal Research Group's Weisner, "Conversations with agency executives on this topic indicate to us that the gap between Facebook and census figures is not widely known." But even now that this has been brought to their attention, they'll probably have a limited ability to really understand what's going on because it's unlikely Facebook is going to provide much more in the way of detail about its reach estimates.

"We think that awareness of general measurement issues causes larger advertisers to require the use of third-party measurement services, including Nielsen's DAR and comScore's vCE, to provide the basis against which Facebook is paid," Weisner suggested. 

He added, "While Facebook's measurement issues won't necessarily deter advertisers from spending money with Facebook, they will help traditional TV sellers justify existing budget shares and could restrain Facebook's growth in video ad sales on the margins."

While it remains to be seen whether advertisers will really shy away from Facebook, with companies pouring more and more money into Facebook and the cost of Facebook ads jumping by nearly a quarter year-over-year in the second quarter, the stakes are increasingly high.

Patricio Robles

Published 7 September, 2017 by Patricio Robles

Patricio Robles is a tech reporter at Econsultancy. Follow him on Twitter.

2642 more posts from this author

You might be interested in

Comments (4)

Jeff Rajeck

Jeff Rajeck, Research Analyst at EconsultancySmall Business

I noticed this glitch about 18 months ago when I was looking at Facebook's Singapore stats:

https://econsultancy.com/blog/67423-how-to-avoid-bad-marketing-statistics-asia-case-study

My ultimate conclusion is that it doesn't really matter. Regardless of inflated user numbers, marketers judge campaigns based on results - be that reach, clicks, conversion, or ROI.

And its not like we can't take our business elsewhere, but few do.

More shocking to me is how little reach you get for a targeted audience - and how high the frequencies become in a short time. More research in that area would be very interesting.

11 months ago

Patricio Robles

Patricio Robles, Tech Reporter at Econsultancy

Jeff,

Interesting. Thanks for sharing that link.

I agree with you that ultimately results matter most but I do think this is important in the sense that some (perhaps many) marketers are making decisions about where to invest based on potential reach. To the extent that Facebook's figures are way off, at least in terms of what marketers think they represent, that's a problem.

11 months ago

Avatar-blank-50x50

andy greenaway, ecd at Dentsu Aegis Network APAC

The last census was 7 years ago. So there would have been an increase in population and aging groups. Also, there may be a substantial amount of non-registered citizens. The ones Trump is trying to kick out. They would not not register their names for the US for fear of deportation, but they would for Facebook.

11 months ago

Avatar-blank-50x50

andy greenaway, ecd at Dentsu Aegis Network APAC

The last census was 7 years ago. So there would have been an increase in population and aging groups. Also, there may be a substantial amount of non-registered citizens. The ones Trump is trying to kick out. They would not not register their names for the US for fear of deportation, but they would for Facebook.

11 months ago

Comment
No-profile-pic
Save or Cancel
Daily_pulse_signup_wide

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.