By Lambert Strether of Corrente.
Readers will be aware that at Naked Capitalism we’re quite skeptical of claims from Silicon Valley behemoths; see Hubert Horan’s (one, two, three, four, five, six, seven, eight, and) nine parts-and counting series on Uber and its unicorn-level valuation, as well as the continuing stories filed under “The Bezzle” in Water Cooler. And so, speaking of Silicon Valley behemoths, Facebook.
This brief post won’t address Facebook as a social phenomenon, let alone its valuation. What I do want to do is get an anecdotal sense — yes, anecdotal, but in the NC commentariat, anecdotes approach the level of Clifford Geertz’s thick description — of how well Facebook’s advertising works (and thus how reliable claims about its advertising metrics are). First, I’ll set the scene by asking what business Facebook is in; then I’ll look at some problems with that business; and then I’ll turn the floor over to you.
First, what business is Facebook in? To answer, let’s take a gander at Facebook’s 10-K statement at the SEC (“For the fiscal year ended December 31, 2016”), since Facebook is a publicly traded company. I’m no expert in reading 10Ks, but Item I is “Business.” And for comparison purposes, I’ll provide the same information from the 10-K for the New York Times (“For the fiscal year ended December 25, 2016”):
|Facebook 10-K||New York Times 10-K|
Our mission is to give people the power to share and make the world more open and connected.
Our top priority is to build useful and engaging products that enable people to connect and share through mobile devices, personal computers, and other surfaces. We also help people discover and learn about what is going on in the world around them, enable people to share their opinions, ideas, photos and videos, and other activities with audiences ranging from their closest friends to the public at large, and stay connected everywhere by accessing our products
|We are a global media organization focused on creating, collecting and distributing high-quality news and information. Our continued commitment to premium content and journalistic excellence makes The New York Times brand a trusted source of news and information for readers and viewers across various platforms.|
|Revenues||We generate substantially all of our revenue from selling advertising placements to marketers. Our ads let marketers reach people based on a variety of factors including age, gender, location, interests, and behaviors. Marketers purchase ads that can appear in multiple places including on Facebook, Instagram, and third-party applications and websites.||We generate revenues principally from circulation and advertising. Circulation revenue is derived from the sale of subscriptions to our print, web and mobile products and single-copy sales of our print newspaper. Advertising revenue is derived from the sale of our advertising products and services on our print, web and mobile platforms.|
|Metrics||The numbers for our key metrics, which include our DAUs [Daily Active Users], MAUs [Monthly Active Users], and average revenue per user (ARPU), are based on the activity of user accounts. While these numbers are based on what we believe to be reasonable estimates of our user base for the applicable period of measurement, there are inherent challenges in measuring usage of our products across large online and mobile populations around the world. In addition, we are continually seeking to improve our estimates of our user base, and such estimates may change due to improvements or changes in our methodology.||
In the United States, The Times had the largest daily and Sunday circulation of all seven-day newspapers for the three-month period ended September 30, 2016, according to data collected by the Alliance for Audited Media (“AAM”), of most U.S. newspapers and magazines
Internationally, average circulation for the international edition of our newspaper (which includes paid circulation of the newspaper in print and electronic replica editions) for the fiscal years ended December 25, 2016 , and December 27, 2015 , was approximately 197,000 (estimated) and 215,000, respectively. These figures follow the guidance of Office de Justification de la Diffusion, an agency based in Paris and a member of the International Federation of Audit Bureaux of Circulations that audits the circulation of most newspapers and magazines in France. The final 2016 figure will not be available until April 2017.
Paid digital-only subscriptions totaled approximately 1,853,000 as of December 25, 2016 , an increase of approximately 46% compared with December 27, 2015 . This amount includes paid subscriptions to our Crossword product, which totaled approximately 245,000 as of December 25, 2016. This amount also includes estimated group corporate and group education subscriptions (which collectively represent approximately 7% of total paid digital subscriptions to our news products)…. According to comScore Media Metrix, , in 2016, NYTimes.com had a monthly average of approximately 85 million unique visitors in the United States on either desktop/laptop computers or mobile devices. Globally, including the United States, NYTimes.com had a 2016 monthly average of approximately 122 million unique visitors on either desktop/laptop computers or mobile devices, according to internal data estimates.
So it would seem that Facebook and the Times are really in the same business: Selling eyeballs to advertisers, although, for historical reasons, the Times sells eyeballs via print, for which is uses a metric called “circulation.”
Second, Facebook faces a business problem; it’s unaudited “internal company data” has been, at times, grossly inaccurate; that is, it may be selling eyeballs, but in 2016 buyers don’t really know which eyeballs, how many, and where. Again from the Facebook 10-K:
For example, we discovered an error in the algorithm we used to attribute our revenue by user geography in late 2015. While this issue did not affect our overall worldwide revenue, it did affect our attribution of revenue to different geographic regions. The fourth quarter of 2015 revenue by user geography and ARPU amounts were adjusted to reflect this reclassification. We regularly review our processes for calculating these metrics, and from time to time we may discover inaccuracies in our metrics or make adjustments to improve their accuracy, including adjustments that may result in the recalculation of our historical metrics. We believe that any such inaccuracies or adjustments are immaterial unless otherwise stated.
“We are doubling down on our efforts at third-party verification,” said Carolyn Everson, Facebook’s vice president of global marketing solutions.
In addition, the social-media network said it is working with ratings firm Nielsen to count Facebook video views, including both on-demand views and live viewing, as part of Nielsen’s Digital Content Ratings metric.
Lastly, Facebook said it plans to form a Measurement Council made up of marketers and ad agency executives, and will roll out a blog to communicate more regularly on updates about measurement.
Facebook revealed a number of new accountability developments in a blog post today, the most significant of which is a commitment to an audit by the Media Rating Council to verify the accuracy of the information it delivers to marketing partners. This and other moves will be integrated over the course of the year. MRC is considered the gold standard of verification in advertising, but even [ P&G’s chief brand officer Marc Pritchard] admitted that it is not a perfect solution. Never the less, MRC verification is an important first step in the face of the many new types of engagements offered by mobile and social media platforms.
(MRC does seem to have some clout; it suspended two Google ad measurement metrics over “non-compliance” issues; here, from their pleasingly retro website, is their mission statement:
The Mission of the MRC is to:
● Secure for the media industry and related users audience measurement that is valid, reliable, and effective
The MRC accomplishes its objective by
● Setting Standards
● Conducting audits performed by an independent CPA firm to verify compliance with our Standards
The head of MRC, George Ivie, describes the audit procedure more precisely (and I would like very much to know the exact standard under which Facebook is to be audited, because Ivie doesn’t say. Readers?). From an interview last week (and I’m going to leave it lengthy so you can get a dawning sense of the potential for skeeviness):
RTBlog: So what’s the goal of the audit?
Ivie: We have to audit the originating data. Is it accurate? And how do we do it? Right now, we’re looking at how to audit YouTube, and what does the audit cover? For example, . We want to understand what the communications path is.
For example, Moat might have 50 metrics it reports on, but a key metric is viewability and the length of time an ad appears. We have to make sure that has enough signaling information to report on the metrics.
RTBlog: What metrics are most important to these companies?
Ivie: I would say viewable impressions is what they’re looking to have validated. For YouTube, it’s video ad impressions that are served and viewable, and duration metrics—those are the most important ones. Facebook has a news feed interface and that drives metrics differently, so there might be different metrics .
RTBlog: What is the audit process like?
Ivie: Our audit process is voluntary. . Advertisers that are MRC members, like P&G, can look at the audit reports. They can see what metrics are accredited or not. My impression is that Google and Facebook will go with the metrics that we suggest and the ones the marketplace wants.
So Facebook chooses what data to submit. I’ll note that if we were back in the print world, this would like locking the auditor into a room, and handing him a report of how many papers were delivered, instead of letting him look at the trucks or the loading dock. In other words, it’s the metric that’s certified, but not the data driving the metric, or the algorithm creating that data. Because I’m a suspicious bastard, I’ll also note that this sounds like a recipe for a phishing equilibrium to me (“If there can be fraud, there already is”).
Of course, the entire online advertising “ecosystem” is, again, notoriously skeevy. It’s also extremely complex, highly technical, rapidly changing, and riddled with middlemen and rent-seekers of all kinds. So it may be too much to expect to get any sort of a handle on Facebook’s advertising metrics through a little online research.
Therefore, dear readers, we turn to you. If Facebook’s user engagement systems and advertising placement services are working well, then they should be working well for you. Do they? If a really stupid advertisement appears, can you account for it? How about a well-tailored one? If the ad is not appropriate for you, does somebody else use your account? Your machine? Is there someone else in your household for which the ad would obviously be appropriate? How about random ads? Are they really random? If you are the sort of person who leads your life at multiple locations, do the ads seem to track your location? What is your location like? (No need to reveal it!) If the ads track accurately, have you tried to defeat the tracking, say by turning off cookies? If you tell Facebook you don’t like an ad, does it seem to take your wishes into account? And so forth. Have at it! Let’s conduct an audit of user experience!
 I use Facebook as little as possible. I also regard advertising on it as a negative (unless the product advertised is one I have already purchased. I have never purchased anything I saw on Facebook).