A new Wall Street Journal story probes the frequency and casualness with which Facebook ran experiments with the explicit aim of manipulating users’ emotions. Some commentators pooh poohed the concern about the study, saying that companies try influencing customers all the time. But the difference here is that manipulation usually takes place in a selling context, where the aims of the vendor, to persuade you to buy their product, are clear. Here, the study exposed initially, that of skewing the mix of articles in nearly 700,000 Facebook subscribers’ news feeds, was done in a context where participants would have no reason to question the information they were being given.
Facebook’s conduct fell so far below acceptable standards for conducing research that it would have been criminal if funded by Federal grants. As Jaron Lanier, an independent scientist at Microsoft, pointed out in a New York Times op-ed:
Research with human subjects is generally governed by strict ethical standards, including the informed consent of the people who are studied. Facebook’s generic click-through agreement, which almost no one reads and which doesn’t mention this kind of experimentation, was the only form of consent cited in the paper. The subjects in the study still, to this day, have not been informed that they were in the study. If there had been federal funding, such a complacent notion of informed consent would probably have been considered a crime. Subjects would most likely have been screened so that those at special risk would be excluded or handled with extra care…
It is unimaginable that a pharmaceutical firm would be allowed to randomly, secretly sneak an experimental drug, no matter how mild, into the drinks of hundreds of thousands of people, just to see what happens, without ever telling those people. Imagine a pharmaceutical researcher saying, “I was only looking at a narrow research question, so I don’t know if my drug harmed anyone, and I haven’t bothered to find out.” Unfortunately, this seems to be an acceptable attitude when it comes to experimenting with people over social networks. It needs to change.
And in case you think this study fell within the ambit of market research, think again. From Forbes:
Defenders of the Facebook study including my colleague Jeff Bercovici say that everyone on the Internet is doing A/B testing — showing users two versions of something to see which resonates more based on how they click, share, and respond. But the Facebook study with its intention to manipulate the Facebook environment for unknowing users to see whether it made them feel elated or depressed seems different to me than the normal “will this make someone more likely to buy this thing” kind of testing. “They actually did a test to see whether it would have a deleterious effect on their users,” says Pam Dixon of the World Privacy Forum. “This isn’t A/B testing. They didn’t just want to change users’ behaviors, they wanted to change their moods.”
The Forbes article also points out that the Facebook user agreement didn’t include “research” as a possible use of user information when the study was underway; Facebook didn’t incorporate than until 4 months after the project was completed, in May 2012. Note that Facebook was also negotiating a consent decree with the Federal Trade Commission over “unfair and deceptive” concerning member privacy. But since that decree wasn’t inked until August 2012, Facebook appears to have believed it could continue to play fast and loose.
Tonight, the Wall Street Journal reports that Facebook had a team of two dozen researchers, the Data Sciences Group, whose job is to run experiments on Facebook users. It has conducted over 1000 tests since 2007. For instance, Facebook locked some users out and required them to prove they were legit as part of an anti-fraud study. From the Journal:
“There’s no review process, per se,” said Andrew Ledvina, a Facebook data scientist from February 2012 to July 2013 who worked on the study that required users to prove they were real. “Anyone on that team could run a test,” Mr. Ledvina said. “They’re always trying to alter peoples’ behavior.”
He recalled a minor experiment in which he and a product manager ran a test without telling anyone else at the company. Tests were run so often, he said, that some data scientists worried that the same users, who were anonymous, might be used in more than one experiment, tainting the results….
One published study deconstructed how families communicate, another delved into the causes of loneliness. One test looked at how social behaviors spread through networks. In 2010, the group measured how “political mobilization messages” sent to 61 million people caused people in social networks to vote in the 2010 congressional elections.
The interest in the effectiveness of political messaging is troubling given Facebook’s connections to the Department of Defense. From SCG News:
In the official credits for the [emotions] study conducted by Facebook you’ll find Jeffrey T. Hancock from Cornell University. If you go to the Minerva initiative website you’ll find that Jeffery Hancock received funding from the Department of Defense for a study called “Cornell: Modeling Discourse and Social Dynamics in Authoritarian Regimes”. If you go to the project site for that study you’ll find a visualization program that models the spread of beliefs and disease.
Cornell University is currently being funded for another DoD study right now called “Cornell: Tracking Critical-Mass Outbreaks in Social Contagions” (you’ll find the description for this project on the Minerva Initiative’s funding page).
The Department of Defense’s investment in the mechanics of psychological contagion and Facebook’s assistance, have some very serious implications, particularly when placed in context with other scandals which have broken in the past two years.
In other words, researchers that the Department of Defense is funding to understand how ideas and news goes viral are doing very similar work for Facebook. The cross pollination is high and means that Facebook users are making a direct contribution not only to the surveillance state having even more data, but to perfecting its methods.
This revelation is unlikely to have any meaningful impact on Facebook in the US. But European countries have much stricter privacy rules, and this news is likely to intensify political pressure to take action to curb US technology giants’ reach and activities. So while Facebook users overseas might eventually see some concrete protections put in place, don’t hold your breath over any changes stateside.
42 comments
Comments are closed.
You can call it research but it’s not. It’s psychological warfare, brain washing, mind manipulation.
Someone should call the FCC for their failure to disclose.
It clearly flouts the Ethics Code of the American Psychological Association, regarding informed consent, summarized here in response to the Facebook study:
http://www.apa.org/news/press/releases/2014/06/informed-consent.aspx
These 8 points (from the above link) are quite clear:
“(1) the purpose of the research, expected duration and procedures; (2) their right to decline to participate and to withdraw from the research once participation has begun; (3) the foreseeable consequences of declining or withdrawing; (4) reasonably foreseeable factors that may be expected to influence their willingness to participate such as potential risks, discomfort or adverse effects; (5) any prospective research benefits; (6) limits of confidentiality; (7) incentives for participation; and (8) whom to contact for questions about the research and research participants’ rights. They provide opportunity for the prospective participants to ask questions and receive answers.
Separately, was it Lambert’s site that mentioned an effort at getting comments from the PNAS editors? Unsettling that the “research” was deemed fit to publish, based on Facebook’s own definition of informed consent. That’s another aspect of the story, also important.
These “experiments” are probably more common than we realize. Consider these papers, perhaps as proposals.
Also, consider the cavalier attitude the authors demonstrate for their sites users indicative of a fact, their users are not the means by which their sites are supported. Primary funding probably comes from elsewhere, namely data collection income streams, of all kinds, especially for the government.
Its perhaps demonstrative of a totalitarian state or a pre-totalitarian state in the making.
Contempt for the real society is normal in totalitarian states. Human experimentation is also the norm under totalitarianism. Irresponsible human experimentation in the US, in the medical and now, “health care economics” context (IMO) is also nothing new in the US. But we had hoped that it was on the wane, at least I did.
Informed consent obviously cannot be given, on a site like FB because most people would strenuously object to being manipulated.
Consider a scenario where the financial model of a big social web site revolves around providing information to those who would pay for it.. government, more than any other customer, they pay, for information about the users. We would expect the users to be seen ambiguously, even contemptuously.
And of course the whole mess, beginning with the true goals of those providing the “service” (to the government and private data providers?) would be dishonest.
Now, consider totalitarian states. A pervasive goal of totalitarian governments, is ending access to anything that doesn’t come from them. Basically, its isolating each person from any kind of organizational structure that is not co-opted, which could give them knowledge, or friends or strength- a process which what Arendt calls “atomisation“.
Under totalitarianism, we would end up with a fake online world where people who the government deems unworthy of “freedom” (those who have been or who hope to be free) were either not allowed online, or if they were, they would be put into digital bubbles.
Certainly, from the viewpoint of criminals, covering up their crimes is of primary importance. But the loss of any real commons to society is literally loss of the means by which society could evolve and adapt ino the future.
Online communities being important places where ideas would be interchanged, censorship should be intolerable in them for almost any reason. (and care needs to eb taken because governments without scrupoles will abuse any loopholes at all, after all “its an emergency”.
Really, the future of the nation is at stake if real online communities are threatened or cease to exist.
We need a functioning commons. FB and other commercial sites are no more a functioning “real commons” than shopping malls are a replacement for real physical interaction in public spaces. They are “Simulacra”.
Internet users need to both speak up against this kind of manipulation and also develop our own alternatives to it. If such a thing is still possible.
Do laws and the growing pursuit of control make that impossible in the US? Will there ever be some place where there is transparency and privacy? Someplace where there still is a real online public commons – controlled by nobody and open to everybody without hierarchy or surveillance?
if there is would it then become both a magnet for the creative and a target for destruction.
Probably both.
@ Lambert, Yves –
‘Defenders of the Facebook study … say that everyone on the Internet is doing A/B testing — showing users two versions’
Directly concerning NC and in a related vein: do you guys know why there’d be at least an hour’s discrepancy between when new NC posts show if I’m using Google Chrome (earlier) and when I’m using Firefox (later)?
I can literally toggle back and forth between the two browsers sometimes and see new NC posts via Chrome that I can’t see via Firefox till an hour or more later. While Chrome presumably has more direct access to Google infrastructure, that should be irrelevant since I have NC bookmarked on either browser. I’ve some hypotheses for what might be happening, but do you guys know?
You’ve got a typo, incidentally, with the computer scientist’s name in 2nd sentence, 2nd graf.
It should be JARON Lanier, not Jason.
Fixed, thanks!
Check your browser’s caching settings first. Set the browser to fetch the page each time. You may have to type in “about:config” in Firefox to see some of them.
But, that said, what you describe definitely happens, a lot. Web sites serve different pages to different people. Thats what dynamic content is.
That makes all the sense in the world. Thanks.
Disclaimer: speculation from fundamental principles by long-time computer enthusiast and professional.
On the client side, caching policy is an implementation choice and is not terribly standardized. A browser rendering stale data feels faster than a browser waiting to fetch fresh data, and Firefox has a bit of bloat to make up for.
On the server side, http://www.nakedcapitalism.com resolves to four IP addresses (two v4 and two v6), which suggests two disjoint cache installations which probably do not update at exactly the same time.
So if Ctl+Shift+R fixes it in Firefox, it’s probably the former. If not, it’s probably the latter.
It makes me wonder if there isn’t a nefarious reason behind Facebook encouraging me to invite my ex- to like every page I happen to visit. Seriously, every page I look at, Facebook thinks I should invite my ex- (and only my ex-, it never suggests anyone else, ever) to like the page. This despite the fact that our differing interests (politics/economics vs. chemistry/physics) was one of the major reasons for our split! Come on, FB, you should know better…
You must have been in the Pain City group for that experiment.
If you’re a Firefox user, the NoScript addon can be set to selectively block other sites from including scripts, web bugs or embeds from Facebook or others. Here follows a stanza from my USER ruleset in the ABE tab under Advanced. The first bits in the allow are so that Facebook itself still works nominally; the google URL bit is to avoid having to manually unescape interstitital redirects from Google search to get to the destination page and is completely optional. NB the list of Facebook domains here is probably not comprehensive, but it’s all I’ve noticed in the wild.
Thanks for this post. It’s worse than I thought. I figured some corporations and maybe some political party affiliates were involved. The DoD and, by extension, the NSA ? yikes. Glad I don’t use Facebook.
The test was completed in Jan 2012. The August 2012 date mentioned brought up a memory: mid-2012 was about the time a lot of stories were coming out in various media suggesting that *not* having a Facebook account was a sign of unhealthy or suspicious behavior. What a coincidence.
http://www.dailymail.co.uk/news/article-2184658/Is-joining-Facebook-sign-youre-psychopath-Some-employers-psychologists-say-suspicious.html
Oh my, you’ve gotta love the picture and caption (many readers, especially Daily Mail readers, will skim the text and just look at the imagery): “Loner: James Holmes, the accused Colorado theatre shooter, does not appear to have friends and did not have a Facebook page”. Talk about guilt by (dis-) association !
Yes, I do remember this sort of meme — I was thinking about it myself when reading through the feature. Looking back with the benefit of hindsight based on what we now know, seems like a classic Nudge Theory http://en.wikipedia.org/wiki/Nudge_theory planted story / media campaign. The finger of suspicion naturally points at Facebook, but I’m wondering if there were other actors at work — probably not in an organised, collaborative fashion but who’s disparate interests just happened to coalesce around the same wish-list.
Which actors ? Well, the usual suspects — governments, big business and the security services spring readily to mind.
And just to not be a smart alec, when reading these stories at the time, I did myself get to thinking (albeit that I didn’t act specifically on those thoughts) “gee, I really should consider keeping my Facebook / Twitter / LinkedIn a bit more up-to-date…”
And I consider myself pretty sensible and not overly swayed by the MSM.
Daily Mail presentation is a bit over the top but now seems apt.
As for which actors ? Yes, the usual suspects do come to mind.
They already consider those with Facebook accounts, who lack friends to be suspicious, but now they are suggesting that anyone who abstains from Facebook altogether may be even more suspicious. If a person doesn’t have a Facebook page, it is a red flag. It may be the first sign that someone is a potential mass murderer! Job seekers who don’t have Facebook accounts send up all sorts of red flags. What are they hiding?
Jeez!!! It is disappointing that there’s such an obsession with social networking. We’re not all addicted to digital-dopes-of-choice. There are plenty of us productive, normal, worthwhile human beings who don’t have or want a Facebook account.
I have often wondered what Henry David Thoreau, the man who “went to the woods to live deliberately” and “front only the essential facts of life,” would think of today’s Facebook and social media?
Upon hearing about the new telegraph, Thoreau speculated, “We are in great haste to construct a magnetic telegraph from Maine to Texas, but Maine and Texas, it may be, have nothing important to communicate. Either is in such a predicament as the man who was earnest to be introduced to a distinguished deaf woman, but when he was presented, and one end of her ear trumpet was put into his hand, had nothing to say. As if the main object were to talk fast and not to talk sensibly… perchance the first news that will leak through into the broad, flapping American ear will be that the Princess Adelaide has the whooping cough….”
The more things change, the more they stay the same. All the incessant mindless jibber-jabber on digital-dopes-of-choice drains our humanity.
DakotabornKansan–great comment.
Thoreau would have also lacked a credit score and would have needed his friend Emerson to co-sign all his loans.
…and he would definitely (still) be a tax-protester.
This is unfair. Why would a respected, global US corporation doing the best for its customers behave in an underhand way as is alleged? I think this is clearly the work of one or more bad apples in the organization and does reflect in any way on the beneficence of Facebook’s glorious management team.
Yeah, they really should be a shoo-in for this year’s Enron Memorial Award for Ethical Conduct.
Who’s the customer? As someone once said about Facebook, if they aren’t paying you, then you are the product. I suppose we should replace “product’ with lab rat.
As mathbabe has made clear, people who put their stuff on FB are not users, they are the product! The users are the companies who advertise on FB. So the experiments, etc. are perfectly understandable from FB point of view. They can look you in the eye and sincerely wonder why you, the product, are upset.
So far, I haven’t seen any political initiative in Europe, except in the UK, against Facebook. This is proof of how low have becaome expectations on policy actions in Europe. I think Facebook wont face big challenges in Europe about this. In some outlets they blame the users for not reading the terms of agreement.
Researchers that the Department of Defense is funding to understand how ideas and news goes viral are doing very similar work for Facebook…
Nafeez Ahmed writes that the Pentagon is preparing for mass civil breakdown. “A US Department of Defense (DoD) research programme is funding universities to model the dynamics, risks and tipping points for large-scale civil unrest across the world … Social science is being militarised to develop ‘operational tools’ to target peaceful activists and protest movements.”
http://www.theguardian.com/environment/earth-insight/2014/jun/12/pentagon-mass-civil-breakdown
Total information awareness in our Facebook nation.
Your tax dollars at work–funding the destruction of your own civil liberties, paying the salaries of your own oppressors. Fortunately for me, I have a dark sense of humor and can laugh about this sort of thing.
Idiots. Utter, total, idiots at the Pentagon.
I’ve studied a lot of history. I know a lot about the dynamics, risks, and tipping points for large-scale civic unrest around the world.
The absolute worst thing that they can try to do is to try to “control” it directly in any way, with troops or police. That’s a guaranteed backfire. A sensible person would only do that if you were *trying* to get the government overthrown. It’s the absolute worst possible thing to do.
The second-worst is to try to manipulate people with dishonest propaganda. It works until it backfires, but when it backfires, it backfires in such a way as to completely destroy all propaganda power. By the way, this has already happened in the US (the backfire).
The best response is, of course, to figure out what the large groups of people are upset about and throw them a few bones by addressing some of their actual problems. That actually works nearly all the time.
The complete mindless idiots running these Pentagon studies seem not to have considered this. General Sherman considered such “political” solutions to problems, which is why he was a great general.
Facebook is simply one part of a technology which people are free to use or not use. People can use it just to stay in touch with friends or they can put their entire lives online – the wannabe celebrity looking for attention. I am stunned by how trusting people are. It’s like Edward Snowden never existed. Essentially it is a matter of convenience. People enjoy doing their FB thing and the threat of lack of secrecy isn’t considered great enough for them to change. In a way FB serves as a window on how people in our society really are. If our elites ever needed confirmation that they can get away with just about anything they need only glance at the narcissism and apathy on FB.
Wrong. FB is much more like your electricity utility board. Try living without them. You can, but not easily nor well in the USA. FB experiments are like your electricity company playing around with your voltage and total wattage consumption.
Notice how the MSM and many “independent sites” have already dropped the Facebook story ? That in itself shows you just how powerful Facebook and its support from TPTB actually is. Independent sites are running out of fear … Thank You Naked Capitalism …
PS … I have noticed that the connectivity for Naked Capitalism has been weak compared to all the other sites I use on my PC …
There’s a saying in computer science that every problem in CS can eventually be solved with another level of indirection. If the government’s directly collecting personal data and running experiments, people would probably be up in arms. Put a corporation in front of the government though …. and the result is different. Facebook users deserves this and more.
NotSoSure, (are you related to future VPOTUS Not Sure, perhaps?) The Twelve Networking Truths don’t just apply to data handling systems, and I’m glad to see others recognize that too. At this rate, I’m glad to see others still have the capacity to generalize knowledge in ways that offend the royal noses of the elites.
per post: “unlikely to have any meaningful impact on Facebook in the US”
How does having a Facebook ‘like’ button on a website affect the business model ? Can it be removed without hurting the broadcast or revenues ?
Testings as such with code goes on all the time like this at Google, Facebook, United Healthcare, and more as it’s what data scientists and developers do looking for value, that’s the key word and value today pretty much means money. I got on an online Sunday morning and saw bright, intelligent people retweeting this and going nuts and the first thing I said is “tha’ts not science” it’s data scientist full of himself wanting some attention and doing a study, and mind you this was two years ago this was done. Think of what’s going on today. Code hosing is nothing new and how do you think things get developed. A few years ago Homeland Security took their DOD box into a mall that reads minds and encroached on unknowing passers by to see what they received from their device. These are tests mind you and error factors can be high as well. The web is 61% bots and here’s a good video from physicist Sean Gourley that show you in the second half that your value on the web as a human is 1-2 cents while a bot is worth $100 on Facebook. Again if the data scientist has not gotten so full of himself we would have never seen this.
http://ducknetweb.blogspot.com/2014/03/algorithms-that-exploit-sean-gourley.html
The value in data selling, an epidemic in the US is driving this in part too. United Healthcare can’t get enough of this stuff and has duped people for years with formulas and math. Did you read the story about the hospital now taking in your Axciom and Master Card records..you should as folks are being judged and analyzed on flawed data. It was powerful enough to draw Acxiom out on twitter to challenge me a bit.
http://ducknetweb.blogspot.com/2014/06/oh-crap-now-hospitals-are-now-buying.html
It’s time to wake up and realize there is a lot of intelligence operating way above the levels of comprehension of government and that is control. Google is the #2 money spender with lobbying and it’s almost scary in the fact they have learned how to work the GOP, and they work both parties, but particularly the GOP to hose and get what they want, they have the total psych thing going in DC too.
Again wake up time folks and realize this is happening all around you and why I curated some smart folks on the Killer Algorithm page to help educate and inform folks on how this works. Code hosing has been around for a long time. Like I said, there’s no magic here as code what it’s programmed to do and again I used to be a developer myself so take notice if you will. If it’s not here, then look at the markets, lot of code manipulation been going on there for years and made many rich.
http://www.ducknet.net/attack-of-the-killer-algorithms/
Like I said Sunday morning I saw the outrage on Twitter and folks “really” thought this was science when in fact it’s just another data scientist playing with some algorithms to formulate some code that can stand to make some more for them and you don’t have the value that the bots do, and yes it is worth learning about for sure so you don’t get duped.
So how many people have stopped using FB because of this? I’d say zero. Can’t expect FB to care if users don’t care.
Exactly. As I said, FB users deserve all these and more.
At least one found it the last straw. The vapidity of the whole enterprise (and of my friends’ friends) didn’t help.
Consumers find it unsettling to realize they are being continuously manipulated by the media around them.
They need to realize that “marketing” communication (which is facebook entirely) is adversarial in nature. They aren’t on your side. And they have no respect for you either.
Facebook can change my mood very easily. Every time it does something that is anything more than a mindless implementation of a rule that I control (such as: “Show me all posts of type X from user Y”) then my mood immediately changes to “suspicious”.
The idea that anybody would trust Facebook enough to curate a news feed for them is astonishing to me. I’d sooner pet a scorpion.
Like many I have outrage at the revelations of the FB research which I read about DAYS before it started making the rounds of the major blogs and MSM. (Yes, I read online that much.) Funniest thing about it is FB and DoD believe that we’re all such a bunch of rubes that we can be manipulated so easily. DoD is all “hey, we gonna own the the mechanics of psychological contagion” with this research. The hubris of the players is truly a riot! Little do they understand FB is a massive fun personal vanity exercise. Most of the time our news and ideas never go viral…on rare occasions they do. And it’s happening all the time in the MSM every day! Remember Judith Miller? Remember “weapons of mass destruction in Iraq” from Colin Powell. We’ve been there, done that. So DoD thinks they’re gonna put a halt to the future armageddon of civil unrest that might happen someday…and they think it’s all gonna unfold on FB. HA! Jesus. As bugs bunny would say, “What a bunch of maroons!”
Well, hey insurance companies when you use their call centers use software that records and analyzes your “current state of mind” it gets scored and probably that’s up for sale too with all the behavioral analytics folks out there. It sucks, but it is what it is. This is even worse than Facebook if you will as unless you read below and see what goes on, you’re duped every time you hear “this call may be recorded”..
http://ducknetweb.blogspot.com/2014/05/this-call-may-be-recorded-for-quality.html
Furthermore, do you like insurers and others, and now a hospital digging through your MasterCard transactions? They are doing it so again same thing here except now, unlike Facebook it’s personal and comes direct down to “scoring you” not the masses. I’m a privacy advocate and I’m working on a project to do step one here and get Congress to pass a law that requires all data sellers to buy a license. Why? So we know who they all are as your data is being bought and sold both companies you don’t even know about and it’s making money, big money, billions. Banks sell your data, Walgreeens makes over a billion a year, just selling data..big money folks and I can’t comment on what the Facebook apps folks do with having access to your data, but it’s bigger than anything else out there.
Would you not like to have a look capability to see who sells what kind of data to who? That’s what the plan is as step one. All privacy efforts have and will continue to fail without an index and that’s what a law to require a license to distribute data would create, a database of who they all are. If that is not done, forget it, we’ll have more years of futile efforts by regulators as they dont know who to chase.
Here’s my campaign and I’m doing a small funding effort here and I already have years into this as well and I am trying to push it forward so $5 or anything you want to kick in would be great.
http://www.gofundme.com/auyxd0
I used to tell my facebooking co-workers at work that Facebook was a CIA conspiracy to trick users into preparing vast dossiers on themselves for government’s convenience. I’m sure I was wrong in detail but correct in general principle.
Did innocent naive facebookers deserve to be experimented on? Reasonable people can reasonably disagree. But adding this news of experiment to Facebook’s openly notorious policy of “No privacy for you!” means that every person who fails to cancel and terminate their involvement with Facebook as of today going forward, absolutely and positively deserves every bad thing that happens to them as a consequence of remaining on Facebook. The same principle holds for every other anti-privacy you-are-the-product private profit Social Media bussiness out there.
Survival is a privilege, not a right. And life is a gift, not a reward.