Did Facebook Manipulate Your Emotions? Not Really. But Journalists and the Blogosphere Are!

There has been substantial outrage by vocal journalists and Internet denizens since Saturday over the disclosure that Facebook and social scientists from 3 universities collaborated to tweak Facebook’s algorithms to adjust the emotional content of a subset of users’ news feeds for a week in 2012. Interestingly, it wasn’t until 12 days after The Wall Street Journal published a story that didn’t touch on the ethical aspect of the research that others became angry. And that was 6 days after it was covered by the Cornell Chronicle. I pondered the ethical aspect and the implied informed consent in a couple of tweets the same day the WSJ article was published, but my wording wasn’t incendiary, it was approaching midnight, and my social influence on Twitter is fairly limited.

I am an avid reader of research in the areas of psychology, sociology, social media, and behavioral economics so I was intrigued and began reading the research paper. I keyed in on the researchers’ position that the users’ agreement to Facebook’s Data Use Policy constitutes informed consent.

IWC was adapted to run on the Hadoop Map/Reduce system (11) and in the News Feed filtering system, such that no text was seen by the researchers. As such, it was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.

Let’s refer to the Data Use Policy. I’ve been kind enough to link directly to the full version of it. To find it, go to facebook.com, click “Terms” at the bottom of the page, click “Data Use Policy”, click “Information we receive and how it is used”, scroll down to the “How we use the information we receive” section, and after reading through content containing 1,751 other words, you’ll see:

“[we may use the information we receive about you] for internal operations, including troubleshooting, data analysis, testing, research and service improvement.”

Only a tiny percentage of people read websites’ terms of service and privacy policies. And most sites’ relevant documents aren’t as verbose and overwhelming as Facebook’s. Regardless, the use of the single word “research” is ambiguous and to say that constitutes “informed consent” is rather disingenuous and it’s a slippery slope. Where is the line drawn? Would it be OK to correct misspellings, punctuation, and grammar errors? Would it be OK to display a false relationship status for your significant other (it’s complicated!) to gauge your reaction? At least so long as it’s in the name of research?

The Facebook data scientist who was involved in the research, Adam D. I. Kramer, has since said that Facebook has been working on improving its internal review practices. I hope they will consider adopting the Common Rule or something comparable that fits their needs and meets user expectations, adopting transparency about the process and the results. I appreciate that telling the users they were selected to participate in the study (especially if allowed to opt-out) could result in at least one type of bias, poisoning the validity of the results. But there’s little reason they couldn’t have been told afterwards. It’s been more than 2 years since the research. Will they be told now?

Kramer also said:

The goal of all of our research at Facebook is to learn how to provide a better service.

Though Facebook benefits from the research, the research wasn’t intended to allow Facebook to learn how to provide a better service. It was funded by the Army and a private foundation, presumably to determine how exposure to online content influences readers’ emotions. Based on the conclusions drawn, should we expect Facebook to censor, filter, limit, or alter content that it expects may have an adverse impact on our moods?

But what’s lost in the outrage is that the research itself is flawed. Highly flawed. It didn’t actually measure emotions or mood – it was a linguistic analysis of word counts of negative and positive words as a proxy for sentiment. It was based on counting the number of positive words and negative words in the users’ news feeds. The tool, LIWC2007, is a great tool for analyzing lengthy texts like books, but it is not designed to analyse short text like the text typically found in Facebook content. Its usefulness for the types of messages found in Facebook News Feeds is highly suspect.

LIWC2007 simply scans for appearance of a pool of almost 4,500 words and word stems and increments a specific category counter if a match is found. Let’s assume the categories are “negative emotion” and “positive emotion” (it’s actually somewhat more complex than this, but for this discussion this will suffice). In the sentence “I’m not pleased with my happiness.” the words “pleased” and “happiness” will increment the positive emotion counter twice and the word “not” will increment the negative emotion counter once. So a sentence which is clearly negative will not be rated that way. And never mind the sentiment expressed by emoticons, images, and emoji which are much more commonly used in News Feeds than longer texts elsewhere. Sarcasm, slang, abbreviations, quoting of others, and other factors also complicate matters.

And if that’s not enough, the statistical significance of the impact which was discovered is rather insignificant.

When positive posts were reduced inthe News Feed, the percentage of positive words in people’sstatus updates decreased by B = − 0.1% compared with control[t(310,044) = − 5.63, P < 0.001, Cohen’s d = 0.02], whereas thepercentage of words that were negative increased by B = 0.04%(t = 2.71, P = 0.007, d = 0.001). Conversely, when negative postswere reduced, the percent of words that were negative decreasedby B = −0.07% [t(310,541) = −5.51, P < 0.001, d = 0.02] and the percentage of words that were positive, conversely, increased by B = 0.06% (t = 2.19, P < 0.003, d = 0.008).

In other words, a decrease in positive words in a user’s News Feed caused an average of a 0.1% decrease in positive words in the user’s status updates and a 0.04% increase in negative words. A decrease in negative words in a user’s News Feed caused an average of a 0.07% decrease in negative words in the user’s status updates and a 0.06% increase in positive words. That’s hardly significant.

I strongly encourage you to read the full research paper and draw your own conclusions.

You can leave a response, or trackback from your own site.

Leave a Reply

Follow me on Twitter!Follow me on Twitter! Subscribe to RSS Feed Follow me on Twitter!