Sarah T. Roberts: Home is No Place for Content Moderation Work

Sarah T. Roberts, assistant professor of information studies at UCLA, published a commentary, “Digital Humanity: Social Media Content Moderation and the Global Tech Workforce in the COVID-19 Era” in Flow Journal on March 19. In it, she outlines the challenges, both technical and ethical, to bringing commercial content moderation more significantly into workers’ homes during the global COVID-19 pandemic. 

In the wake of the Facebook announcement on March 16 that the company was having its employees work at home whenever possible, Roberts writes that the company’s growing reliance on artificial intelligence (AI) and automated moderation systems resulted in the deletion of benign and even newsworthy content about COVID-19.

“Eleven days ago, I communicated with a reporter colleague to give my sense of how a virus-related shutdown in the Philippines could affect American social media giants,” writes Roberts. “I told him that while a lot of the most irritating and highest-volume unwanted content (as deemed by the platforms) can be found and removed by automated tools—here I refer to spam, pornographic content, copyright violations, and other already known-bad material—they tend to be imperfect and blunt instruments whose interventions can be calibrated to be more sophisticated or to cast a wider net.

“But the loss of a major moderation site that would mean a switchover to reliance on these tools would invariably cause disruption in social media’s production chain, and could even potentially lead to quality issues perceived by users. That appears to be the very case … where we see users become frustrated by false positives: cases where overzealous and undersophisticated AI tools aggressively remove reasonable content, because its judgment is too rudimentary.”

Professor Roberts notes that while a lot of CCM was typically done by workers at their homes before the pandemic, there are legal and logistical factors to consider when moving operations almost completely off-site.

“In the case of AAA social media firms … constraints like privacy agreements and data protection policies in various jurisdictions may preclude this,” she writes. “There is also a nontrivial infrastructure that goes into setting up a computing center with requisite hardware, software (internally developed and maintained systems) and routing of data. The call center locations themselves are often highly secure, with nothing allowed on the floor where workers are logged in. Working from home eliminates the ability for oversight and surveillance of workers and their practices, both what they are doing and what they are not, to the extent that can be achieved on-site. This alone is possibly a deal-breaker for moving the work home.”

Roberts also states that the psychological wellbeing of CCM workers is doubly at stake without ready access to moral support from coworkers or from psychological services provided by employers.

“Bringing the work consisting of the rapid review of thousands of images and videos, many of which can be psychologically difficult and taxing, into the home also may be considered too much to ask of workers in a time of crisis,” she writes. “Workers in call centers rely on each other and their teams for support doing commercial content moderation, and may have access to an on-site or on-call therapists, counselors or other mental health professionals.” 

Roberts is the author of “Behind the Screen: Content Moderation in the Shadows of Social Media.” She co-directs the Center for Critical Internet Inquiry at UCLA with Safiya U. Noble, UCLA associate professor of Information Studies. 

Professor Roberts is a member of the American Society for Information Science and Technology, the Association of Internet Researchers, the International Association for Media and Communication Research, the Society for the History of Technology, and the Union for Democratic Communications. She is a 2018 Carnegie Fellow and a 2018 winner of the EFF Pioneer Award.

To read “Digital Humanity: Social Media Content Moderation and the Global Tech Workforce in the COVID-19 Era,” visit this link.

Visit these links for Professor Roberts in The New York Times and The Washington Post.

Photo by Stella Kalinina