Evelyn Douek (Harvard Law School) has posted “The Siren Call of Content Moderation Formalism” (New Technologies of Communication and The First Amendment (Bollinger & Stone eds., 2022) on SSRN. Here is the abstract:
Systems of online content moderation governance are becoming some of the most elaborate and extensive bureaucracies in history, and they are deeply imperfect and need reform. Would-be reformers of content moderation systems are drawn to a highly rule-bound and formalistic vision of how these bureaucracies should operate, but the sprawling chaos of online speech is too vast, ever-changing, and varied to be brought into consistent compliance with rigid rules. This essay argues that the quest to make content moderation systems ever more formalistic will not fix public and regulatory concerns about the legitimacy and accountability of how platforms moderate content on their services. The largest social media platforms operate massive unelected, unaccountable, and increasingly complex bureaucracies that decide to act or not act on millions of pieces of content uploaded to their platforms every day. A formalistic model, invoking judicial-style norms of reasoning and precedent, is doomed to fail at this scale and complexity. As these governance systems mature, it is time to be content moderation realists about the task ahead.
AI Law Blawg 