Elizabeth E. Joh (UC Davis – School of Law) has posted “The Unexpected Consequences of Automation in Policing” (SMU Law Review 2022 forthcoming) on SSRN. Here is the abstract:
This essay has two aims. First, it explains how automated decisionmaking can produce unexpected results. This is a problem long understood in the field of industrial organization. To identify such effects in policing is no easy task. The police are a notoriously difficult institution to study. They are insular, dislike outsiders, and especially dislike critical outsiders. Fortunately, we have the benefit of a decade’s worth of experimentation in the police use of automated decisionmaking, and the resulting political backlash against some of these uses. As a result, some large urban police departments have undergone external investigations to see whether tools like predictive policing or individual criminal risk assessments are biased or ineffective or simply too costly in light of their benefits. One of these recent reports, on the use of acoustic gunshot detection software in Chicago, provides a window into one type of police automation.
This leads to the article’s second observation. Automation is not just a set of tools that the police use; it changes the environment of policing in unexpected ways. There are now some widely-known criticisms of the increasing use of automated tools in policing, but they focus primarily on the flaws of the technologies used. The training data in facial recognition algorithms may be biased along lines of race, gender, and ethnicity. Risk assessments for gun violence may in truth be poor guides for police intervention. These claims are singularly technology-focused. Accordingly, errors and inefficiencies merit technological improvements. Even calls for bans on technologies like facial recognition are responses to the technology itself. As Chicago’s experience with acoustic gunshot detection technology demonstrates, however, automation serves not just as a tool for the police, but also led to changes in police behavior. These changes in police conduct are documented in a 2021 report from the Chicago Office of Inspector General. And they are noteworthy. If automation unexpectedly changes police behaviors, these changes have implications for how we understand policing through the lens of inequality and unaccountability.