Michael Conklin (Angelo State U Business Law) and Christopher Houston (Angelo State U) have posted “Measuring the Rapidly Increasing Use of Artificial Intelligence in Legal Scholarship” on SSRN. Here is the abstract:

The rapid advancement of artificial intelligence (AI) has had a profound impact on nearly every industry, including legal academia. As AI-driven tools like ChatGPT become more prevalent, they raise critical questions about authorship, academic integrity, and the evolving nature of legal writing. While AI offers promising benefits—such as improved efficiency in research, drafting, and analysis—it also presents ethical dilemmas related to originality, bias, and the potential homogenization of legal discourse.

One of the challenges in assessing AI’s influence on legal scholarship is the difficulty of identifying AI-generated content. Traditional plagiarism-detection methods are often inadequate, as AI does not merely copy existing text but generates novel outputs based on probabilistic language modeling. This first-of-its-kind study uses the existence of an AI idiosyncrasy to measure the use of AI in legal scholarship. This provides the first-ever empirical evidence of a sharp increase in the use of AI in legal scholarship, thus raising pressing questions about the proper role of AI in shaping legal scholarship and the practice of law. By applying a novel framework to highlight the rapidly evolving challenges at the intersection of AI and legal academia, this Essay will hopefully spark future debate on the careful balance in this area.

Daniel Schwarcz (U Minnesota Law) et al. have posted “AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice” on SSRN. Here is the abstract:

Generative AI is set to transform the legal profession, but its full impact remains uncertain. While AI models like GPT-4 improve the efficiency with which legal work can be completed, they can at times make up cases and “hallucinate” facts, thereby undermining legal judgment, particularly in complex tasks handled by skilled lawyers. This article examines two emerging AI innovations that may mitigate these lingering issues: Retrieval Augmented Generation (RAG), which grounds AI-powered analysis in legal sources, and AI reasoning models, which structure complex reasoning before generating output. We conducted the first randomized controlled trial assessing these technologies, assigning upper-level law students to complete six legal tasks using a RAG-powered legal AI tool (Vincent AI), an AI reasoning model (OpenAI’s o1-preview), or no AI. We find that both AI tools significantly enhanced legal work quality, a marked contrast with previous research examining older large language models like GPT-4. Moreover, we find that these models maintain the efficiency benefits associated with use of older AI technologies. Our findings show that AI assistance significantly boosts productivity in five out of six tested legal tasks, with Vincent yielding statistically significant gains of approximately 38% to 115% and o1-preview increasing productivity by 34% to 140%, with particularly strong effects in complex tasks like drafting persuasive letters and analyzing complaints. Notably, o1-preview improved the analytical depth of participants’ work product but resulted in some hallucinations, whereas Vincent AI-aided participants produced roughly the same amount of hallucinations as participants who did not use AI at all. These findings suggest that integrating domain-specific RAG capabilities with reasoning models could yield synergistic improvements, shaping the next generation of AI-powered legal tools and the future of lawyering more generally.

Andrew M. Perlman (Suffolk U Law) has posted “Generative AI and the Future of Legal Scholarship” on SSRN. Here is the abstract:

Since ChatGPT’s release in November 2022, legal scholars have grappled with generative AI’s implications for the law, lawyers, and legal education. Articles have examined the technology’s potential to transform the delivery of legal services, explored the attendant legal ethics concerns, identified legal and regulatory issues arising from generative AI’s widespread use, and discussed the impact of the technology on teaching and learning in law school.

By late 2024, generative AI has become so sophisticated that legal scholars now need to consider a new set of issues that relate to a core feature of the law professor’s work: the production of legal scholarship itself.

To demonstrate the growing ability of generative AI to yield new insights and draft sophisticated scholarly text, the rest of this piece contains a new theory of legal scholarship drafted exclusively by ChatGPT. In other words, the article simultaneously articulates the way in which legal scholarship will change due to AI and uses the technology itself to demonstrate the point.

The entire piece, except for the epilogue, was created by ChatGPT (OpenAI o1) in December 2024. The full transcript of the prompts and outputs is available here,https://chatgpt.com/share/676cc449-af50-8002-9145-efbfdf8ebb02, but every word of the article was drafted by generative AI. Moreover, there was no effort to generate multiple responses and then publish the best ones, though ChatGPT had to be prompted in one instance to rewrite a section in narrative form rather than as an outline.

The methodology for generating the piece was intentionally simple and started with the following prompt:

“Develop a novel conception of the future of legal scholarship that rivals some of the leading conceptions of legal scholarship. The new conception should integrate developments in generative AI and explain how scholars might use it. It should end with a series of questions that legal scholars and law schools will need to address in light of this new conception.”

After ChatGPT provided an extensive overview of its response, it was asked to generate each section of the piece using text “suitable for submission to a highly selective law review.” The first such prompt asked only for a draft of the introduction. The introduction identified four parts to the article, so ChatGPT was then asked to draft Parts I, II, III and IV in separate prompts until the entire piece was completed. Because of output limits that restrict how much content can be generated in response to a single prompt, each section of the article is relatively brief. A much more thorough version of the article could have been generated if ChatGPT had been prompted to create each sub-part of the article separately rather prompting it to produce entire parts all at once.

The epilogue offers my own reflections on the resulting draft, which (in my view) demonstrates the creativity and linguistic sophistication of a competent legal scholar. Of course, as with any competent piece of scholarship, the article has gaps and flaws. In other words, it is far from perfect. But then again, very few pieces of legal scholarship are otherwise. Rather than focusing on these flaws, scholars should consider the profound implications of these new tools for the scholarly enterprise. I discuss some of those implications in the epilogue, but apropos of the theme of the piece, generative AI has some useful ideas for us to consider in this regard.

Nachman N. Gutowski (U Nevada) and Jeremy Hurley (Appalachian Law) have posted “Forging Ahead or Proceeding with Caution; Developing Policy for Generative Artificial Intelligence in Legal Education” (Forthcoming, University of Louisville Law Review Spring 2025) on SSRN. Here is the abstract:

Generative Artificial Intelligence is rapidly being integrated into every facet of society, including a growing impact on law schools. It has become abundantly clear that there is a need to develop clear governing policies for its use and adoption in legal education. This article offers an introductory analysis of related approaches currently taken in various law schools, exploring the factors influencing these policies and their ethical implication. A comparative review of institutional policies reveals both similarities and unique approaches. Common themes include the need for balance between limited use and outright reliance, as well as the need for transparency and the promotion of academic integrity. Similarly, additional recurring concerns and considerations are explored, such as the potential impact on curricular integration and academic rigor.

Ethical and professional implications surrounding using these tools and platforms in legal education set the stage; delving into the importance of understanding the limitations and risks, a discussion of educating students about the appropriate contexts for using AI as a learning tool is presented. Additionally, the unique role of law school faculty governance in shaping these policies is explored, emphasizing the critical decision-making processes involved in establishing enforceable and implementable guardrails and guidelines. By looking at the focus behind policies across multiple institutions, best practices and approaches begin to emerge. Takeaways include future implications and recommendations for law schools and faculty in effectively governing the emerging use of generative artificial intelligence in legal education. The implications go beyond the walls of academia and impact practicing attorneys significantly. To prepare for this reality, law schools must think carefully about, and generate policy approaches in line with universal goals and considerations. This article aims to provide valuable insights and recommendations for prudent governance, ultimately contributing to the ongoing discourse on its responsible and effective use within the legal academic sphere.

Amy Emerson (Villanova U Charles Widger Law) has posted “Assessing Information Literacy in the Age of Generative AI: A Call to the National Conference of Bar Examiners” on SSRN. Here is the abstract:

Information literacy is crucial to satisfying a lawyer’s duty of technology competence by virtue of its inherent role in conducting legal research-a skill now recognized by the National Conference of Bar Examiners (NCBE) as a priority as it prepares for the NextGen Bar Exam. In light of the rapid rise in the number of attorneys facing disciplinary issues across the country, it is the NCBE’s responsibility to draw upon its rich history to address information literacy as a technological competency on the Multistate Professional Responsibility Exam to protect the public from newly licensed lawyers’ incompetent use of generative artificial intelligence.

Matthew Dahl (Yale Law School) has posted “Bye-bye, Bluebook? Automating Legal Procedure with Large Language Models” on arXhiv. Here is the abstract:

Legal practice requires careful adherence to procedural rules. In the United States, few are more complex than those found in The Bluebook: A Uniform System of Citation. Compliance with this system’s 500+ pages of byzantine formatting instructions is the raison d’etre of thousands of student law review editors and the bete noire of lawyers everywhere. To evaluate whether large language models (LLMs) are able to adhere to the procedures of such a complicated system, we construct an original dataset of 866 Bluebook tasks and test flagship LLMs from OpenAI, Anthropic, Google, Meta, and DeepSeek. We show (1) that these models produce fully compliant Bluebook citations only 69%-74% of the time and (2) that in-context learning on the Bluebook’s underlying system of rules raises accuracy only to 77%. These results caution against using off-the-shelf LLMs to automate aspects of the law where fidelity to procedure is paramount.

Lea Bishop (Yale U Yale Information Society Project) has posted “Generative” on SSRN. Here is the abstract:

To expand our collective imagination about the potential, limits, and risks of artificial intelligence, we cannot simply read about it. We must experience it first-hand. This premise makes Generative different from anything else you have read about artificial intelligence.

In an entertaining and thought-provoking package, Generative departs from the mainstream narratives of hype and fear to challenge readers’ assumptions about what ChatGPT and its cousins can (and cannot) do. Through a series of playful Socratic dialogues with actual AI chatbots, Generative vividly demonstrates the technology’s creative power.

Readers will come away with new insights on topics like plagiarism, critical thinking, innovation, education, and the future of artificial intelligence. Like the original Dialogues of Socrates and Plato, Generative’s powerful model of critical inquiry will inspire readers to question our assumptions about artificial intelligence… and everything else.

Anna Conley (U Montana Alexander Blewett III Law) has posted “Understanding the Duty of Competence for Attorneys Using Generative AI” on SSRN. Here is the abstract:

Ethical duties appear poised to be the primary regulatory tool for responsible use of generative AI (“GAI”) by attorneys. This reality necessitates a clear understanding of what the duty of competence requires for attorneys using GAI. Recent state bar and American Bar Association (“ABA”) guidance have coalesced around a foundational concept of informed decision-making, which requires that attorneys have sufficient knowledge about the GAI tool they are using and the specific task at hand to make an informed decision that employing the tool for that task is in the client’s best interests. Competence also requires attorneys avoid automation bias and mitigate against GAI tools’ limitations, including not only hallucinations but also incomplete, inaccurate and misgrounded outputs. Recent ethical guidance requires that attorneys retain cognitive agency when completing tasks that require human judgment and reasoning. These tasks render GAI’s “black box” or lack of explainability, particularly problematic and require attorneys to resist automation complacency, which refers to humans’ reduced capacity to understand or complete the tasks for which they rely on GAI.

Catherine Gage O’grady (U Arizona James E. Rogers College Law) and Casey O’grady (Harvard U Harvard Law) have posted “Agentic Workflows in the Practice of Law—AI Agents as Ethics Counsel” (Agentic Workflows in the Practice of Law—AI Agents as Ethics Counsel, 39 Geo. J. Legal Ethics (forthcoming 2025)) on SSRN. Here is the abstract:

Generative AI is reshaping legal practice as law firms invest in AI technology and prepare for a future where AI agents operate alongside human lawyers. While such a future raises numerous ethical concerns, it also opens new opportunities for ethical practice. This article explores the potential for AI agents to improve ethical decision-making within legal practices. We start by defining key concepts such as AI agents and agentic workflows. We then provide a brief overview of the role of ethics counsel in law firms and discuss critical behavioral challenges in the human practice of law.  Finally, we introduce a new model of AI agents acting as dedicated ethics counsel. These agents should be specialized, accountable, and systematic to provide comprehensive ethical guidance in a firm’s legal workflows. These ethical agents have the potential to mitigate the human biases in traditional legal practice and offer an efficient and scalable approach to ensure ethical compliance in an increasingly AI-driven legal landscape. To demonstrate this model, we end with real-world examples of what initial ethical agent contributions could look like in practice.

Eric A. Posner (U Chicago Law) and Shivam Saran (U Chicago Law) have posted “Judge AI: Assessing Large Language Models in Judicial Decision-Making” on SSRN. Here is the abstract:

Abstract. Can large language models (LLMs) replace human judges? By replicating a prior 2 x 2 factorial experiment conducted on 31 U.S. federal judges, we evaluate the legal reasoning of OpenAI’s GPT-4o. The experiment involves a simulated appeal in an international war crimes case, with two altered variables: the degree to which the defendant is sympathetically portrayed and the consistency of the lower court’s decision with precedent. We find that GPT-4o is strongly affected by precedent but not by sympathy, similar to students who were subjects in the same experiment but the opposite of the professional judges, who were influenced by sympathy. We try prompt engineering techniques to spur the LLM to act more like human judges, but with no success. “Judge AI” is a formalist judge, not a human judge.