Raymond H. Brescia (Albany Law) has posted “What’s a Lawyer For?: Artificial Intelligence and Third-Wave Lawyering” (FSU Law Review, Forthcoming) on SSRN. Here is the abstract:

The American legal profession is at a critical inflection point, one that will likely result in dramatic changes in the ways in which consumers access legal guidance and the manner in which lawyers and others deliver it. Chat-enabled artificial intelligence, algorithmic decision-making, digitization, and commoditization threaten existing practices within the legal profession as it is currently constituted by making legal services and information easier to deliver, less expensive to provide, and less difficult for consumers to access. New technologies could lower the cost of legal services generally and make many forms of legal information easier to disseminate, and, as a result, more widely distributed. Because of this, more consumers are likely to gain access to some type or form of legal assistance, even if it does not mean they will necessarily receive the direct services of a lawyer. It likely also means that the traditional methods by which legal services have been delivered will become obsolete in at least some contexts, and with that, many traditional legal services jobs and careers as well. This will, of course, have a dramatic impact on what lawyers do, who delivers services that look like legal services, what law students learn, and what law schools teach. Much could easily be lost as guidance to address legal problems is digitized, commoditized, and delivered in accessible and affordable ways, just not by lawyers. At critical inflection points in the American legal profession’s history, it has responded to demands from within and outside the profession to address the ways in which the profession was not serving its appropriate functions in society and failing to uphold what should be its values. At one of the more significant of these inflection points, which occurred in the turn of the 19th to the 20th century, the profession went through dramatic change: moving it from what I call the profession’s “first wave,” where a loosely organized bar made up almost exclusively of white men of Northern European descent faced few barriers to entry to the profession, to its “second wave,” when the profession erected significant barriers to entry and institutionalized such barriers in an effort to maintain greater control over the practice of law. I argue here that we are on the cusp of what may be a new “wave”—a third wave—where technology impacts the practice of law and the ways in which consumers access legal assistance in dramatic ways. But to change for the sake of change alone is not a good enough reason to applaud the coming disruptions in the delivery of legal services due to new technologies. Any profession promotes a set of professional values and serves a particular role in society. The legal profession, like any profession, should serve its appropriate role in society; it should fulfill its purpose. A critical question for the profession, and society at large, is whether new technologies undermine that role or advance it. What is lost and what is gained with respect to the values the profession is supposed to uphold and those functions it is supposed to fill when new technologies might displace traditional modes of delivering legal services? To answer these questions, one must first conduct an assessment of the values and functions of the American legal profession. Once such an assessment is complete, one can embark upon a broader effort, one that reviews the ways in which new technologies are being deployed, and will be deployed in the future, and calibrate such uses in ways that advance a purpose-driven legal services model in a technology-enhanced legal ecosystem. What I hope to accomplish in this essay is to lay out the parameters of the debate around the coming disruptions to the delivery of legal services due to emerging technologies and identify the considerations that should go into any assessment of the proper role that the legal profession should play in the wake of this current inflection point.

Mark Shope (National Yang Ming Chiao Tung University; Indiana University Robert H. McKinney School of Law) has posted “GPT Performance on the Bar Exam in Taiwan” on SSRN. Here is the abstract:

This paper reports the performance of the GPT-4 Model of ChatGPT Plus (“ChatGPT4”) on the multiple-choice section of the 2022 Lawyer’s Bar Exam in Taiwan. ChatGPT4 outperforms approximately half of human test-takers on the multiple-choice section with a score of 342. This score, however, would not advance a test taker to the second and final essay portion of the exam. Therefore, this paper will not include an evaluation of ChatGPT4’s performance on the essay portion of the exam.

Erdem Dogukan Yilmaz (Erasmus Univeristy Rotterdam), Ivana Naumovska (INSEAD), and Vikas A. Aggarwal (INSEAD) have posted “AI-Driven Labor Substitution: Evidence from Google Translate and ChatGPT” on SSRN. Here is the abstract:

Although artificial intelligence (AI) has the potential to significantly disrupt businesses across a range of industries, we have limited empirical evidence for its substitution effect on human labor. We use Google’s introduction of neural network-based translation (GNNT) in 2016-2017 as a natural experiment to examine the substitution of human translators by AI in the context of a large online labor market. Using a difference-in-differences design, we show that the introduction of GNNT reduced the number of (human translation) transactions at both the overall market and individual translator levels. In addition, we show that GNNT had a stronger effect on translation tasks with analytical elements, as compared to those with cultural and emotional elements. In supplemental analyses, we document a similar pattern after the launch of ChatGPT using question and answer patterns in Stack Exchange forums. Our study thus offers robust and causal empirical evidence for a heterogeneous substitution effect of human tasks by skilled knowledge workers. We discuss the relevance of our findings for research on competitive advantage, technology adoption, and strategy microfoundations.

Rupert Macey-Dare (St Cross College – University of Oxford; Middle Temple; Minerva Chambers) has posted “How ChatGPT and Generative AI Systems will Revolutionize Legal Services and the Legal Profession” on SSRN. Here is the abstract:

In this paper, ChatGPT, is asked to provide c.150+ paragraphs of detailed prediction and insight into the following overlapping questions, concerning the potential impact of ChatGPT and successor generative AI systems on the evolving practice of law and the legal professions as we know them:

• Which are the individual legal business areas where ChatGPT could make a significant/ transformative impact and reduce costs and increase efficiencies?
• Where can ChatGPT use its special NLP abilities to assist in legal analysis and advice?
• Which are the specific areas where generative AI systems like ChatGPT can revolutionize and improve the legal profession?
• How can systems like ChatGPT help ordinary people with legal questions and legal problems?
• What is the likely timeframe for ChatGPT and other generative AI systems to transform legal services and the legal profession?
• What are the potential implications for new and intending law students?
• How will ChatGPT and similar systems impact professional lawyers in future?

Some of ChatGPT’s key insights and predictions (see full paper attached for detailed responses and analysis) are as follows:

ChatGPT identifies the following key individual legal business areas where it could make a significant/ transformative impact and reduce costs and increase efficiencies: Alternative dispute resolution, Automated billing, Case analysis, Case management, Compliance monitoring, Contract management, Contract review, Document automation, Document review, Discovery and E-discovery, Drafting legal documents, Due diligence, Expertise matching, Intellectual Property and IP management, Legal advice, Legal analytics, Legal chatbots, Legal drafting, Legal document review, Legal education, Legal marketing, Legal research, Litigation support, Natural language processing (NLP), Patent analysis, Predictive analytics, Regulatory compliance, Research, Risk assessment, Training and education, Translation and Virtual assistants.

ChatGPT flags up its special NLP abilities to assist in legal analysis and advice, particularly in the following key areas: Contract analysis, Document classification, Document summarization, Due diligence, Legal chatbots, Legal document review, Legal document summarization, Legal drafting, Legal language translation, Legal research, Named entity recognition, Predictive analytics, Regulatory compliance, Sentiment analysis and Topic modelling.

On the question of which are the specific areas where generative AI systems like ChatGPT can revolutionize and improve the legal profession, ChatGPT identifies: Accessibility, Accuracy, Collaboration, Cost reduction, Customization, Decision-making, Efficiency, Error-reduction, New business and innovation, Job displacement potential, Legal research, Risk management and Scalability.

On the question of how can systems like ChatGPT help ordinary people with legal questions and legal problems, ChatGPT identifies the following areas: 24/7 availability, Automated legal services, Consistency of advice, Contract review, Cost-effectiveness, Court filings, Customization, Document preparation, Education, Empowerment, Faster response times, Language translation, Legal advice, Legal chatbots, Legal education, Legal research, Mediation and dispute resolution, Privacy, Scalability and Simplified language.

On the question of the likely timeframe for ChatGPT and other generative AI systems to transform legal services and the legal profession, Chat GPT comments that “It is difficult to predict with certainty, as it will depend on a variety of factors such as technological advancements, regulatory changes, and market demand. [However] There are several reasons to believe that the transformation of legal services through generative AI systems like ChatGPT will happen relatively quickly…the transformation of legal services through generative AI systems is likely to happen relatively quickly, potentially within the next 5-10 years.”

On the potential implications for new and intending law students, ChatGPT comments that: “It is essential for students contemplating studying law to be aware of the ability of ChatGPT and generative AI systems to perform legal jobs instead of humans in the future. These technologies are already transforming the legal industry, and their impact is likely to continue to grow in the years to come. Understanding how these systems work and how they can be used in legal practice will be crucial for anyone seeking a career in law, and may also provide a competitive advantage in the job market. By understanding the potential impact of these technologies on the legal industry, students can better prepare themselves for the changing nature of legal work and take advantage of the new opportunities that are emerging as a result of these developments.”

On the all-important question for lawyers of how ChatGPT and similar systems will impact professional lawyers in future, ChatGPT comments that: “Generative AI systems like ChatGPT have the potential to significantly impact the demand and payment for professional lawyers in the future. As these systems become more advanced and capable of performing a wider range of legal tasks, it is likely that they will begin to replace some of the work that is currently performed by human lawyers.”

“One area where this is already happening is in document review. AI systems can review and analyze large volumes of documents much more quickly and accurately than human lawyers, which can save significant time and cost for law firms and their clients. As these systems become more sophisticated, they may also be able to perform other tasks such as legal research and analysis, drafting legal documents, and even providing legal advice.”

“The impact of generative AI systems on the demand for lawyers will depend on the specific tasks and areas of law that they are able to perform. It is likely that some areas of law will be more heavily impacted than others. For example, areas such as contract law and intellectual property may be more susceptible to automation, while litigation and dispute resolution may be less so.”

“In terms of payment for professional lawyers, the impact of generative AI systems is also likely to vary depending on the specific tasks and areas of law. In some cases, these systems may allow lawyers to perform their work more quickly and efficiently, which could potentially lead to higher billable hours and increased income. However, if these systems are able to replace some of the work that is currently performed by human lawyers, it could also lead to a reduction in demand for these services and a decrease in fees.”

“One potential impact of systems like ChatGPT on the legal industry is a reduction in the demand for certain types of legal work that can be automated or performed more efficiently by AI systems. For example, tasks like document review, contract drafting, and legal research may be performed more accurately and quickly by AI systems than by humans, leading to a decrease in the number of lawyers needed to perform these tasks.”

“It is also possible that the development of AI systems like ChatGPT will lead to changes in the way that legal services are priced and delivered. As these technologies become more common, it is likely that clients will begin to expect lower costs and faster turnaround times for certain types of legal work. This could lead to increased competition among legal service providers, which in turn could put pressure on lawyers to lower their rates or find ways to deliver legal services more efficiently….it is clear that these technologies have the potential to significantly change the legal industry, and that lawyers will need to adapt in order to remain competitive and relevant in a rapidly changing market. This may involve developing new skills and knowledge related to working alongside AI systems, or focusing on areas of law that are less susceptible to automation.”

Interestingly, although ChatGPT does discuss practical contract management, IP and evidence, it does not seem to predict inroads being made into academic legal analysis, statutory construction, complex case analysis or the development of new legal thinking and principles, so not into the theoretical domain of law professors and senior lawyers and judges, (although there are additional reasons why there are likely to be knock-on reductions in demand for these specialist lawyers too).

But for the vast majority of procedural (routinely turning-the-handle type) practitioner law and practice, ChatGPT seems to be predicting a seismic sectoral shock, a reduction in human-centric legal work, an increase in legal self-help for clients and the public, and a technological transformation in and fundamental repricing and manpower shock for the legal sector within a timeframe of 5-10 years.

N.B. This is only one set of predictions, which could prove right or wrong, indeed from an unconscious chatbot machine ChatGPT. However it has the credibility of being made based on both a huge body of knowledge data, and on the consistent rules programmed into ChatGPT itself, and by apparently coherently reasoned responses. Time will soon tell of course…

Tammy Pettinato Oltz (University of North Dakota School of Law) has posted “ChatGPT, Professor of Law” on SSRN. Here is the abstract:

Although ChatGPT was just released by OpenAI in November 2022, legal scholars have already been delving into the implications of the new tool for legal education and the legal profession. Several scholars have recently written fascinating pieces examining ChatGPT’s ability to pass the bar, write a law review article, create legal documents, or pass a law school exam. In the spirit of those experiments, I decided to see whether ChatGPT had potential for lightening the service and teaching loads of law school professors.

To conduct my experiment, I created an imaginary law school professor with a tough but typical week of teaching- and service- related tasks ahead of her. I chose seven common tasks: creating a practice exam question, designing a hand-out for a class, writing a letter of recommendation, submitting a biography for a speaking engagement, writing opening remarks for a symposium, developing a document for a law school committee, and designing a syllabus for a new course. I then ran prompts for each task through ChatGPT to see how well the system performed the tasks.

Remarkably, ChatGPT was able to provide useable first drafts for six out of seven of the tasks assigned in only 23 minutes. Overall and unsurprisingly, ChatGPT proved to be best at those tasks that are most routine. Tasks that require more sophistication, particularly those related to teaching, were harder for ChatGPT, but still showed potential for time savings.

In this paper, I describe a typical work scenario for a hypothetical law professor, show how she might use ChatGPT, and analyze the results. I conclude that ChatGPT can drastically reduce the service-related workload of law school faculty and can also shave off time on back-end teaching tasks. This freed-up time could be used to either enhance scholarly productivity or further develop more sophisticated teaching skills.

Stuart Hargreaves (The Chinese University of Hong Kong (CUHK) – Faculty of Law) has posted “‘Words Are Flowing Out Like Endless Rain Into a Paper Cup’: ChatGPT & Law School Assessments” on SSRN. Here is the abstract:

ChatGPT is a sophisticated large-language model able to answer high-level questions in a way that is undetectable by conventional plagiarism detectors. Concerns have been raised it poses a significant risk of academic dishonesty in ‘take-home’ assessments in higher education. To evaluate this risk in the context of legal education, this project had ChatGPT generate answers to twenty-four different exams from an English-language law school based in a common law jurisdiction. It found that the system performed best on exams that were essay-based and asked students to discuss international legal instruments or general legal principles not necessarily specific to any jurisdiction. It performed worst on exams that featured problem-style or “issue spotting” questions asking students to apply an invented factual scenario to local legislation or jurisprudence. While the project suggests that for the most part conventional law school assessments are for the time being relatively immune from the threat ChatGPT brings, this is unlikely to remain the case as the technology advances. However, rather than attempt to block students from using AI as part of learning and assessment, this paper instead proposes three ways students may be taught to use it in appropriate and ethical ways. While it is clear that ChatGPT and similar AI technologies will change how universities teach and assess (across disciplines), a solution of prevention or denial is no solution at all.

Rebecca Kunkel (Rutgers Law) has posted “Artificial Intelligence, Automation, and Proletarianization of the Legal Profession” (Creighton Law Review, Vol. 56, 2022) on SSRN. Here is the abstract:

Recent advances in computer programming, broadly categorized as “artificial intelligence,” (“Al”) have renewed debates over machines as viable replacements for human lawyers. Some prominent lawyers and legal scholars now adhere to a vision of the future heavily seasoned with Silicon Valley-style techno-utopianism: the legal profession may endure but only in a form in which it would be almost unrecognizable today, while legal innovators will need to immerse themselves in the possibilities opened up by artificial intelligence in order to survive. For others, the view of artificial intelligence and its potential application to law is more limited, as they argue for the impossibility of automating many essential aspects of legal service. These views share key assumptions about the nature of Al technology: that technological development follows its own course and that the widespread adoption of technologies is primarily determined by objective measures of efficacy. This essay offers an alternate Marxian account of legal Al which places it in the larger history of automation and proletarianization.

Ilias Chalkidis (University of Copenhagen) has posted “ChatGPT May Pass the Bar Exam Soon, but Has a Long Way to Go for the LexGLUE Benchmark” on SSRN. Here is the abstract:

Following the hype around OpenAI’s ChatGPT conversational agent, the last straw in the recent development of Large Language Models (LLMs) that demonstrate emergent unprecedented zero-shot capabilities, we audit the latest OpenAI’s GPT-3.5 model, ‘gpt-3.5-turbo’, the first available ChatGPT model, in the LexGLUE benchmark in a zero-shot fashion providing examples in a templated instruction-following format. The results indicate that ChatGPT achieves an average micro-F1 score of 49.0% across LexGLUE tasks, surpassing the baseline guessing rates. Notably, the model performs exceptionally well in some datasets, achieving micro-F1 scores of 62.8% and 70.1% in the ECtHR B and LEDGAR datasets, respectively. The code base and model predictions are available at https://github.com/coastalcph/zeroshot_lexglue.

Daniel Martin Katz (Illinois Tech – Chicago Kent College of Law; Bucerius Center for Legal Technology & Data Science; Stanford CodeX – The Center for Legal Informatics; 273 Ventures), Dirk Hartung (Bucerius Law School – Center for Legal Technology and Data Science; Stanford University – Stanford Codex Center), Lauritz Gerlach (Bucerius Law School), Abhik Jana
(University of Hamburg; Language Technology Group, Department of Informatics, Universität Hamburg), and Michael James Bommarito (273 Ventures; Licensio, LLC; Stanford Center for Legal Informatics; Michigan State College of Law; Bommarito Consulting, LLC) have posted “Natural Language Processing in the Legal Domain” on SSRN. Here is the abstract:

In this paper, we summarize the current state of the field of NLP and Law with a specific focus on recent technical and substantive developments. To support our analysis, we construct and analyze a corpus of more than six hundred NLP and Law related papers published over the past decade. Our analysis highlights several major trends. Namely, we document an increasing number of papers written, tasks undertaken, and languages covered over the course of the past decade. We observe an increase in the sophistication of the methods which researchers deployed in this applied context. Slowly but surely, Legal NLP is beginning to match the methodological sophistication of general NLP. We believe this to be a positive trend for the future of the field, but many questions in both the academic and commercial sphere still remain open.

Jonathan H. Choi (U Minnesota Law) et al. have posted “ChatGPT Goes to Law School” on SSRN. Here is the abstract:

How well can AI models write law school exams without human assistance? To find out, we used the widely publicized AI model ChatGPT to generate answers on four real exams at the University of Minnesota Law School. We then blindly graded these exams as part of our regular grading processes for each class. Over 95 multiple choice questions and 12 essay questions, ChatGPT performed on average at the level of a C+ student, achieving a low but passing grade in all four courses. After detailing these results, we discuss their implications for legal education and lawyering. We also provide example prompts and advice on how ChatGPT can assist with legal writing.