Ep.278 - Peter Oh on AI and Veil Piercing

Speaker A: Foreign. Welcome to the Business Scholarship Podcast, a place for interdisciplinary conversations in the broad world of business research. My name is Andrew Jennings and it's my pleasure to be your host. If you like what you hear today, please subscribe to the podcast on Apple, Spotify or wherever you get your podcast. Plus leave a rating and let other people know about the show too. And if you have ideas for the show, please let me know. My email address is andrewdrewkginnings.com and I look forward to hearing from you. All right, time for the episode. Our guest today is Peter oh, Professor of Law at the University of Pittsburgh. We'll be discussing his article the Large Language of Veil Piercing, which he co authored with Douglas Bernard of mit. The article is forthcoming in the journal Geometrics. I'll link to the article in the show notes for the episode. Peter welcome to the Business Scholarship Podcast.

Speaker B: Thank you, Andrew. Glad to be here.

Speaker A: Peter. For those who are law teachers in the, uh, audience, they are probably folks who taught business associations or corporations at some point, or for those who are lawyers, they've probably taken those classes at some point. As part of those classes they've been introduced to a concept called veil piercing, which has been described as somewhat of a morass of a doctrine, something that sort of lurks in the midst of metaphor, to quote one famous judge. I wonder if you could introduce the topic of veil piercing for the listeners. Maybe some of the lawyers could use a refresh. Maybe some of the non lawyer listeners could use an introduction. And could you tell us a little bit why veil piercing as a topic has sparked literally hundreds of articles by law professors and practitioners over the decades.

Speaker B: So if you may recall back into your basic business organization's course, one of the bedrock principles of, uh, corporate law, and specifically sort of corporate entities, is the idea of limited liability. The idea essentially that you have a shield that is provided for the entity, specifically for the individual shareholders and their personal assets. We believe that one of the hallmarks of a corporate entity is this principle of limited liability. And veil piercing is really the single exception to that basic principle. It's the idea that under extraordinary circumstances we think it is justified to actually allow third parties to in fact actually reach into the assets of those individual shareholders and be able to actually redress certain kinds of holes that exist within limited liability. So the question of how you think about when limited liability is really justified and when it is not is really part and parcel of this question of veil piercing. If you may also recall back in your business organizations course, this is a doctrine that is a creature of the common law here in the United States as well as in the United Kingdom. And uh, ever since its inception has really been a source of confusion, consternation and challenge because scholars, uh, and judges and lawyers and entrepreneurs have all had the same basic problem, which is trying to formulate a doctrine that seems to make sense and enables us to have fairly predictable outcomes that we feel are just. That's a common problem I think within law, but in particular it's notorious when it comes to bail piercing because the test that we have is a mercurial one. It's one that varies from federal versus state jurisdictions. It's one that acts, uh, actually also varies from jurisdiction to jurisdiction because no one jurisdiction necessarily looks at the doctrine in exactly the same way. And then when we get on a granular level we look at individual courts. Courts from time to time have very inconsistent sort of decisions that they may reach. And I remember as an actual second year law student reading about a study that was published by Robert Thompson who is now at Georgetown, but he had decided to undertake this pioneering empirical study and he examined all of the actual veil piercing cases that have been decided over time to see if he could identify different types of patterns. Bob's study is really a breakthrough not only within the field specific of veil piercing, but actually within empirical legal scholarship. His study is really famous because there are many gems and also many puzzles and oddities in his actual findings that as a student and those of my classmates and also my professor, Professor Easterbrook, all struggled essentially to be able to determine what the answers were for this particular doctrine. And that article really inspired me that many years later when I had the opportunity to become a law professor, that I actually decided to replicate that study. And that was my entry into this complete universe of all these different papers and uh, cases involving Gail Pears.

Speaker A: I'd like to delve a little bit more into that universe of papers. A lot of scholarship has been written on vel piercing over the years. That is in a doctrinal or a theoretical bent where scholars are analyzing the cases or thinking deeply about the justifications for the doctrine and instances in which it's appropriately applied. But as you note, starting with Bob Thompson's 1990s study, there's been an important strand of empirical scholarship in this area too, which you've contributed to, as you note with a, ah, paper a few years ago. Can you give us an overview of that strand of empirical Valkyristian scholarship? What sort of methods have scholars used in that scholarship, including maybe yourself and your prior paper. And there's one study in particular that you focus on, the Macy and Mitt study. So could you tell us a little bit more about that one?

Speaker B: So I'll start actually first by going abroad and describing the kind of work that's actually done in the United Kingdom, because not only have I actually done some empirical work here in the United States, but I've also published a number of empirical studies on UK case law. And the reason, reason why I say that is because. So if you look at the kind of scholarship that's actually done in the United Kingdom, it's best described perhaps as this doctrinal, um, narrative in which you have academics who are writing about different cases and trying to find some type of a golden thread. They're trying to look at some type of a principle that really synthesizes and explains decisions that have been issued by UK courts over time. And here's the other part that's also tricky, which is I had the privilege of actually meeting with a Justice on the Supreme Court of the United United Kingdom. And the gateway question that he presented to me and my co author Alan Dignam, who's at Queen Mary University, London School of Law, was here are 10 different cases. And can you actually tell me whether in fact actually veil piercing cases? The thrust of the question is that even though sometimes cases are identified as veil piercing cases, in the United Kingdom, at least what they say is not always what is in fact actually the case. And that's very useful for us to understand because when we look here at the United States, for instance, and the different kinds of cases that have been issued, bail piercing appears in many different forms. It can appear in the bankruptcy context, in the environmental law context, in the workers compensation context. There's a myriad of different kinds of statutes that are well beyond the traditional kind of classic examples and scenarios that we typically think about in the context of piercing of the corporate veil, which might be for instance, just contract cases involving a corporation or torts involving that type of a corporate entity. And so that's one of the kind of challenges about the kind of scholarship that we're actually doing, which is to try and provide really a portrait. That's what Bob Thompson endeavored to do. They gave us really our first, best glimpse of this. And uh, that really then after my study was one of many different actual studies that came out at that time to try and give us an empirical portrait, because I think the area of empirical legal scholarship was just really starting to explode a sense of the different kinds of scholarship that are out there would range from for instance, David Hoffman and Christina Boyd, around the same time that I published my original study back in 2010, examined actually dockets and looked at veil piercing cases to see whether those kinds of claims, basically the evolution of those particular claims through the process of actual litigation. So that is a really interesting and fascinating idea because obviously most of the types of cases that we see in their final reported form are just simply a fraction, a tip of the actual iceberg of the cases because many of them actually settle. So that's an important component of all of this. My study has really examined published types of cases, but then we also have actually articles from yourself, Andrew, which is using other kinds of methods that still qualify as empirical methods, for instance, talking to different kinds of entrepreneurs. And so what I think we're all trying to do here is to figure out what is actually happening with veil piercing, what actually matters to entrepreneurs and the extent to which they actually think about these things as they go about their daily business. And in the context of courts, is the question really about what are courts really looking at? Are they actually doing what they say? Are they reaching the right particular decisions and are they reaching and actually providing us with the rationales and justifications for that? All of this is a segue then really into a another path breaking paper that was published about 10 years ago by, by Jonathan Macy at Yale and Josh Mitts at Columbia. And I like to call them Eminem M M. Jonathan has told me that he likes to refer to the pair of them as J and J. You can pick your particular set of acronyms there. But anyways, Macy and Mitts had another breakthrough in terms of empirical legal scholarship and our understanding about bail piercing. Because most of the methods that have been used are what I describe as old fashioned bail piercing, which is manual coding of different cases. If you look at Bob Thompson's data set, you'll see a description within his paper in which he had teams of different law students that actually pored over individual actual cases and had to code them all, collect data, process all of that data and then extract information that were then used inside spreadsheets to actually do different types of statistical analysis, ranging from simple actual number counting to actual regression analysis. I did exactly the same thing, which is that I read thousands and thousands of cases and extracted information from that. And as had a research assistant then who proceeded to go ahead and aggregate all that information to produce the kind of statistics that were published in my particular paper. This is time intensive, it's exhausting. It's Also prone to various kinds of errors, which is that, huh? No matter how many individuals you use in this type of old fashioned manual coding and empirical research, you're always prone to the fact that there can be different subjective judgments about a variety of different things that doesn't make any difference if it's one person or if it makes an entire team of different law students that have all been checked very thoroughly. And so Macy omits as a means to try and solve this particular problem took advantage of uh, what was nascent technology at that time, which was machine learning, which is that what they proceeded to do was to actually assemble a data set of all the veil piercing cases and then proceeded to actually use an algorithm that they trained. They trained it by coding a small group of cases, about a thousand cases, and that information then was used to feed into the particular algorithm that was then ended up actually applied to a larger data set. And that algorithm was used then to ex and different textual features of those particular opinions to be able to try and examine and assess a particular hypothesis that they had pitched. So their hypothesis is almost as remarkable and as novel as the methods that they actually used to validate that which was that according to them, the entire existing universe of veil piercing opinions can be explained as a tripartite taxonomy, that is that all of existing veil piercing cases and decisions can be understood as essentially permutations of either fraud or misrepresentation. What they describe as bankruptcy values and finally statutory application or the promotion essentially are different types of statutory objectives. In their view. All opinions can be explained through one of these three things. And this is really quite novel and quite a radical idea because challenges some of the basic conventional views that academics and lawyers and even courts have actually held onto for many years, which is that there are a variety of other different factors such as undercapitalization, meaning the amount of capital that exists within the corporation either at its inception or at the time of the action that gives rise to the bail piercing claim. Or alternatively corporate formalities, which are things for instance like minutes and regular meetings. The kinds of accoutrements that we expect to have from a truly legal and functioning entity are all things that many people believe to have some salience basically within nail piercing cases. And according to Macy Mitts, it has not. And within their paper they proceeded to actually lay out their entire methodology and design for their particular study that ultimately ends up finding that all three of those particular categories are exactly the categories that are best suited to explain most robustly the entire universe of alpiers and cases. So that's a pretty extraordinary result. That is a paper that I had designs basically to think about what to do about that. But for me the challenge had been really I didn't have the technological wherewithal to be able to design this type of study to examine it. So I had my own data set. And I could tell from just looking through my own existing data set and records that I knew that there were cases, for instance, in which under capitalization was actually a instrumental or dispositive variable in court cases. So I could point to individual cases, but I couldn't actually look at the entire universe and determine whether or not their study was actually correct. And the ways in which, to the extent that I had different results or I knew that there were outliers of cases to be able to explain those particular results. For a number of years I sat around and had been working on other aspects of veil piercing, but more in a theoretical and normative aspect as opposed to an empirical one. That led me really to a pretty important moment which was that ah, Doug Barnard, who is my co author, came out of the woodwork and was a person who contacted me and said that he had been very interested in Macy and Mitch's paper. He is a graduate of MIT and a graduate of University uh, of Minnesota Law School and also a former partner at Kirkland Ellis. But most importantly, he's an individual who actually has trained himself with the kind of technology that Macy Mitz had deployed. And he had been tinkering with the idea of trying to redo the entire study. And so he thought that it would be interesting to collaborate and invited me to collaborate with him to see whether or not we could actually replicate their particular results.

Speaker A: That's a nice kismet that somebody who'd been thinking about the same thing that you had came along and you're able to write this paper together. You didn't necessarily have the tools back 10 years ago to do this work that you wanted to do, but a lot has changed to say the least in the last 10 years. And so you've embarked upon this paper. I wonder if you could tell us a little bit about the research questions that you and your co author pursued in this paper and introduce the methodology that used.

Speaker B: This is a paper that is actually part one of a larger project. So the way to understand this is that it's actually not one paper, but it turns out that if you look on SSRN or you're also welcome just to simply contact me and I'd be happy to share those papers with you. But we actually have three different papers and we have a fourth online resource. We have the main paper which presents our key findings that essentially tries to describe our methodology and replicate the basic results of Macy and Mitz's study. We also have posted on SSRN a online appendix which actually includes some additional features to our particular study that will enable readers to be able to replicate our particular study. We also have another article which is actually describes a component of our particular study that I'll uh, explain in just a bit once I actually walk you through the study and its methodology. And then finally we actually have some online resources on uh, our actual cases and the links to those particular cases that were used in an entire data set that will enable you to access those particular cases, see how they're actually classified and used within our particular study study. And all of this is really because this first part of the project is what we think of as a methodology. So what we wanted to do was to actually try and reexamine Macy and Mitz's paper. But in the words of George Stigler, it takes a theory to beat a theory. What we decided to do was not to actually replicate their study word for word, bit by bit. In fact, actually it's not even possible because within their actual paper, and we actually also had the opportunity to talk with them about their studies design. But they're just decisions on a granular level that are subjective. And unless we actually were there to be able to watch and see how they actually dealt with individual types of cases, there's no way to actually replicate their particular study. We decided actually that with the benefit of time, what we wanted to do was actually introduce a new tool. And so the new tool that Doug really deserves a tremendous amount of credit for deploying is artificial intelligence. So what he has done is employed different kinds of tests that include not only the traditional sort of manual coding and training of algorithms into what's called a naive Bayes test that Macy and Mitts had actually used. But in addition to that, he actually did a whole series of different other tests that involved hybrids and also pure versions of a artificial intelligence tool known as claude. And so what we decided to do with CLAUDE was we actually provided it with instructions as to how to actually read veil piercing cases and extract information so it would identify cases and determine whether those cases were in fact relevant, meaning that they were actually veil piercing cases. And then secondly, what Claude would proceed to do would actually be able then to make predictions about whether or not veil piercing in fact actually has taken place or whether or not the request for veil piercing has been rejected, and then proceeded to actually identify what the possible reasons as to why the court decided the way that it did it. And Claude did this without any access to any actual manual coding. This is what we call a blind prediction system. Claude did this simply on the basis of the cases that we had uploaded and secondly the instructions that we had provided to it. And we wanted to see whether Claude could actually make predictions with a level of accuracy that was comparable, if not perhaps even actually better than what Macy Emits were able to do. And then secondly also our version of Macy Emits, which was to actually use a manually trained algorithm that was actually using our own actual codings as a means, basically to then determine whether the actual universe of cases involved relevant cases and how courts actually decided those things. And so what we want to do with these papers is we essentially painstakingly have documented every single step that we have effectively taken in design of our particular study. And invariably of course there are parts where there are some subjective decisions that have to be made and we've done our best to try and describe those particular subjective decisions, explain how we decided that and explain what our reasons for that were. But that enables uh, anybody essentially to be able to look at our particular papers and hopefully be able to replicate our particular results or the study design. Or if for instance, you actually have an interest in simply using this technology for something else that has nothing to do with either veil piercing or corporate law, you'd be able to extract all this information. We're also making ourselves available to share whatever other additional resources and help people to be able to think their way through this. But for our purposes what we're really trying to do was step one was just try and figure out whether we could actually get the same results that Macy Mits obtained. And then secondly, the next phase of the project then was based on whether Claude was actually a sufficiently reliable actual resource and tool that we could use. Then we feel that then it would be ready to be able to use to actually deal with a brand new data set of veil piercing decisions. But that required us to actually determine whether Claude was sufficiently reliable in relation basically to what Macy Mitz has done, as well as the empirical standards for this type of research.

Speaker A: Let's turn back to that. Step one, what findings did you produce in this study and how do they compare to M and M? Or as Jonathan might prefer, J and J?

Speaker B: So what we did first of all was we essentially retraced all of their particular steps so we ended up taking just a random sample of different cases and then we coded them and we coded them and not all of those cases were necessarily veil piercing cases. And so that was important to understand exactly what the percentage of that actually was. But then secondly, we proceeded then to go ahead and take whatever the relevant cases were and manually code them and use that information then basically to feed that into the particular algorithm. Once we actually had that particular algorithm, we could actually apply that algorithm then to the actual data set, right? And then we would take the actual results, regress those particular results results, and to do something called text analysis to be able to figure out which words and which rationales essentially were best matched to explain the particular outcomes that courts had actually reached. So that's really what we would call data set 1. Data set 2 is the one where we actually had Claude descend upon. And so what essentially happened was, is that we would take the algorithm from Data Set one, basically apply it to Dataset two. But also separately, we actually wrote out these long instructions for Claude and we said, here is an entire, entire set of different actual cases. And what we want you to do with these particular cases is essentially to identify whether they're relevant and what the particular outcomes were. Now, in order to be able to do this with Claude, we first actually ran through a series of different kinds of checks to see whether Claude was in fact working appropriately. The paper goes into great detail about the steps that we actually took to determine whether we felt that CLAUDE had the capacity basically to mimic the kinds of results that we got with our algorithm with those 1,000 cases. And the way that we did that was twofold. One is that we actually had not only humans, which is that we actually had Doug, along with another independent legal researcher, examine a set of different cases and try and ask the question, do we actually agree as to whether or not these particular cases are in fact nail piercing cases and what the outcome of that case was and what the actual reason for that particular case was. But then secondly, what we did was we had Claude almost do a self validation check. So we asked Claude to scrutinize its own particular examination of these particular cases. And this is the article that's kind of separately online, which is how you could actually use artificial intelligence to have it examine its own actual work and determine whether or not its assessment is sufficiently accurate. And without further ado, essentially the answer to all of this is that a number one, when it comes to Macy and Midsis actual studies, we come fairly close, but we're not quite exact. In terms of replicating the level of accuracy that they have had. But secondly, the level of accuracy that we obtained through Claude itself was greater than the level of accuracy that we had from our own initial manual coding, along with the actual level of accuracy that Macy Mitz even had actually reported for their own particular study. And that finally there's something called an XGBOOST ensemble test, which essentially takes the best of both of our manual codings as well as the results and predictions from clients thought, and it combines them together and that gets us our best possible results, which are within the range of what we think is sufficiently accurate that we feel like this can be a reliable and trusted tool for the purposes of, uh, veil piercing analysis.

Speaker A: Peter, I'd like to return to the methodology a little bit for listeners. Maybe they're legal academics, maybe they're litigators, they're corporate planners, they're judges even. How does this paper advance their understanding of bell piercing doctrine? And what does it say about maybe the promises and risks that, that your LLM, um, methodology raises? And there's a lot of skepticism and m precautions right now around LLMs in the legal and judicial communities. So what might you say to some of the audiences for this paper, whether they're academics or people practicing or adjudicating law?

Speaker B: So I'll answer that question really in two different ways. One is a very focused and result specific answer, which is that you'll see within our particular paper that, uh, one of our particular substantive findings were that formalities and capitalization are actually factors that are salient in our analysis of veiled piercing opinions. So these are two of the factors that historically people have thought were relevant, which we have essentially firmed, but which Maci admits, did not admit as part of their particular study. And I would tell practitioners who are interested in the veil piercing analysis twofold, number one, which is that this particular study confirms our suspicions that when it comes to the actual dispositive factors, bankruptcy, fraud and misrepresentation, along with various kinds of statutory policy, are salient. But formalities and capitalization also appear to be salient within court decisions, or at least existing court decisions. And so from the standpoint of planning purposes, these are things that should also be taken into account. We hope to ultimately actually develop a data set that can provide more robust answers than that. Secondly, what we want to say is that from a general standpoint, these large language models offer us an incredible tool that we feel can obtain a level of accuracy that is useful for the purposes of doing this type of work. So one of the common threads that we see within empirical legal scholarship is that a, there's kind of a bias really towards more federal cases rather than state cases, because federal cases usually come in a nicer data set which is more easily accessible and usually can actually be analyzed much more quickly. But when it comes to, for instance, corporate law is largely a function of state law. And it's important for us operating in this particular space, for instance, to be able to have access to a tool that can actually examine all the state data. And artificial intelligence is one of those kinds of tools, which is that rather than saying that this is too time consuming of a particular task to do an empirical study for using Claude, and using this type of a hybrid approach that we have enables now people to be able to look at large bodies of data and be able to analyze it in systematic ways that we feel are sufficiently reliable to provide us with a first cut of information. Now, I know that artificial intelligence is really exploding right now. I know that it has lots of ethical and other types of controversies that are associated with it. Some people think of artificial intelligence as an echo chamber. Some people think of it as a plagiarism machine. But what we're hoping to do here is to demonstrate at least that artificial intelligence has a role to play in designing empirical studies and empirical legal scholarship to be able to now make these large data sets accessible to us that can give us answers that hopefully we as academics and as lawyers and as jurists can interpret and use using our analytical tools that we have as lawyers, to be able to understand what matters and what doesn't matter in prescribing what the law actually should be.

Speaker A: Are there any key takeaways or closing thoughts you'd like listeners of this episode or readers of the paper to have?

Speaker B: I think the bottom line, unfortunately, is that on the one hand, hand when it comes to the substantive area, veil piercing, we still have the mess that we have always had. It's a mess that Bob Thompson, that myself and also now Jonathan Macy and Josh Mitts have all attempted, I think, to provide some clarity. But unfortunately we're still in a state where we're not able to actually provide the kind of guidance to entrepreneurs and also courts about whether the law is actually correctly designed and if it's not, how it should in fact actually be designed. Design. On the other hand, we do think that the one takeaway here is this, that the results of our particular study give us great confidence that the artificial intelligence tools that are now, uh, available can be wielded in a way that is both reliable and also useful to get us the kind of robust results. And so it's really a game changer when it comes to empirical legal scholarship. We're excited to see what this next generation of, uh, tools can do, not only in corporate law, but also beyond.

Speaker A: Our guest today has been Peter oh, Professor of Law at the University of Pittsburgh. We've discussed his article the Large Language of Val Piercing, which he co authored with Douglas Barnard of mit. The article is forthcoming in the journal Geometrics. I'll add a link to the paper in the show Notes for the Episode Peter, thank you for joining the Business Scholarship Podcast.

Speaker B: Thanks Andrew. It's a real pleasure and I appreciate your service to the academic community and the legal community at large.

Speaker A: Thank you for listening to another episode of the Business Scholarship Podcast. If you like what you heard today, be sure to subscribe the podcast on Apple, Spotify, or wherever you get your podcasts. Rate the show and let other people know about it too. If you have ideas for future episodes, let me know. My email address is andrewdrewkjennings.com and I look forward to hearing from you. Until the next time. I'm your host, Andrew Jennings.

Speaker B: It.