Episode Transcript
[00:00:00] Speaker A: Imagine a justice system built on rigorous evidence, not gut instincts or educated guesses about what works and what doesn't.
More people could access the civil justice they deserve.
The criminal justice system could be smaller, more effective, and more humane.
The Access to Justice Lab here at Harvard Law School is producing that needed evidence. And this podcast is about the challenge of transforming law into an evidence based field.
I'm your host, Jim Greiner and this is Proof Over Precedent.
This week we're bringing you a student voice.
[00:00:37] Speaker B: Hello, my name is Sydney Nelson, a student at Harvard Law School, and today I am joined by Elizabeth Guao, a fellow student and classmate. We'll be talking today about a blog post that I wrote, the first and hopefully a series about the importance of accessible court data.
[00:00:53] Speaker C: Hi Sydney, happy to be here. So, just to start off, could you please share what made you interested in this topic?
[00:00:59] Speaker B: Yeah, of course. Thanks for asking.
So last year I spent a semester researching the litigation experience of pro se prisoner litigants versus non pro se or non prisoner litigants. And to understand the experience of litigants, I need to know things like how long was their case open?
How long before a judge issued a reported recommendation, which is was something required in the prisoner context? How many dockets entries were there for a given case, and what was the outcome of the case? To answer these questions, I turned to pacer, the Case Record System for the Federal Judiciary. PACER stands for Public Access to Court Records, and as a service of the Federal Judiciary branch, PACER houses 200 federal court cases. Nearly all the documents filed by a judge are the parties in any case.
However, just because the documents there are in PACER doesn't mean that it was easy to find or that information was readily accessible. What I quickly realized was that there was no simple way to use the data that was located in pacer. The only way I could figure out how to do it was to copy and paste search results into an Excel spreadsheet and then set up filters to narrow down a set of like 500 or so cases into the 20 I was most interested in. And then from there, from those 20 cases, I had to go into the dockets of each case, pull additional data from the documents and filings in order to determine how long a case was open. And those other metrics I was using to measure the difference in experience.
And often the docket when I was in there was confusing in itself and I felt like I was often reading a foreign language. So all of this got me thinking, like, why is this so hard?
[00:02:30] Speaker C: Wow. So it Sounds like your exploration of one problem. Researching the litigation experience of litigants led you to another. The accessibility of court data in pacer. Was that the moment you decided to write this blog?
[00:02:43] Speaker B: Sadly, not quite. It took me experiencing the same issue in a state system as well before I finally put pen to paper. To make a long story short, earlier this year, I was doing research on default judgments for plaintiffs and eviction cases in a state case record system, and I encountered the same issues there that it took so much manual effort to pull information from the case record system into a format that was actually understandable and usable.
So across both of these systems, the federal side and the state side, I'd encountered immense difficulty in pulling information and a lack of clarity in what the information even represented. And this was all obscuring insights that could have been revealed by the data and was when I finally got into a format that could be analyzed. And so I got to thinking, if this was so hard for me as a law student, how did any other member of the public manage to figure out and find what they were looking for?
And to begin with, what kinds of data should even be accessible by the public?
[00:03:40] Speaker C: Right. And what did you find as it pertains to the kinds of court data that should be accessible by the public?
[00:03:45] Speaker B: Yeah, well, I think it's helpful to start with the scale of the court data that exists, just to get your mind around it. So, within the federal judiciary, There are nearly 200 federal courts and 900 judges, and these are overseeing hundreds of thousands of filings a year.
However, 95% of the country's cases are happening at the state level, with state courts handling nearly 66 million cases a year. That's a lot of cases. So we have millions, hundreds, like, millions of cases occurring in the country each year, and then each case in itself could contain hundreds of data points across the core phases of litigation.
So when you're thinking about, like, these points where data could be sourced, think of, for example, the filing of a case, the judicial process itself, and then the outcome. And that's just one framework for where you begin to mine data and where it can be captured. And this data could tell us an awful lot about how our core systems are or are not working. It would help individuals answer the question of, like, where the heck is my case at? While enabling researchers and law students like myself to identify trends and patterns within our legal system. And ideally, courts would also use this data to improve their own processes and enhance transparency and increase access to justice.
But the problem is that for all the data these millions of cases could produce and the questions that could be answered. There's no official standards governing availability of court related data here in the United States. So this leaves folks like myself who want answers to these questions spending hours in case record system attempting to pull together enough data to get a hint at what's going on.
[00:05:22] Speaker C: Okay, so what I'm hearing is that there's a ton of data that could exist and information that could tell us a lot about how our justice system works, but it doesn't sound like there is a way to easily access that information.
So where do we go from here?
[00:05:37] Speaker B: I know it sounds like a daunting problem to solve, but I think that there's actually a lot of hope and potential here and that we as a country can begin to shift the narrative around access to port data.
So a little context here that access to the kind of data that I'm talking about is considered open data. This means that anyone can really access, use, reuse and redistribute, distribute this data for any purpose without restrictions.
And this term was really coined at the turn of the 21st century with the increased use of technology accelerating data collection and sharing. But this concept has been around since the mid 20th century with the idea that scientific research should be free to all to use and access.
Then here in the public sector, this concept has taken on a little bit of a different bent called like open government data. And that originated from the idea that because our governing institutions collect large amounts of data and information of us as citizens, it should also be available for use by us, the citizens. And this usability piece, that is the critical distinction.
So much of the data available on government websites is published but not usable. Think about like a PDF, a report that's meant to be read in isolation and doesn't easily allow you to grab data off of that PDF.
Rather, to qualify as open data needs to be reusable, meaning can be downloaded in open formats and read by software, and that we have the right to reuse it. It's about the ease of use of that data.
[00:07:03] Speaker C: Oh, interesting. Okay, so there's this whole field dedicated to the topic of open government data. So now I'm wondering, how do our court record systems stack up against the principles of open data?
[00:07:15] Speaker B: Not well.
Yeah. Unfortunately, when compared against the principles of open data, our court systems at the federal and state levels are failing on several fronts.
I'm just going to walk through a couple of examples, but there's way more if you really get into it. So one easy example is that data to be comply with open data, it should be accessible and easily modifiable.
And in reality, extracting data from court websites requires extensive manual effort to gather and organize information from disparate sources. My Excel spreadsheets of search results are case in point.
Additionally, data should be complete and include as many details as possible.
In contrast, our court record systems often categorize cases at largely generalized levels, leaving more granular items buried behind dockets and documents.
Then, speaking from personal experience, data entry errors in court record systems are not uncommon. So this means that information that is available might not always be correct.
And one final example, and a big one, is that while the public can access these case record systems, which is another principle publicly available, many of these systems charge a fee. So, for example, pacer, the federal system I was talking about earlier, charges you 10 cents per search page. So anytime a search is performed, you are charged a fee based on the number of pages generated in that search, even if the search replays displays like no matches found.
So this means that you have no idea how much a search could cost you until it's too late. And while PACER will say that there's a $30 threshold before the fees are collected, so it's not really a big deal, in reality, 25% of Pacer users are paying these fees in a given quarter.
And the potential of fees and the unknown of how much you might get charged is still a huge return to using these systems.
Another example is at state level.
And so Utah's case record system exchange charges 35 cents per search and a dollar for documents.
Charging these fees to access data goes against the principles of open data and ultimately limits access to court information based on financial means. These are just a couple ways that, like data should be accessible and modifiable, data should be complete, data should be accurate, and data should be readily accessible. Those are just some of the principles of open data where our court systems and case record systems are currently failing us.
[00:09:34] Speaker C: I see. Okay, so my next question would be, what's the state of open government data here in the United States?
[00:09:40] Speaker B: Yeah, so I talked a little bit about how our courts are not adhering to these open data principles at the moment. But luckily, the concept of open data isn't foreign in other places in the United States government.
So when you look at the executive branch on President Obama's first day in office in 2009, he issued a memo on transparency in open Government, where he directed the Office of Management and Budget and the General Services Administration to develop a plan to promote transparency Participation and collaboration by publishing government data and information for public use. This was then followed by the Executive's open government directive, where agencies were instructed to adopt a presumption of Data openness.
Then 10 years later, Congress followed up and solidified this through the Open Government Data act, which mandated that federal agencies publish their information online as open data using standardized machine readable data formats on a website called data.gov I highly recommend listeners take a look at it. It's very neat. Nearly hundreds of thousands of data sets have been uploaded since then. And Data.gov also collaborates with non federal data sources like states, school cities and counties whose data sites match the requirements of data.gov, and you can access those data sets through them as well. It's been very neat to see how this movement of open data and open government data is catching around our country.
[00:11:00] Speaker C: That's so interesting that the Executive Branch and then Congress themselves had been pushing agencies to adopt open data principles. But you mentioned that the courts fail on a lot of open data fronts. So why haven't these federal efforts spilled over into the judiciary branch?
[00:11:16] Speaker B: Yeah, so this is because the Executive memo and then that Congressional act that followed covered only federal agencies and that the judiciary is not compelled by the same requirements.
This is not unusual. The federal judiciary is exempt from other transparency and open data laws that apply to the Executive branch. Foia, the Freedom of Information act, is a common one. And this act requires disclosure of previously unreleased information and documents held by US Agencies upon request. However, the judiciary is exempted from this requirement.
So not unusual that, like the judiciary, has a slightly different set of norms and laws governing their transparency.
But however, during the 117th Congress, the Senate and the House attempted to kind of write this imbalance by entertaining the Open Courts act of 2021.
This act, the Open Courts act, was spurred by a federal appeals court decision concluding that pacer, that federal case record system, overcharged fees to users in excess of $190 million in six years beyond PACER statutory authority.
And so PACER, the system that charges fees and saying, don't worry, we're not charging, you know, crazy amounts of actually charging the American populace nearly $200 million in excess fee.
And so this bill, the Open Courts act, was drafted in response to that. And one of my favorite quotes from the bill's co sponsors was where they emphasized that the current system had harmed access to justice, infringe on the foundational right of the public access to court records, exposed sensitive data to external threats, stifled innovation, and fostered wasteful Spending and this legislation. This act directed the federal judiciary to create a modernized court record system with data standards for searchability, accessibility, quality and affordability.
However, despite advancing out of both corresponding committees in the House and the Senate, neither version of the bill made it to a full vote in its respective chambers. So it ultimately expired at the end of the session.
[00:13:18] Speaker C: Do you have any insight on why this bill didn't make it to a full vote?
[00:13:22] Speaker B: It's hard to say exactly why, but I have some theories.
In response to the Act's introduction, the Judiciary released a statement themselves asking Congress to defer the legislation, and they wanted to have a broader discussion between the legislation, legislative branch and the court and possibly amend this Act. So I think what's interesting is when you read the Judiciary statement, in particular, the Judiciary expressed concern over eliminating fees for high volume users of pacers, the timeline of modernization, and legislation's requirement that the executive branch be directly involved in improving the current system.
So you know how I mentioned above, the Judiciary has had less oversight and more independence as compared to the executive agencies. This whole act and the conversation around it has the echoes of those same trends. The tension between the desire to further public services, including through legislative means, and the desire of the judiciary to practice their own independence and autonomy. And my suspicion is that that tension eventually got the best of this bill.
[00:14:22] Speaker C: Got it. Okay. So it sounds like at the federal courts, open data hasn't been adopted, but I know you mentioned that 95% of our cases originate at the state level. So what's the story there?
[00:14:33] Speaker B: Yeah, so unfortunately, the state judiciary systems don't seem in a much better position. It doesn't seem that any state has passed an open data act regarding judiciary case records. But if someone knows someone out there, give me a holler.
And to furthermore complicate this, all states vary widely in the type of case filing system they use, the data recorded, and the information publicly available.
This can make the process of gathering data across states incredibly burdensome. For example, one of my professors was talking about how she was attempting to collect information on misdemeanor cases and she had to contact the administrative office of each state court system separately and then attempt to reconcile the data received to be able to compare among the states.
So it can be a pain. We're looking at the national across all the states and trying to compare data across these case record systems.
[00:15:21] Speaker C: Okay. And just to spell it out, why is it a problem that courts don't adhere to open data standards?
[00:15:27] Speaker B: Yeah. So I've been talking a little bit about some Broader themes of just the amount of data that exists, the use of that data, whether for individuals, researchers, or courts themselves.
But I also want to hit on like a specific example to really illustrate this.
And so I'm assuming you've heard of the book Evicted. It was published in 2016 by Professor Matthew Desmond and it made a big splash because his book came out of a large scale study on eviction rates. And had he uncovered trends in court data that hadn't been discussed before, his book then led to immense public attention to this issue, led to a national movement for tenants right to counsel, and cities and states began to recognize this right.
And this all showcased the power of information to create change and improve access to justice. Professor Desmond also launched the Eviction Lab, which tracks evictions from around the country by pulling case filings from public record systems. So he's trying to solve for the same issue that I'm describing here, that there are really powerful trends hidden in these court record systems that are really hard to access and understand.
And imagine if, like, it didn't require writing a book to uncover these gaps in our justice system and like, where all this information is hidden in the data.
And I want to emphasize that the power of case data isn't just contained to evictions.
It can be used to unveil larger inequities that exist within our legal system. For example, on the conversation of racial disparities that exist, a case study on Pennsylvania counties looked at 50,000 court cases, and they discovered that judges were more likely to assign cash bail to black defendants than to white defendants, even when the risk of skipping court was equal.
And in another study, this one by ProPublica, they discovered that the risk of judgment when it came to debt collections was twice as high in majority black communities than white communities, even with consistent income. And they reviewed hundreds of thousands of court judgments to determine this. But in their analysis, ProPublica emphasized that while they identified this disparity, they were unable to say why, in part due to the lack of research on debt collection lawsuits.
Another example is in the gender disparity realm. And I'm actually going to share another personal example and talk about this one, because back in undergrad I conducted a study on this myself. Following the summer working at the US Supreme Court as an intern, I became super interested in that a dramatic disparity in the gender of advocates appearing before the court.
While women made up only 18% of the attorneys presenting in oral arguments, and recent research discovered that female lawyers were treated differently than their male counterparts by justice that they were more likely to be interrupted than male attorneys. So for my senior thesis, I examined if these manifestations of implicit gender bias, these interruptions could be reduced by something called the mere exposure effect. This is a psychological tendency where preferences are developed because of familiarity. Basically, the more you see it, the more you like it. And my hope was that as a woman attorney made repeat appearances in front of the court, justice would begin to view her as an attorney first and a woman second, reducing the implicit bias that manifested to conduct this. I took a sample of seven female and seven male attorneys who had all appeared in front of the supreme court at least 11 times and pulled the transcripts for the cases they appeared in and tracked how often they were interrupted and how many words they were able to get out.
Unfortunately, with all this research, I discovered that the number of appearances did not reduce manifestations of implicit gender bias for female attorneys during these oral arguments that they are interrupted and the rate of interruption did not change. And even worse, studies conducted by other researchers have shown that when female attorneys speak more, they are more likely to receive an unfavorable decision in comparison to their male counterparts, who are more likely to receive a favorable decision for the same behavior.
So this is like, such useful information to have if you're an attorney appearing before the court. As a hopeful future woman attorney, I would love to know, like, what kind of treatment I should be expecting when I'm appearing before a panel of justices. But the thing is, like so many of these other trends we've pulled out, you won't know this exists unless you put in the effort to pull the transcripts, pull the data, pull the information, and start to identify patterns. And one more thing I want to hit on is that, like, we're in a really tricky time right now. As the federal judiciary, there's a historically low public trust in the federal judiciary, and rule of law is currently under attack. So it's really important to think about ways that we can show the public and increase trust in our systems and our judiciary. And open data is a great way to do that. Wow.
[00:20:05] Speaker C: Okay, so this all sounds really important and really interesting. And so now I imagine our listeners are wondering, what should we do next?
[00:20:12] Speaker B: So I'm super excited to explore this issue further.
I'm really passionate about increasing access to case data in hopes of improving access to justice more broadly. Whether that's you as an individual trying to understand your own case, you as a researcher, helping to identify trends, and ideally court systems, too, who are seeking to improve the services they offer. And their work as a public institution.
I'm interested in exploring further this tension that I mentioned between the judiciary wanting to dictate its own operations and the ability for Congress, and maybe by extension the American public, to have oversight. We saw this played out with the stalled Open Courts Act.
I'm also really interested in the user friendliness of our court record systems. So not just that the data isn't accessible, but even if it was, how easy is it for the average person to navigate the systems but find what they need and understand what's there?
So, hoping to continue having conversations and doing research along this tension line and the user friendliness of our systems.
However, I want to emphasize there's a number of bright spots already happening in this space that make me excited about the future of open court data. There's so many innovative civil society efforts happening here.
So, like I mentioned, Professor Desmond's Eviction Lab, that's one of them. I highly recommend you check them out. They have a really cool map that shows how you can take all of this court data on filings and overlay it across the geography of the United States to understand the hotspots of eviction.
Another favorite I want to give a shout out to is the Free Law Project. So they are focused on improving transparency in the judicial system by advocating for an open legal research ecosystem. And one of their projects, recap, which is PACER spelled backwards, focuses on opening up federal court data. So they basically allow those that access PACER to to then upload their search results to recap so that anyone else can then just view these, this case information, this data through recap without encountering the PACE or fee.
And at the state level, there's the Court Statistics Project. So this is a nonprofit initiative that consolidates annual caseload data from states in hopes of uncovering trends in shaping policy. So a lot of folks are having conversations about this, whether it's access to court data, specific topics that they're passionate about, like eviction, or broader conversations about digital transparency. And I'm excited to be able to add to that conversation today.
[00:22:30] Speaker C: That's so great to hear that there are folks out there doing great work in this space, and I'm excited to see what your future research generates.
[00:22:37] Speaker B: Thanks so much, Elizabeth. But most importantly, the first step in all of this is getting people to care that this data exists and that we currently can't access it easily. So I really appreciate you chatting with me about this today, and I look forward to investigating this issue further and sharing what I learned here. Thank you. Thanks so much.
[00:22:54] Speaker A: Proof Over Precedent is a production of the Access to Justice Lab at Harvard Law School.
Views expressed in student podcasts are not necessarily those of the A J Lab.
Thanks for listening. If we piqued your interest, please subscribe wherever you get your podcasts. Even better, leave us a rating or share an episode with a friend or on social media.
Here's a sneak preview of what we'll bring you next week.
[00:23:19] Speaker B: Hello, my name is Sydney Nelson, a student at Harvard Law School, and today I'm joined by Jacqueline Lin, a fellow student and classmate. We'll be speaking today about a blog post that I wrote, the second in a series about exploring how to increase access to case information to improve access to justice more broadly.
Today we're going to be talking about the importance of prioritizing pro se litigants in the federal judiciary's court record platforms.
