Episode Transcript
[00:00:00] Speaker A: Imagine a justice system built on rigorous evidence, not gut instincts or educated guesses about what works and what doesn't.
More people could access the civil justice they deserve.
The criminal justice system could be smaller, more effective and more humane.
The Access to Justice Lab here at Harvard Law School is producing that needed evidence. And this podcast is about the challenge of transforming law into an evidence based field.
I'm your host, Jim Greiner and this is Proof Over Precedent.
Welcome to another edition of Proof Over Precedent, the podcast the Access to Justice Lab. I'm your host for today, Jim Greiner the faculty director of the Access to Justice Lab. And this is the third of our series of three podcasts that we're producing on the Access to Justice Lab. And Marilyn Harp and Kansas Legal Services study on the effectiveness of criminal justice record clearing or expungement.
And we have previous in the first podcast episode we discussed the field operation, how it exactly worked, as well as the fact that the field operation depended on the idea that those who were randomized seeking expungement, who were randomized to a traditional attorney client relationship, full representation from Kansas Legal Services, would achieve expungement at higher rates than those who were randomized to self help materials and to a consult with Kansas Legal Services. As we discussed in that process, that assumption that hope, whatever turned out to be true, so very strongly true.
In the second podcast we discussed whether having an expungement or achieving an expungement improved participants job prospects, employment, what we could tell, which was little with respect to wages and income. And the overall conclusion was that unfortunately achieving an expungement raised the hope that those job or employment indicators would improve, but that there was no indication that they actually did.
And in particular there was no indication that there was an increase in the job application rate.
And so it was a little bit tricky to figure out a mechanism of how job prospects could have improved but actually evaded detection from our study.
And so today we're going to the third of the three. We're going to be discussing the other two primary outcomes that we studied in this research effort. Those are related to housing and identity to see whether there is any improvement in those two areas. And so with that framing, let me remind everyone who's here and is joining me on the podcast. So Marilyn, can you introduce yourself?
[00:02:50] Speaker B: Sure. Marilyn Harp, I'm the retired executive director of Kansas Legal Services.
[00:02:55] Speaker C: Terrific. Patricia hi, I'm Patricia Gantz. I'm a data analyst at the Access to Justice Lab.
[00:03:01] Speaker D: Renee hi, I'm Renee Dancer, researcher at the Access To Justice Lab.
[00:03:07] Speaker A: I'm Ryan.
[00:03:08] Speaker E: I'm Ryan Halen. I'm a data analyst at the Access to Justice Lab.
[00:03:12] Speaker C: Okay.
[00:03:12] Speaker A: And so. Super. So Ryan, let's start with you just for the elevator story here, the elevator pitch story here. With respect to housing, did we see any movement in terms of people's participants optimism with respect to housing and then any movement with respect to actual indicators of housing? So just overview, what did we find with respect to housing?
[00:03:35] Speaker E: Yeah, so we specifically found that in terms of whether or not participants felt deterred from applying for new housing with a criminal record, getting the expungement, decreased VAT deterrent, they felt like they had a better chance of getting housing. Unfortunately, they did not observe any differences from the self representation group in terms of actual observable housing outcomes, etc. How they felt about it.
[00:04:05] Speaker A: And that lack of an effect included the idea that there was no increase in applications for new housing, is that correct?
[00:04:13] Speaker E: Yes. So we, yeah, we looked at both the actual incidence of physical movements from addresses using things like utility data and responses to survey questions. But we also asked about attempts to find new housing. And neither the observed actual physical movements or the attempt to move showed any statistically significant differences between the two treatment groups.
[00:04:36] Speaker A: Okay, so that's the elevator conclusion with respect to housing. Let's just get the elevator conclusion. With respect to indicators of life satisfaction or, or improved identity or happiness, did we see any movement across those various indicators?
[00:04:53] Speaker E: Yeah, the story is the same here. We didn't see any differences, any statistically distinguishable differences on terms of satisfaction with individual's life with their, with the way their life was going, or any other kind of happiness or satisfaction metric. We, we asked on surveys.
[00:05:11] Speaker A: Super. Okay. Overall, pretty depressing findings. Would you agree with me?
[00:05:15] Speaker E: Yeah. Assuming that you wanted expungement to help people. This is pretty sad.
[00:05:21] Speaker C: Yeah.
[00:05:21] Speaker A: So let's go back then and get a little bit more detail.
Renee, could we turn to you and say. And ask you to just tell us a little bit more about the sources of information that we pulled? These two indicators. These, the indicators for these two topics. So for housing, what sort of data were we relying on and where did it come from?
[00:05:41] Speaker D: Yeah, we were looking at address data and to some extent the ability to have address history data if we were able to. And we used public utility records for part of that. We all, you know, we of course obtained address data at enrollment from directly from the participant. We also would see address data on Department of revenue records, so tax records, but public utilities helped us especially to Understand address moves and with the proxy of turn on and shut offs of utilities would help us with that. Now we also asked about moves in our surveys.
[00:06:24] Speaker A: Yeah, the surveys again had a very high response rate Overall, about an 80 to 83% response rate across both conditions.
And so that, so just those were questions that we included in the, in the surveys, is that right?
[00:06:38] Speaker D: Yes, that's right.
[00:06:40] Speaker A: And those were, the questions included new job, excuse me, new housing applications and moves. Were there also some sources of information about missed rent and homelessness and housing satisfaction?
[00:06:56] Speaker D: Yes, we asked all of those questions in the surveys. So we again asked these questions, all of these questions at baseline or at enrollment and then periodically throughout the life of the follow up period for folks, the two year survey period. And so we would see changes as well in their responses, if any existed.
[00:07:18] Speaker A: And I know you did some of the data processing with respect to the utility records. Can you tell us anything about those?
[00:07:26] Speaker C: Yes, that's an interesting data source to use. Obviously we were not looking at it primarily in terms of how much you paying for utilities, other aspects of your housing. We actually used a couple different methods to get addresses from that in that utility records are all based on meters, right? You have an electrical meter, et cetera, outside your house, which is tied to a certain address.
So you can have a date and account open, you can have dates that the meter was read that payments were made in. And so from all of those financial transactions and all those different things, we could get a pretty good idea of where someone was paying utility bills at various dates and obviously mostly paying the utility bill to the place that's where you.
So you get a fair idea of where people lived and went. And we could also get a little bit of an idea of whether they might be in any financial troubles from their late payments since obviously typically you want your lights to stay on. So if you have the means and the overall picture for our participants was that their housing was very stable if the veterinary participants did have housing. Most of our participants are not homeless. They move rarely. Most of them were stable addressed and they had very few late payments. Most of them were able to keep their lights on. So our participants are not in extreme circumstances at baseline. Our particular population are not people who, they're homeless, they're making repeated late payments, they're in danger of losing their housing. That's not necessarily who's in our study. And that did not change substantially over the course of the our study. They were mostly stable to start with and they stayed stable after.
[00:08:57] Speaker A: And again, the suggestion May be that this group of people who participated in our study may not be representative of all of those who are eligible for expungement. They may, however, be representative of people who are seeking expungement. Because in terms as we've discussed in other work that you participated in, Marilyn and Renee, seeking an expungement in Kansas, as is true of most states, is a non trivial process. There's a significant access to justice challenge. And so it takes a fair amount of stuffing to actually try to get there.
We have information on that. We broke into missed rent, homelessness moves, housing satisfaction, new housing attempt, housing deterrence, and we've discussed that there was a statistically significant effect on new housing deterrence. In other words, there was a reduction of about three and a half percentage points in reports of being deterred on the basis of a criminal record with a statistically significant P value of a little bit under 0.03.
But then, Ryan, with respect to missed rent, homelessness moves, housing satisfaction and new housing attempt, meaning an application for new housing, what does it all look like?
[00:10:14] Speaker E: It all looks like nothing. It all looks like that self that regardless of expungement status or treatment, the same outcomes are observed for the participants.
[00:10:24] Speaker A: And with respect to all of these different metrics of housing security, were the intervals tight enough to actually exclude policy relevant effects? In other words, we have a 348 participant sample size.
That's not the largest study that the world has ever seen, which can sometimes lead to intervals of uncertain wide intervals of uncertainty and beat for. With respect to these variables, do you think that we achieved some actual useful results in terms of ruling out policy relevant improvements?
[00:10:56] Speaker E: Do I think yes. Like we're talking in terms of housing, these are on the scale of 1 to 3 percentage point differences that we can rule out anything like larger than that. So if you decrease your rate of.
If it were the case that getting the expungement decreased the percentage chance from of applying for a new house from like 52% to 50%. That doesn't feel like it's really moving the needle and we can exclude anything larger than that.
[00:11:28] Speaker A: So Marilyn, let me turn to you. I'm depressed by these findings and I crave your guidance in how to understand them. What can you offer?
[00:11:38] Speaker B: I think one of the factors, and we've mentioned this before, is that the length of people have had to settle into their life with their criminal record and from working with actual participants.
I know a number of them talked about a marriage to a crime free record, free spouse, provided them a significant level of security, and they, they had found they had adapted or they have a job and are paying rent at some place that they're satisfied with because that's the best they can do in their circumstances.
I, you know, it'd be great if expungement solved all problems, but there's very little in the criminal justice system that solves all problems. So it's just another of those.
[00:12:35] Speaker A: I will say that this may be, and this may be a theme we return to when we talk about the life satisfaction outcomes, which of course, unfortunately are going to look similar.
It may be unusual to say from an rct.
The intervention appeared to have no statistically significant effect. Therefore, we need to expand access to it.
But that may be the answer here. In other words, I think what you're talking about, Marilyn, is the waiting periods are very long after the criminal justice records are generated. And it may be that if the idea is to benefit people who, by removing the criminal records from public access, you may need to do it earlier and thus expand the remedy in order to make it effective. And that isn't what we usually conclude from an RCT that shows no effect.
We typically conclude, let's go try something else or let's change it. But here it may be that, as you say, if the waiting periods for expungement eligibility are so long that people have settled in by the time that they're eligible, it may be that the thing to do is to make them make those waiting periods earlier. Marilyn, I don't know if you have any comment on that theory.
[00:13:48] Speaker B: Yeah, one of the things that
[00:13:52] Speaker C: I
[00:13:53] Speaker B: would, I would be interested in, and I'm not sure I know Ryan and Renee have been pulled into this idea, is we have a little bit of data about how long ago the record was created and therefore eligibility ended, and a little bit of looking at whether at least among the people who the represented group of people, could that finer look at this break out a little bit of different result that would begin to answer that question about whether doing it earlier had some different effects.
[00:14:31] Speaker A: I think we're good. We'll try to take a look and see if the data will tolerate it. We may be hampered by the small population size of the study, that we only have around 178 participants who were in the legal services group, and about 80% of those achieved expungement. So we may be hampered in that effort, but we're going to take a look at it to see if we can see anything. I guess the other thing to think about is to the what this is what other scholars have called scarring, that there's a scarring effect of the criminal justice records. And we need to think about how quickly that scarring likely occurs.
If it happens sooner than five years, say for certain types of criminal records, we aren't going to be able to see in the data because pretty much everybody had to wait that long.
And so these are all things that we'll try to take a look at.
Moving on then, because we can probably again tell the story fairly quickly because it's going to look similar with the life satisfaction outcomes. Renee, the life satisfaction outcomes, things like identity and overall happiness.
And and we asked a question about a sense of agency. How much people felt or how much agency people felt in their lives, meaning the ab to change or affect or direct their lives in the way that they wanted. Where did that information come from?
[00:15:51] Speaker D: All of that came from surveys.
[00:15:53] Speaker E: Got it.
[00:15:53] Speaker A: And so this was all survey information. We had a whole survey module on this question.
And again going Ryan, back to you was to remind everyone, was life satisfaction or identity, was that a a principal reason why people sought expungements?
[00:16:11] Speaker E: Yes, it was the overwhelming like 98% of respondents chose an identity based reason as one of many reasons that they or potentially many reasons that they sought an expungement. So yes, the clearing of their record for purposes of like how they viewed themselves as a strong motivating factor for almost everyone in the study.
[00:16:33] Speaker A: And again, I think in terms that we discussed in one of our previous podcasts, in terms of rank, the identity variables were the most popular. The identity rationale was the most selected, put it that way, followed by employment, followed by housing. So identity was nearly 100%, employment was around 80%. And if you combine the section 8 and apply for a new apartment rationales, then they hit around 50%. So these are things that are on people's minds when they're seeking expungement. Ryan, again, big picture with respect to the identity variables, was there any discernible effect with respect to achieving expungement?
[00:17:11] Speaker E: Again, no. The self help and the legal representation groups look similar to each other statistically in terms of their life satisfaction and identity constructed scale variables to each other
[00:17:27] Speaker A: here it's a little bit harder to tell whether we can exclude policy relevant effects because nobody has really been able to figure out what policy relevant effects are for the statistically uninitiated. Ryan, what did we measure this in terms of? In other words, what were we say that we can probably it's probably not true that the effect is larger than something. But what are we measuring the something in?
[00:17:53] Speaker E: Yeah, so we have to reference back to the question that was initially asked on the survey. And so for life satisfaction, we literally asked just overall, how satisfied are you with your life nowadays? And individuals were able to respond on a scale from 0 to 10. So 11, 11 points on that scale. And we were able to exclude that differences greater than roughly two and a third point so that the differences between the self help and the legal representation were greater than two points on that scale.
So what that what, a five, what an average score of a five versus a seven versus a three means is. Is not specifically well defined, but we do know that anything smaller than those gaps or anything larger than those gaps can be ruled out.
[00:18:44] Speaker A: And then for the life composite measure that we used, can you. I mean, it's okay to use a nerdy term or two here. Can you just let everybody know very briefly how that was constructed?
[00:18:56] Speaker E: Yeah. So for the life composite score, we asked a series of questions that all relate to each other in terms of life satisfaction and identity. These range from things like the life satisfaction, like how satisfied are you? But also things like how happy are you feeling? Or how anxious are you feeling? Instead of individually looking at each one of these scales, we did something called a principal component analysis, where essentially you combine each of these separate questions that you think are all measuring something related to each other, and you let basically the math construct, a single variable that represents how alike all of these measures are to each other. So that composite score that we pull out of it is essentially the kind of interconnectedness of all of the separate components that we use to build it. So all the separate items that we asked about life satisfaction and life happiness and anxiousness, and all these things we put into one component score. This isn't on any type of. It isn't on a scale anymore. It's what a statistician would call normally distributed. So it's centered on zero, and it kind of moves between negative four and positive four. It can move farther than that, but it gets much rarer to go farther on this specific analysis or for this specific variable component. There's a lot of complicated math in terms of how that works, but that's a, That's a broad overview for it.
[00:20:23] Speaker A: And is this a fairly standard technique?
[00:20:26] Speaker B: Yeah, yeah.
[00:20:26] Speaker E: Statisticians love using this. Psychologists specifically just. They love take. They love just throwing as many things inside of a principal component analysis they can to. To reduce the amount of things you have to work with. It's a fairly powerful tool and it's one that's very well understood in terms of how statistically it works.
[00:20:45] Speaker A: And just to clarify, the scale here was for the life satisfaction. Was it on the points of the 11 point or the 11 point scale or was it on participant standard errors?
[00:20:58] Speaker E: Oh, right.
[00:20:59] Speaker A: I think it was participant standard errors. Right. So we're measuring how much variation there was within each participant and then seeing if each participant can include. That was just the scale that we used. And we use the same thing for the, for the principal component analysis. So for those of you who are technically adept listeners, you will know what this is talking about. The big picture message, though, again, as you said, Ryan, was no statistically significant difference. And we are somewhat confident that the surveys we could detect statistically significant differences because we did with respect to certain questions. We detected statistically significant differences with respect to the deterrence for both job application and housing application, but not for any of these other variables. So, Marilyn, I'm going to return to you and ask you the same question, which is basically, I'm depressed. And especially since this was the life satisfaction, the identity, the idea that my criminal record is not who I am anymore was the overwhelmingly most popular choice among rationales for why folks were choosing expungement. And it doesn't look like achieving expungement actually helps those variables. I'm wondering, can you give me some guidance? Even if it's the same thing as what you said before, I'm going to
[00:22:15] Speaker B: give you a little different guidance this time because anecdotally,
[00:22:21] Speaker C: I
[00:22:24] Speaker B: very rarely hear somebody not say, you don't understand what it means to have that cloud lifted from me to begin, even if we're not expunging all their records to begin to see that my changing my life is recognized by the court system that helped create my identity as a criminal.
So I think whether they're reflecting that long term in a survey, it may not be getting into their core, but they're seeing it a little bit. The other part of this is worth saying, and this may apply to all of it, is because of this waiting period, for some people, we are expunging every. We're completely expunging their criminal record. But there are some of our participants who continue to have a record even though we've expunged everything that's eligible. They haven't been able to pay all their fines or they haven't waited long enough or whatever. So they're not necessarily record free when they, when we're looking at these statistics got It.
[00:23:42] Speaker A: Although I think, and for the latter, I think that may very well be the case in terms of the people that you continue to work with, Marilyn Ryan, if I'm right, with respect to this, these participants, kls, am I correct, was able to completely remove the record for the overwhelming majority of them, is that right?
[00:24:01] Speaker E: Yes. So Marilyn and KLS were kind enough to differentiate between individuals who got full expungements and those who had partial expungements. And so for these analyses, we did consider individuals with partial expungements as not having succeeded in clearing their record.
[00:24:17] Speaker C: Yeah.
[00:24:17] Speaker A: And in terms, and sorry, just in terms of the fraction of people who were able to achieve total expungements, I seem to recall that was very high. Is that right?
[00:24:27] Speaker E: Oh, yeah, no, sorry. I'm sorry. Actually, we went back and forth on that. So I'm actually not sure that that actually did exclude the partial expungement. But irregardless, it was like less than 2% of, of the group that of people who had some expungement but met were not fully record complete.
[00:24:45] Speaker A: So it was an. It was. We were startled, quite frankly, Maryland. By the success of KLS and achieving these complete expungements. So it may be that's a partial explanation. It may be that simply our, our survey questions were too coarse. They were insufficiently sensitive instruments to pick up some of the, some of the life satisfaction improvements, but we just didn't see anything so far.
This is where we're going to need to wrap it up. And so let me just see if there are any final thoughts. Patricia, we'll start with you about whether there are any final thoughts before we call it a day in terms of our report, our podcast about this particular study.
[00:25:21] Speaker C: Yeah. For our life satisfaction. Okay. My background is in psychology. So thinking from that perspective, it's not always easy to measure stable changes in that. And part of the reason is because we adapt. Right. So let's say you get a much better job. You're being paid twice as much. There's going to be a period where we're feeling over the moon. You think your life is the best, but you're not going to keep feeling that for the rest of your life. Right. That becomes your new normal. Your average day is just better now. But then as time passes, how's your day going? It's average. It's better than it was in the past, but you're not describing it better because that's normal for you now. So it's not always easy to capture those kinds of Longer term, even if you are happier, the way you describe it may not be like I am ecstatic every day.
So it's not an easy outcome to measure in a survey.
[00:26:06] Speaker A: Yeah, excellent. We did not find any validated survey measures for the types of things we were interested in, which is a further issue. We were kind of making these questions up. Marilyn, a final word from you.
[00:26:17] Speaker B: I hope this doesn't lead people to decide that expungement is not a worthwhile endeavor because it that my experience is just to the contrary and for whatever reason, I hope people will continue doing that and making those services available to people.
[00:26:39] Speaker A: Ryan, final word from you.
[00:26:41] Speaker E: Yeah. I want to echo something Patricia said last time and that Marilyn has brought up again, which is to say that we might not have seen many concretely positive outcomes, but we're not saying bad things either. Like if the fight isn't benefiting from continuing to maintain access to these records, if we can't get, if it's not helping, then we should be expunging people's records. We should be letting people move past these past mistakes because it's not costing us anything. And it, and even if all they feel is some sort of positive effect, sometimes short lived or positive notion attached for identity, that's that seems worthwhile to let people move past.
[00:27:20] Speaker A: And Renee is the principal architect and implementer of this study. You get the last word.
[00:27:26] Speaker D: Oh, what a joy.
Now we've done a number of episodes about this project and of course we've been working on this project for a number of years. And thank you Marilyn, for all the effort you've put in in that time in addition to everyone else on this call.
And I continued to think about the scarring idea, Jim, that you brought up before. And I summarize that as the damage has been done by the time the waiting period is over. And so I'm really interested in this work and contributing to the idea that maybe we should now be looking at policymaking around waiting periods. And we've talked before about automatic record clearing and then evaluating those techniques as potentially moving the needle because I think I hear on this call everyone saying this is not a record clearing is not a bad thing. We shouldn't stop trying to help people clear their records. But if we want to have this larger impact, maybe we need to consider other solutions.
[00:28:30] Speaker A: Super. I will just note that there are potential arguments against criminal justice record clearing and I for one am actually persuaded by some of them. I don't know where I come out on the whole big picture policy records but there are at least arguments surrounding accountability and visibility of an essential government function to the public.
We'll leave those for another time.
So thank you so much to everyone involved. Patricia Gansert, Renee Dancer, Marilyn Hart, Ryan Halen, and I'm your host, Jim Greiner. Thank you so much everyone. Proof Over Precedent is a production of the Access to Justice Lab at Harvard Law School.
Views expressed in student podcasts are not necessarily those of the A J Lab.
Thanks for listening. If we piqued your interest, please subscribe wherever you get your podcasts. Even better, leave us a rating or share an episode with a friend or on social media.
Here's a sneak preview of what we'll bring you next week.
[00:29:27] Speaker F: So the Law Enforcement Officer Bill of Rights was first passed as a sort of landmark law in Maryland around the early 70s.
And in it it had a sort of laundry list of protections that police officers had in the course of their work. And soon these police protections really that were directly modeled after the Maryland statute spread across a lot of the country. Like I said, sometimes it was through state legislation, sometimes these conditions were inserted into police union contracts during bargaining efforts. And so lots and lots of states now have these types of law enforcement officer bill of rights laws. Sometimes they differ on minor points, but the general structure of it is quite extensive.
From one source I researched, it said that 19 states currently have some form of law enforcement officer Bill of Rights, or leober, as some people shorten it to. And so the leober has been a very, very pertinent issue, especially, you know, even just a few years ago with a lot of public scrutiny and attention on police brutality, especially against communities of color. Right. And so people began questioning whether these enhanced protections for police officers were impeding access to justice in some way.
