Episode Transcript
[00:00:00] Speaker A: Imagine a justice system built on rigorous evidence, not gut instincts or educated guesses about what works and what doesn't.
More people could access the civil justice they deserve.
The criminal justice system could be smaller, more effective, and more humane.
The Access to justice Lab here at Harvard Law School is producing that needed evidence. And this podcast is about the challenge of transforming law into an evidence based field.
I'm your host, Jim Greiner, and this is proof over precedent.
[00:00:34] Speaker B: Okay, here we are with proof over precedent.
Back with Renee Dancer. She's been on our program before.
Welcome, Renee.
[00:00:42] Speaker C: Thank you, Michelle. It's nice to see you.
[00:00:44] Speaker B: And you.
So we've learned some fun facts about Renee in the past, and we're going to find out some more. I hope, if you have any hidden talents that we don't know about you.
[00:00:53] Speaker C: I have been preparing some.
Okay. So this time my fun fact is that I have a little bit of an addiction to periodicals. I really like to have actual periodicals, and I subscribe to them and I change which ones I subscribe to. Some of this is because my nieces are in the girl scouts and they do periodical sales. But I particularly like high fashion periodicals.
But I'm a little. I've been a little disappointed because I noticed that they stopped putting perfume samples, so you can no longer, you know, tear the thing and smell the perfume. But I did get a new one this last week, and it is Elle. So if we're, you know, if we need sponsors, they have brought back the perfume samples.
[00:01:47] Speaker B: Oh, there we go. That must have sold you.
[00:01:49] Speaker C: Yes. I'm not sure, like, what.
Why it changed. And I'm sure there are many good reasons, but I do miss them.
[00:01:57] Speaker B: I'm getting such an interesting portrait of a world WWF fan who likes her fashion magazines.
So it's just. Yes, it's very interesting every time you're on the show.
All right, so, Renee, you're here to talk about a recently launched project. Why don't you give us a brief summary of the study?
[00:02:15] Speaker C: Sure. So this is the community diversion program.
It is a project that evaluates a diversion program which is. Is a program in the criminal justice system which generally diversion programs attempt to pull people from that traditional case processing and put them into some different track in the justice system. This one is a diversion program in municipal courts and is focusing on persistent low level offenders. So individuals who have been arrested for low level offenses and that have been arrested a few times for low level offenses. So at least two, but sometimes more.
[00:03:02] Speaker B: Can you give us an example of what some of those Low level offenses might consist of.
[00:03:06] Speaker C: Sure. So typically these are misdemeanors. You're not going to see any felonies on this. Municipal court jurisdiction is often for first appearances, but these are cases that could be disposed of in the municipal court as well, which means their jurisdiction is often lower level crimes. So that could be shoplifting, it could be resisting arrest, it could be public or just obstruction of public systems. So if you kind of get in the way of someone else getting arrested, you might be charged. There are some drug offenses, low level drug offenses. There are some intoxication related offenses as well.
They actually have 97 different offenses that have gone through this program, though, in the past.
It runs a lot of the low level offenses.
[00:04:00] Speaker B: Okay, all right. Widespread. But are juvenile offenders included?
[00:04:04] Speaker C: No, juvenile offenders are not included in this study. And often juvenile offenders are processing in a different way through the justice system. There is a separate juvenile justice process for those who are juveniles, but for our study, we are not including juveniles in the study.
[00:04:23] Speaker B: Okay. And can you tell us more about the study, how it came about and who approached whom, where it's taken place?
[00:04:29] Speaker C: Sure. Yeah, absolutely.
So this study is taking place in Toledo. We're working with the Toledo Municipal Court.
This study came about from an old fashioned phone call.
The court administrator in the Toledo Municipal Court, Lisa Falgiano, called me on the phone.
[00:04:49] Speaker B: Wow.
[00:04:50] Speaker C: And told me about their diversion program and about how they would really like to have a rigorous causal evaluation to understand its effects on the community and the court generally. And so we had a conversation on the phone. We have continued to have conference calls rather than video meetings. It is. They are a phone call court. And so that's been a fun change.
[00:05:21] Speaker B: Refreshing.
[00:05:22] Speaker C: Yes, exactly.
[00:05:23] Speaker B: Was this a relationship you had with Lisa previously, or she found out?
[00:05:26] Speaker C: It is not actually. She just.
They found us. The Toledo Municipal Court found us because of our work, you know, our exclusive work on randomized control trials and causal evaluations in the law. And then.
Excuse me, Lisa read through all the bios of everyone on our website and realize that I have a court administration background and that's why she called me. She did tell me that.
[00:05:53] Speaker B: So, yeah, that does sound very old fashioned.
So can you tell me about the initial reception from the community about the study? How is it being embraced by the community?
[00:06:05] Speaker C: So we had one of the kind of most eventful launches of, with this study of any study I've had. So I did a number of site visits prior to launching, and then on the launch visit, the court and the city arranged for A press release and a press conference.
And they asked me to speak and I hadn't prepared because I did not know.
But that was my own, my own blind spot. I should have realized that was going to happen when I. Because I knew about the press conference. But.
And so we had, we did a lot of preparatory conversations with a lot of different stakeholders within the community and within the city and county level government. And so it was.
I mean, it was a really great launch. It was just the most eventful one I've participated in.
[00:07:08] Speaker B: So that's not the norm.
[00:07:10] Speaker C: That is not the norm. Although, I mean, they want everyone to know about the evaluation because they want people to.
They want to know truly the results. They want to know what's happening with their program, is it affecting the community in the ways they think it is. And so they're proud of this program. So I mean, I appreciate that they did all that legwork to make sure the community knows what they're up to.
[00:07:41] Speaker B: That's great to hear that they're proud of it.
I believe you participated in a demonstration diversion class for community stakeholders. Can you tell us what that looked like?
[00:07:50] Speaker C: Yeah, absolutely, we did. That was one of the events that they arranged around the kind of launch events is to have not just me and Miguel, our colleague here at the lab, participate in the actual diversion, the class component of the diversion program, but they had all of the relevant stakeholders, all of those offices represented, most of them represented by their highest official. So that included the prosecutor, the chief prosecutor was there, that included the public defender's office, that included law enforcement, all of the different jurisdictions. So you might know that often law enforcement jurisdictions kind of overlap or there are kind of smaller catchment areas that they have within a larger city or county, and that included other kind of community members and other partners within the community that might also work with individuals who might be participating in this diversion program.
So it was great. Their facilitator is really fantastic. The goal of the program is to have kind of two. It's twofold. There's an education component that is focusing on giving people kind of space and room to talk about what happened and how they might have changed their thinking and reaction to what was happening to them in a way that might have produced different outcomes, specifically maybe avoiding the arrest.
So the education component is really about understanding accountability, thinking about decision making, how to de escalate situations.
But then the second component, which is, you know, kind of another core tenant of this diversion program is this deliberate connection with community based resources that are tailored to each person in the class about what their needs are and what they've talked about in the class and been willing to share to really try to address some of what might be the underlying issues that have perhaps resulted in persistent engagement with the criminal justice system. And so we think of these as potential exacerbations of poverty.
[00:10:25] Speaker B: Right.
[00:10:25] Speaker C: So these are things like helping you figure out how to get your driver's license back so that you can drive to work, figuring out how to apply for other benefits programs, helping you get connected to job training.
So there's lots of resources for various job training. And so it's really about helping people have stable housing, have stable employment, work on family issues. They have clear up anything that might be a barrier to their success on those elements.
[00:11:01] Speaker B: So can we also talk about recidivism as a measurement of, you know, quote, success in the program? How does. How do you measure that?
[00:11:12] Speaker C: Yeah, so recidivism is definitely a measure of success in this program, specifically because we're focusing on folks who have already recidivated, right. They have already been arrested at least twice for similar.
Similar levels of offenses. Maybe not similar offenses, but probably also similar offenses. So for us, we are defining recidivism in this study as future arrest.
[00:11:41] Speaker B: Not.
[00:11:41] Speaker C: Future charges, future arrests. Because it is the arrest that brings a person to their first appearance, which is, regardless of what happens, they will eventually come to that first appearance. We want to really look at arrests, but also the theory of change in the program that I just described is that educational component attempting to kind of address the issue of, or help people navigate situations where arrest might actually be an outcome, but it doesn't have to be. Right.
[00:12:17] Speaker B: So then would a measure of success be zero arrest? Is that basically what you're looking for from participants?
[00:12:24] Speaker C: I mean, that would be a great measure of success. I think that's probably unrealistic, right? I think what we are expecting is that folks who participate in the diversion program as compared to the alternate condition, will have fewer instances of recidivism than folks who participate in the alternate condition. And the alternate condition in this study, if I'm jumping your question, go right ahead. Is community service. And so oftentimes people think of randomized controlled trials as the program or. Or nothing. In this instance, we are, instead of saying it's the program or processing through the criminal justice system as one normally would, we're offering folks the opportunity to do community service as an alternative condition.
And for us, we think that is a suitable alternate condition because it definitely does not have the same components as that diversion program. There's no.
There's no systematic and deliberate education. There might be education and community service, but it would be haphazard and random.
And there's definitely no deliberate connection to those resources.
And it does require that kind of the same amount of agency to get oneself to the community service provider or to get to the diversion class as well. So in any event, we would expect folks who aren't having the benefit of those tenets of the diversion program to have continue on a similar path of recidivism as what we were seeing prior.
[00:14:13] Speaker B: Okay.
And so are some of the socioeconomic outcomes also measures of success?
What you're talking about the employment. If you find that this participant now has a job and now has housing, is that something that you're checking certain boxes that yes, they are on their way or.
[00:14:31] Speaker C: Yes, absolutely. And we could have chosen any number of socioeconomic outcomes because the diversion program itself really is focused on the individual. And so what the individual needs, they will, you know, try to link them with some sort of community resource that might help them.
What we found were the most needed resources are housing and employment related resources. So we chose to evaluate those two socioeconomic specific economic outcomes specifically.
[00:15:06] Speaker B: That's great. And so when you start one of these studies, do you go into it with your own hypothesis at this point? Can you share that?
[00:15:17] Speaker C: I think it'd be hard not to.
So we definitely want to work with the field partner to know what they think their program is doing or their intervention or why would we study this thing? Why do you think it's worth studying? And that actually is usually how we get to what are the outcomes we're looking at. Because this is their theory of change. Right. But we, of course, look, also have some expertise and opinions about what we think the program might be doing as well. So in this program, I think we thought, and I think many of the organizations we started to work with on bringing this research idea to reality thought this is a really light touch program.
It's three and a half hours one time, but it does require the individual to actually schedule and go for that three and a half hours.
And these are low level offenses. And so there might be.
So the punishment that they might face is not the same kind of severity that they might face in higher level crimes. So for us, we thought this is a light touch program. And so the effects might be kind of difficult to actually see.
But the program is so replicable, it's super inexpensive for courts to stand up. And there are. It is scaling up like as we speak. And more importantly, like we are arresting people for these crimes. So it begs the question, there are a couple of questions that are begged by that.
Should we be.
We'll just throw that out there. But also like these are not.
This is, this is not the type of offense that I think we traditionally think of as like confining people for long periods of time. Right. So what else. What is the plan? If we're not going to. If that's not the. If we're not going to do that. I mean, so I have an opinion, but we'll.
[00:17:42] Speaker B: Okay. Well it sounds like there's a no fail trial at least to see anyways.
Are we aware of a weigh in from psychologists or sociologists on these type of programs? Are you aware of any, I don't know, similar studies I guess that they would have done?
[00:18:01] Speaker C: That's an interesting question.
I don't know that I have done a literature scan from other disciplines looking from an outside perspective in. But we have done a literature scan on diversion programs and research around diversion programs, specifically causal research.
We typically do try to find other causal research to see what has happened. And in this case there is not cause. This is a unique program because of the population. It focuses on persistent low level offenders. Many diversion programs focus on low level offenses but focus on first time offenders.
And also this goal of studying socioeconomic outcomes in addition to recidivism also makes this evaluation unique from those that have come before. Those that have come before have really focused exclusively on recidivism as an outcome.
So they're different populations outcomes. But the studies that have happened and there actually is a modest amount of research in diversion programs generally have shown null results, which means that there's no outsized effect one way or the other. It doesn't mean there's harmful effect, but it also doesn't show helpful or that this program is far and away the best program.
So we have reason to believe that could happen here. But we also know this diversion program is just different and this evaluation is just different. So it may not, but if it does, I don't think we're afraid of that either. Right. We're okay with the null result.
[00:19:59] Speaker B: Yeah, knowledge is power. We'll figure it out.
One thing we haven't, if a participant wants to. That was redundant.
If someone wants to participate, do they have a choice or is it as an rct? Are they just randomly assigned a role?
[00:20:21] Speaker C: That's a great question. So yes, everyone has a choice in this study, unique to other studies, we have a study Coordinator who is on site at. At the Toledo Municipal Court because our invitations to participate in the study are happening at that first appearance prior to the actual court being in session.
So everyone is told about the study, told what it means to participate in the study, told about what data we'll look at, and then they're invited to participate in the study. They can say no, it won't matter. It won't change the trajectory of their case.
We also, of course, tell them about the random assignment so that they recognize that it's a 50, 50 chance to be either in the diversion program or in the community service in this study, unlike many of our other studies. So we are. If somebody says they don't want to participate in the study, they may still be referred to the diversion program and still have the opportunity to participate in the diversion program. We just won't be evaluated. They won't be a part of our data set.
And that is happening. There are people who are declining to participate in this study, but still might participate in the diversion program.
And you know, we spend a lot of effort, actually the court spent a lot of effort really talking with all the stakeholders about the evaluation and about how we want to make sure that the people that typically should be referred to the diversion program are and that they are also a part of that. We're giving them an opportunity to participate in this study. And so we have had a higher than expected enrollment in the study, but we've also had people who are so excited to do the diversion program that they don't want to risk the chance of being randomly assigned to community service, even though it's kind of both are a similar burden.
[00:22:28] Speaker B: Got it. And what numbers are we talking to when you say that you've had a higher than expected?
[00:22:33] Speaker C: Yeah, we have 121. I looked today. We have 121 people in the evaluation.
And we launched in March of mid March of this year.
So we are above pace, Ahead of pace, ahead of our expected pace.
[00:22:53] Speaker B: Excellent.
[00:22:54] Speaker C: Yes.
[00:22:57] Speaker B: So correct me if I'm wrong. The. The Toledo Municipal Court launched their diversion program in 2018. Is that right?
Did I come across that information incorrectly?
[00:23:06] Speaker C: I don't remember.
[00:23:08] Speaker B: I guess my question is more if that is the case, why would they be looking at conducting an RCT at this point, seven years down the road?
[00:23:16] Speaker C: Oh yeah.
[00:23:17] Speaker B: But that might be more question for the court.
[00:23:19] Speaker C: Yeah, so they actually that first call from the court administrator was in 2019.
So we have been working to start this evaluation for a really long time. And I wasn't certain we would get there. And it has. That had nothing to do with the court. The court actually like really hung in with us. It was all about finding research funding to do the evaluation. We were struggling to, and this goes back to something I said earlier. We were kind of struggling to convince funders that evaluating a Light Touch program was worthwhile and that these programs are being used, they are being replicated. Real people are affected by these programs. It is imperative that we evaluate them.
[00:24:13] Speaker B: So in case you haven't, listeners haven't heard about how long these RCTs can take, this is a reminder that it took what, six years or something just to get rolling here. So, yeah, a way to stick with it.
So you did find funding? Obviously, we did. So who were. Or is that something I can ask?
So can you tell us about who is funding this project?
[00:24:35] Speaker C: Yeah, we are receiving support from Arnold Ventures.
[00:24:39] Speaker B: Great. And so Arnold Ventures, they're your funding partner, Toledo Municipal Court, helping you out in the field.
Is there anything else regarding your partnerships and who's supporting this project that we should be aware of?
[00:24:53] Speaker C: So we, you know, I mentioned the kind of eventful launch that really also was part of the just community wide support for the project. So we attended a city council meeting where they like, you know, all of those are open to the public.
We worked very closely with the prosecutor's office, very closely with the public defender's office in Toledo. So we work very closely with law enforcement and the incarceration facilities.
And we have some pretty amazing data partners as well.
We're working with the Department of Corrections. Toledo. And Ohio generally is part of this kind of consortium of criminal justice data where all of that data goes to one place and then you get it from that one place and it's such a dream.
[00:25:52] Speaker B: It's the way life should be.
[00:25:54] Speaker C: We're also working with some of the housing services providers and the departments of labor and it's been really just a community wide effort to do this evaluation. We've had great support.
[00:26:09] Speaker B: Excellent.
So as far as the timing of the study is concerned, when.
Okay, we've already talked about the build up to this and to get to this point of enrollment, when do you think you'll get to the point where you have some shareable results?
[00:26:21] Speaker C: Oh, this is going to disappoint you. Oh no, because we talked about the buildup.
So we are, Arnold Ventures is supporting us to enroll for 36 months. So that means we'll be bringing people into the study for 36 months. We're following everyone in the study for a period of time, two years. So once somebody enrolls, we'll continue to look at their criminal justice system, housing related and employment related data for a period of two years. And so like that last person that enrolls, we still have two years to follow them after that. And then we've been given nine months to do a full round of data analysis and report writing. So all told, we're in for six years.
I will say that is an unusual length of time for the RF for the request for proposals that we responded to. For Arnold Ventures, we kept our timeline within their regular normal timeline, but they do peer review for all of their proposals. And every single reviewer said, you know, if you enrolled longer, you'd have more statistical power.
And we said, yes, we know. And Arnold Ventures says, yes, that's right.
So they graciously and generously supported a longer period of time. Now the court really has their own kind of internal goal of hitting our target enrollment sooner than 36 months. And they're doing a great job of it right now. So that could happen. Okay.
[00:28:01] Speaker B: Okay. So hang in there, listeners. We'll have an update down the road a couple years probably at least.
You know, assuming we come out of this with some idea of a best practice going forward, do you yourself have any thoughts on how we can best share this evidence based approach with other court systems?
[00:28:21] Speaker C: Well.
Oh, I was like, that was such a bait and switch question.
[00:28:27] Speaker B: Yeah.
[00:28:31] Speaker C: I thought you were going to ask a different question when you started that we can, you know, how best to share it.
I think courts are really interested in like understanding the evidence around the programs that they're doing. So for me, I mean, honestly, Judge Kuhlman, who is the champion judge of this, of this diversion program and of this evaluation, just giving him the details of the results would be enough because he will be out there telling everyone.
So he's really amazing. It's very important to have somebody like Judge Kuhlman on your project team. And so I hope everyone has a Judge Kuhlman.
But he's already also laid the groundwork for working with kind of partner organizations who are also interested in what comes next after the evidence, putting the policy into practice, helping to replicate the program. If the results suggest that replication.
There are other core who have already expressed the desire to replicate if results show replication. So I think we'll have a lot to say and there'll be a lot of individuals, a lot of organizations wanting to listen.
So, you know, that does also put us in a position where I think 36 months is a long time to wait. So six years is an even longer time to wait. And so I think we can generally commit to doing some periodic updates that can't, of course, give us any causal data because we won't have concluded compiling it. But we can talk about what's happening in the evaluation, who's participating, what trends we might be seeing. Those are things we've done in the past and it makes sense to do here as well.
[00:30:22] Speaker B: Definitely. Absolutely. Well, before we close out, is there anything else you'd like to share about the study? Anything?
Anything you'd like to share?
[00:30:30] Speaker C: Sure.
This study, I think, really highlights the important role that municipal courts play in the overall justice system. And municipal courts are dealing with a myriad of issues. And I think often we don't talk enough about all the things municipal courts are doing. They're high volume case types that are really dealing with individuals at their core, in their personal lives. So it's not just these first appearances where they're seeing every case that comes through, even if their jurisdiction doesn't allow them to keep it. But they're also doing things like evictions and housing related cases and consumer debt cases. They're doing all these like really these type case types that kind of touch the community at a core level. And so I'm really proud of us and excited to be highlighting the important work of municipal courts kind of through this evaluation.
[00:31:28] Speaker B: Excellent. Thank you, Renee, for joining us today.
[00:31:31] Speaker C: Yeah, it's so exciting to have a new host.
[00:31:34] Speaker B: Thank you. We look forward to sharing updates on the Community Diversion Program and discussing one of the many other studies you lead at the Access to Justice Lab going forward.
[00:31:44] Speaker C: Thank you so much.
[00:31:45] Speaker B: Take care.
[00:31:45] Speaker A: Proof over Precedent is a production of the Access to Justice Lab at Harvard Law School.
Views expressed in student podcasts are not necessarily those of the A J Lab.
Thanks for listening. If we piqued your interest, please subscribe wherever you get your podcasts. Even better, leave us a rating or share an episode with a friend or on social media.
Here's a sneak preview of what we'll bring you next week.
[00:32:10] Speaker B: You know, studies have found, including by Professor Yang here at Harvard Law school, that chapter 13 does have, you know.
[00:32:18] Speaker C: Financial benefits for the people who are.
[00:32:20] Speaker B: Able to complete it. But again, the failure rates are pretty high because of that difficulty. So as you might imagine, there really is something of a difference of the people who might be best equipped to navigate a chapter 13 versus people who might be better served by going the simpler route and filing Chapter seven.
