Episode Transcript
[00:00:00] Speaker A: Imagine a justice system built on rigorous evidence, not gut instincts or educated guesses about what works and what doesn't.
More people could access the civil justice they deserve.
The criminal justice system could be smaller, more effective and more humane.
The Access to Justice Lab here at Harvard Law School is producing that needed evidence. And this podcast is about the challenge of transforming law into an evidence based field.
I'm your host, Jim Greiner, and this is Proof Over Precedent.
Welcome to another edition of Proof Over Precedent. My name is Jim Greiner. I'm the faculty director of the Access to Justice Lab. And I have a remarkable group today where there are six of us on this podcast recording because so many of us have worked on today's topic, which is the Mother UP Pre pilot program, an evaluation of a guaranteed income project targeted at participants who have experienced or in danger of experiencing getting involved with the child welfare system in D.C.
and so I'm going to ask everyone one by one to go around and introduce themselves and then we'll get started with today's topic. So. So, Mandy Mobley Lee, can you introduce yourself to start with?
[00:01:26] Speaker B: Sure.
My name is Mandy Mobley Lee. I'm an associate director at the lab and my role on the Mother up project was that I was the lead researcher on phase two of the project,
[00:01:39] Speaker A: which is the topic that we're talking about today because we have a report on the phase two pre pilot. Super. So just in the order that you're appearing in my zoom screen. Renee, who are you?
[00:01:51] Speaker C: Hi, I'm Renee Dancer, researcher at the Access to Justice Lab, working on the continuation of this project.
[00:01:56] Speaker A: Terrific.
Miguel, who are you?
[00:02:01] Speaker D: Hi, my name is Miguel Von Fedak. I'm a project manager with the Access to Justice Lab and I helped analyze the interview transcripts for this study.
[00:02:09] Speaker A: Fantastic. Patricia, who are you?
[00:02:12] Speaker E: Hi, I'm Patricia Gansert. I'm a data analyst at the lab and I analyze the quantitative data.
[00:02:17] Speaker A: Terrific. And Alina, who are you?
[00:02:19] Speaker F: Hi, my name is Alina Yashanu. I'm a research assistant at the lab. I helped conduct the interviews for phase two of MemberApp and analyzed the data emerging from the interviews.
[00:02:30] Speaker A: Terrific. So, Mandy, why don't we start with you? Could you just set the stage? What was this project all about? What's it for? And what are. What are the. What is your understanding?
You know, why Is this guaranteed income program or different from other guaranteed income programs?
[00:02:47] Speaker B: Yes. So Mother partnered with an organization based in Washington, D.C.
called the Mother's Outreach Network, and they work primarily with black moms who interact with the D.C. child Welfare Services.
And so what they wanted to do is investigate whether direct cash or monthly cash transfers, two individuals who either had initially been reported to child welfare or had investigations that ended up not being resulting in any determinative findings, whether cash monthly cash transfers would assist them in either resolving an ongoing case or preventing future interactions with the child welfare services from occurring again.
[00:03:41] Speaker A: Terrific. And we may fall into the use of acronyms as they're so common in the law. The acronym CWS child stands for Child Welfare Services. What does the acronym CFSA stand for? Which is another one that we use in the report?
[00:03:55] Speaker B: So that is Child and Family Services Association.
[00:04:00] Speaker A: Yeah, Services Agency.
[00:04:03] Speaker B: I use it so often, I don't. I don't know what it.
[00:04:05] Speaker A: Services agency. Yes. And that is the name of the CWS entity within dc Exactly.
Super.
And so there have been other guaranteed income programs before. In fact, one of the original versions was back in the 1970s during the Nixon administration. It went by a different name, not guaranteeing income. It was called negative income tax.
So the idea that a positive income tax is when a private individual has to pay money to the government, and negative income tax is when the government pays money to the individual.
First of all, Mandy, continuing with you, what distinguishes guaranteed income from a benefits scheme like, say, Social Security disability benefits or something like that? What is the difference between the two?
[00:04:54] Speaker B: Great question.
So guaranteed income is exactly what it sounds like. It's guaranteed, meaning there's no qualification mechanism to determine whether or not you receive the income.
Unlike benefits, sometimes benefits has a threshold threshold where, you know, once you pass a certain income level or certain criteria, you no longer qualify. But guaranteed income is guaranteed if you are selected to receive it. You will receive it regardless of your situation and how it might change.
[00:05:28] Speaker A: Super. So people often use the phrase no strings attached, and that's what this was.
Super. And what Guaranteed income, as I mentioned, has been tested before or assessed before in various situations.
What made this. What made the Mothers Outreach Network, we go by the acronym MON.
What made MON's program different from other programs that have tested guaranteed MON's program?
[00:05:53] Speaker B: They had different eligibility criteria. And so they did look at their own population that they primarily work with, which is black mothers. They also had a criteria that the children or child or children were remaining in the home, and so they had not been removed from the home.
And then another requirement they had was they had to have had some interaction within the last year with cfsa.
[00:06:25] Speaker A: Terrific.
And then in terms of its source of its funding is mond's desire to increase the size of the child welfare system by asking, say, governments to pump more money into it so that then the cws, meaning child welfare system, could give out that money. Or is mon's agenda to decrease the size of it potentially by raising money from, for example, private sources?
[00:06:52] Speaker B: So not to speak for mon, but from my understanding, it is the latter.
They made a very specific design decision not to partner directly with the cfsa.
Part of the reason for that was because there tended to be a level of distrust between the individuals who were being investigated by the agency and the agency. And so mon, in order to establish trust with their participants of the study, the MOTHER UP program, they wanted to make clear that they were in no way partnering or sharing information with the agency.
[00:07:35] Speaker A: Terrific.
And we will, by the way, I will be interviewing Melody Webb, who is the executive director and founder of mon.
And so we will ask, as you suggest, Melody to speak for herself on what mon's goals are.
But that seems clear enough. It's on their website. In terms of avoiding as much as possible the existing child welfare system.
Mandy, one final question for you.
One of the reasons why I was so excited to get in touch or excuse me, to participate in this project was that laws just all over the child welfare system generally, it's not just a sort of standard social services agency. There's a lot of legal involvement with it. And in some ways it resembles the criminal justice system in perhaps some disturbing ways. Could you talk a little bit about that? In other words? And this also would answer the question, why is the Access to Justice Lab doing this project as opposed to a standard economics evaluation shop?
[00:08:43] Speaker B: Sure. Well, the agency does participate in investigations which can lead to hearings and they mirror closely like a trial in the criminal system.
Those hearings could potentially result in the removal of a child from their home.
And depending on how far along that process goes, it could lead to terminating parental rights. And so when you think about your basic human rights and things that you would want to be protected by a justice system, this is happening outside the traditional court justice system. And so that's why, you know, access to justice is getting involved, because it is, it is something that, you know, can significantly impact individuals lives and they don't necessarily have the right to an appeal that they would in a court system.
[00:09:41] Speaker A: Surely then if we're talking about some a process that could result in termination of parental rights and separation of the child from the family and actually ultimately permanent separation of the child from the family, there is a federal constitutional right to counsel in these proceedings. Where there's always a lawyer available, a state appointed lawyer if an individual can't afford one. Is that, is that correct? Is there a federal constitutional right to counsel?
[00:10:09] Speaker B: Not to my knowledge, Jim. I know there is in the criminal system, but not in situations like this.
[00:10:14] Speaker A: Not in situations like this. And actually there's a Supreme Court case that is the very first case that I assigned to my incoming one ls, my incoming law students every year.
That's the first case they read called Laster vs Department of Social Services coming out of North Carolina in which the Supreme Court explicitly adjudicated and declined to establish a federal constitutional right to counsel. It's perhaps one of the most disturbing.
Not the whole thing so much, but the whole case, the whole way the case is written. And we'll include a link on it, a link to the opinions in the podcast on the podcast website. So the design for the mother up Pre pilot, phase two pre pilot was just to enroll about around 20 participants and to randomize the participants to one of two groups. One was called the direct cash transfers group, which was the group that received $500 a week for a month for a year for so 12$500 payments. And then the. And also the opportunity to receive comp for participating in surveys and interviews.
And the compensated research group received the opportunity to participate in surveys and interviews with compensation, but not the opportunity to receive the $500 a month for a year. And so it was just simply a randomization. And the purpose of the the phase two pre pilot was to test the operational feasibility of a larger full scale rct.
And so that was the first purpose. In addition, if we could find some interesting scientific findings, even if they wouldn't have come from a sufficiently powered study, if we could show some of the things that we would hope that a fully powered study would produce, we would hope to do that as well.
And so for insider baseball folks in RCTs, in other words, for RCT nerds, the easy conclusion, and the most important conclusion was that the phase two pre pilot did in fact demonstrate that a full scale study is viable and that we can pull one off.
The recruitment and enrollment process is harder on the front end than we'd anticipated, but everything else more or less ran smoothly. And in fact, we discovered that the interviews that we had scheduled for about an hour actually were able to conclude typically in around 15 or 20 minutes. So that in a future, in a future study, we could ask more questions and get even more detail than we were able to get.
And so that was the sort of easy conclusion of the study and the report, which of course again we'll link to details that but let's take a look at what, how, let's get dive deeper into how things went, how they operated and then what we were able to find. And then with you Renee, what we will be trying to find out in the future in a phase three pre pilot. So to begin with we did some surveys and the surveys included some quantitative measurements.
We were again the study, this pre pilot only had 20 participants. So we produced examples of what we were able to, what we would be able to do in a full scale study, but didn't anticipate that we'd have sufficient power to do any real comparisons. And so Patricia, you did most of the data analysis for the quantitative part. Can you just walk us through generally what we were, what we produced and then from the quantitative measurements and what sorts of graphs we were able to show from this phase two pre pilot effort?
[00:14:07] Speaker E: Sure. So we measured a number of different things in our survey, all looking at the person's overall well being. Right. So we're measuring a bunch of different parenting and family support, like family functioning, nurturing, how much social support they have, perceived stress, their general self efficacy. So that's kind of personal well being. We're doing a measure of financial well being as well as looking at their income to see how is this changing their financial circumstances. And we have measures of health and of housing satisfaction since obviously that can also be connected to this issue. Right.
And for our full study we're planning to add a new measure of food security since obviously that is often a pressing issue that can get you reported to child and family services.
Now obviously we're not able to do any, you know, very in depth analysis yet. This is a small pre pilot, but we were able to do some take a, take a look at baseline comparisons to two groups and some very basic just visually examining what some longitudinal graphs might look at without doing a in depth analysis.
[00:15:07] Speaker A: And you just used the term this
[00:15:08] Speaker E: is getting a little nerd.
[00:15:10] Speaker A: Oh, sorry. Yeah.
[00:15:11] Speaker E: What exactly is longitudinal an analysis over time. Right. So we are going to be serving people multiple times and we want to see how are their answers changing over time and are the two groups changing in different ways? Because that could show us how this direct cash transfer is affecting people with time. This is getting a little stats nerdy. But one thing that we can look at in a pre pilot like this is just how are the scales behaving? We're asking people to fill out these surveys and we're calculating numbers from them.
I would say overall, they're behaving well. One common problem you can have with these types of surveys is if you have answers that are too extreme, like everybody says they're doing well or everybody says they're doing really badly. Here we're seeing a good variation where we have a range of answers and people are sort of. The means are in the middle of the scales, which tells us that this is doing a good job measuring the ways that people in our sample differ from each other. This is actually capturing all those differences so that we can actually compare them.
[00:16:08] Speaker A: Terrific. And let's just provide this is. Take one example of these. I'm just looking, for example, at our graph, our sample graph on the Financial well being scale.
And that was a. That's a scale that the CFPB spent several years constructing.
And it uses a variety of questions about how does a person, for example, what's their level of agreement with a statement like I have money left over at the end of each month. And obviously low agreement with that. With that statement would indicate more financial. Less financial well being. Excuse me, Or I feel like finances determine my existence.
And in higher agreement with that would indicate less financial well being.
Can you just. You, you did. You did exactly what you said. Just the graphs over time on that scale. What is it, Gemini? What is the range of the scale? For example, does it go from 1 to 5 or what does it do?
[00:17:06] Speaker E: Yeah, so this is a very interesting, complex scale made by the cfpb. They used what's called item response theory to create this. So scores can range from 16 to 91, which may seem a little strange to you how you get those numbers, but this is because they use a complex algorithm to calculate it out of the items they have. So the numbers may feel a little funny if to compare it to some scales, people might be more used to and higher numbers are better. In this case, that means that you report more financial well being.
Our groups in our initial means at the baseline, we're kind of in the middle of that, you know, in the 40s range.
And we, you know, we don't want to draw any firm conclusions from our longitudinal analysis just yet, obviously, since, you know, we have a small sample here. But overall, in just our little pre pilot, they were relatively steady.
[00:17:56] Speaker A: And just to say though, that the pre pilot that took took observations because we were just assessing the operational feasibility of the program, we only took measurements up to about 12 weeks out. So it's not clear that we would have.
[00:18:09] Speaker E: So we wouldn't necessarily expect a dramatic change in that period of time. Right. Since the scale measures people's confidence in their financial ability. So after 12 weeks, we're not necessarily expecting people to really have a whole new mindset on their finances.
[00:18:22] Speaker A: Terrific.
Anything else that you'd like to add, Patricia, on the. On the numerical aspects of the study, on the quantitative aspects of the study?
[00:18:31] Speaker E: No, I think as we get into the full study, we'll have more exciting findings to report quantitatively.
[00:18:38] Speaker A: Terrific. Okay. And then one of the other aspects of the study of this pre pilot was doing something that the Access to Justice Lab had never actually done before, which was implementing an interview program with participants and interviewing them at enrollment and then at three subsequent times.
And that produced follow up, that produced transcripts, which we then did some qualitative coding of. And so Alina, let me talk with you about how that went, because you did many of these interviews. I cannot remember. Were you involved in any of the enrollment interviews or did you join the lab after the enrollment interviews were finished?
[00:19:15] Speaker F: Yeah, so I did a couple enrollment interviews. We actually ended up having 19 participants in total.
And I think I must have enrolled maybe 15 participants. And then I did the follow up interview rounds for all participants.
[00:19:33] Speaker A: And one thing, people who are. Who implement RCTs or might be interested in how they work, it sometimes has to. You sometimes have to have a conversation with people while you have them on zoom, telling them that they didn't receive something like guaranteed income, even though they were hoping that they would.
What was that like? Because you had to do that, didn't you?
[00:19:55] Speaker F: Yeah, that was in complete honesty, as the person doing the interviews. That was the most nerve wracking part because it was important to maintain the relationship with the participant and ensure that everyone in the study continues to be in the study. It would have probably been an issue if we would have had a huge imbalance between people in the regular pre trial. I'm sorry, in the compensated research group and the direct cash transfer group.
But fortunately that was not an issue.
I think the participants that ended up in the compensated research group, they understood the stakes involved in the study. It is important to establish very clearly how the randomization works. So I think as long as there is that understanding at the beginning, then the participants will be open and responsive and continue being part of the study and also being part of the study. Even in the compensated research group, as you mentioned, that still involves cash payments for all the surveys and interviews. So that incentive still remains for everyone in the study.
[00:21:07] Speaker A: Terrific. And just to Give people an idea. Approximately how much money could folks receive as compensation for participating in the interviews and the surveys?
[00:21:20] Speaker F: Yeah, so for each survey and interview a participant would be receive $60. So given that there were free follow up rounds and the enrollment round that amounted to $240 per person.
[00:21:35] Speaker A: Terrific. And then you did some of the interview, the subsequent interview conversations as well after enrollment. Just give us an idea, what was it like to conduct those interviews? Were, you know, were participants, were they comfortable talking with you? Did they feel comfortable sharing parts of their lives with you? What was it like?
[00:21:56] Speaker F: Yeah, I think the interview questions were phrased very well and they're very respectful. And the transition from one topic to another was relatively smooth. We started out with questions that were pretty straightforward about finance management and participants relationship with their children. And then once we set the stage we moved to the more complicated questions about participants experience with cfsa. And the question that examined this topic was again I think it was phrased very well. It was specifically asking participants to what extent they think that finances played a role in their CFSA involvement. So I think the question was not at all accusatory and participants could express their thoughts about the financial aspects of their case.
And I think we, as the researcher, I was always very careful not to cross participants boundaries and whenever they did not feel comfortable elaborating, I would move on to the next question. So in that case sometimes we may have brief responses to more complicated questions. But at the same time that ensured that the relationship with participants remained consistent and relatively good.
[00:23:20] Speaker A: Terrific.
Terrific. Miguel, let me ask you to comment here. Once we had the transcripts, this was a new experience for the Access to Justice Lab. We had the transcripts and had to figure out what to do with them.
And you had some background in doing this. You had done this before.
But you pointed out for example that there were two sort of big picture ways that one could approach interviews and transcripts and then some details about each of the ways, some in common, some separate depending on your approaches. So could you just give us a broad top level overview of what one does with transcripts once you have them?
[00:23:53] Speaker D: Sure.
One thing that I think people sometimes forget about transcripts is that we do not write how we talk. And so it's seems like something that you don't have to consider all that much. But it becomes a huge problem when you know, people cry or people are stuttering or people are having trouble even just getting their point across. And I mentioned this before getting into how to analyze interviews because just transcribing the interview is a research choice. How do you get emotion into the transcript? How do you get the way that someone talks in the transcript? How do you get the way that there's a rapport between the interviewer and the interviewee into the transcript? These are all big decisions. And I think that this study did a really good job of this because there was an understanding that emotional responses were a part of what we were getting out of the interviews. This is a very emotionally charged experience for most of these participants, and they're discussing traumatic experiences in their lives. So I just wanted to emphasize that a lot of interview analysis happens way before you actually think you're analyzing the interview itself. But then once you are more explicitly trying to get a theme or trying to get some kind of conclusion out of interview transcripts, there's two general camps that it falls into. And obviously, this is kind of a spectrum. No method is just one or the other.
And they're called deductive and inductive interview analysis. Deductive interview analysis just means that you have identified either from literature, from other subjects, from other kinds of studies in this similar area, the themes that you're looking for in these interviews. So you have taken some kind of basis to define the things you're looking for, and then you go and you find all the evidence in the transcripts that speak to that. So that one is very much based on maybe a study that's replicating a different study or a study that's trying to, like, engage with the existing literature and existing research conversation. And then the other form of analyzing interviews is called inductive interview analysis. Inductive interview analysis goes to the transcripts as they are.
In so much as such a thing exists, there is no raw transcript.
But you go into the transcripts as you have them, and you see what themes are being elicited by what's being said. What is a commonality amongst the interview transcripts that starts to speak to some kind of pattern, that starts to speak to some kind of general dynamic. And what's really interesting when you do that kind of thing is that unexpected themes can pop up. Themes that haven't been gotten in the literature, themes that haven't really been talked about before.
And we did both for this study specifically because we understood while, yes, there are guaranteed income programs and there are ways of analyzing guaranteed income effects, we are doing this with a study population that is really particular, really unique, and they experience a very high level of scrutiny in relation to other study populations. And so we understood that we were probably going to get very unique insights into what a guaranteed income program could do for this study population. So we had to do both in some ways because we had to connect it to the larger literature. And then we also had to show that this study population is seeing things and experiencing things that might have just been impossible to get had we just considered what the literature had seen before.
[00:27:48] Speaker A: Terrific, Terrific. And then so to follow up on that, we had the deductive and the inductive. And Alina, you had the task of doing the deductive coding for the transcripts. And I will be actually. And Miguel, then you did the inductive. I'm actually going to be reading some of the. To you, just talking about our big picture conclusions and then reading an example or two of the transcripts, in other words, what the participants said. Because I think the most powerful things to emerge from this phase two pre pilot effort were the interviews and the quotes from the participants in the interviews. But just give us, Alina, just an overall impression of your experience and what you heard and basically what you read when you were doing the deductive coding of the transcripts.
[00:28:40] Speaker C: Yeah.
[00:28:41] Speaker F: So deductive coding means that I started with creating a list of predefined themes. So based on the literature and the research questions that we and Mon had, and I ended up having maybe 10 or so themes.
[00:29:00] Speaker A: What's an example? What's an example of a theme or two?
[00:29:03] Speaker F: Yeah. An example of the theme was basic needs, because the interview questions asked about how participants manage their finances. So we were interested in seeing if they're able to meet all their fundamental needs or if they still had some gaps.
Another theme that I coded was financial setbacks.
So they were very much focused on finances. And then there were more personal themes like parent child relationship, which focused on the relationship between participants and their children.
And so with having created those themes, I went into the interview transcripts and I highlighted different sentences that participants have said, and then I just associated it with that relevant theme. I should note that I used NVivo for this process. I don't know if it's useful for anyone else who is debating which app or software to use when they're coding their interview data. I found it relatively useful, but it is difficult to share it outside of nvivo. So I ended up having to write some Python code to create a CSV file to extract the data from nvo. That was quite messy. So I think retrospectively I might have done it without anything though. But it did simplify the process of matching interview excerpts with a specific theme. So that was relatively straightforward.
[00:30:42] Speaker A: Terrific. And Miguel, you did the inductive coding. Give us just a broad overview of your experience in doing that coding.
[00:30:52] Speaker D: Yeah.
So with inductive coding there are two main choices that you have to make when you first start. And the choice really is how many transcripts are you going to read and how in depth are you going to read them. So I tried to be as expansive in both senses. So I tried to read all the transcripts and I tried to go in and find every little detail that I could within the transcripts. That's doable. When you have less than 20 transcripts, which was our situation beyond 20 transcripts, there's no way to do that. It is way too much information. Interviews are incredibly rich in data and evidence.
And when I was doing it for this study, you kind of fall into this pattern of the first few interviews that you do kind of set a tone. So to try and minimize any kind of framing effects or any kind of bias to like the first interviews that we do, it's a good idea to not read the interviews in the order they were done. It's a good idea to randomize, so to speak, which interviews you're reading and how you're coding them. And so what I would do is just read a transcript and as I was reading Mark, Interesting quotes, Mark, things that are speaking to something bigger that I found in other interviews. And you go along like this and you're trying to just get a sense of larger themes and then what the people are saying about these themes. So there is a sort of deductive expectation here. Like we're talking about income, we're talking about people that have had experiences with child welfare agencies. You're looking for things that relate to income and to child welfare agency experiences. You're not looking for favorite food or favorite color. That's not really important information.
But that being said, you're trying to be inductive in what is the interesting information, what is being said that tells you something.
So I got information from how people would stutter, how people would deny things that they had actually affirmed earlier in the interview. These are interesting dynamics when people self contradicted, when people acted as if they didn't know the answer to a question and then they answered the question later in the interview. Those were really important to me because that spoke to something happening either with our ability to communicate what we meant or with the interviewers with the interviewees ability to make sense of their own situation. And I think that's what was really, really impactful from the indic coding is that you make more of a snapshot of where each interviewer, each interviewee is at. You get a sense of exactly where their expectations are, exactly where their feelings about the situation are kind of convoluted, or where they're clear. And I think that that allows the research, whether it be statistical, whether it be qualitative, to be a lot more applicable to specific examples.
And it makes it a lot more attentive to what people are going through.
[00:34:11] Speaker A: Terrific. Well then, Alina, let's go back to you for just. If you could give us.
And of course, the data set is vastly more rich than this. And I will be sharing some again, some of the quotes from the participants shortly.
Two sentences, maybe three sentences.
What was one thing that you found striking or that you discovered from these transcripts and doing these interviews that you didn't know before about, about guaranteed income and about its potential impact upon mothers who are in this situation? These are single parents who have child welfare involvement. What's one thing that you would take away? And then, Miguel, I'm going to ask you the same sentence, same question. Excuse me.
[00:34:54] Speaker F: Okay, so I'm just looking at our report and this quote is under, under the subsection parent child relationship and Participant 16.
We obviously de identified the names of the participants. So Participant 16 remarked, and I'm starting the quote, I talked to my care worker and it's really important that you give your kids outing times and stuff like that. Being able to do stuff with them outside the home, it makes kids feel good. You know, it can help with behavior problems. I do deal that with my 7 year old and things of that nature. Kids get really antsy when all they have to do is sit in the house. End of the quote. So I think that was really powerful because that was said in the context of the fact that the guaranteed income allowed the participant to spend more time with her children outside the home. And that was a very tangible way in which the guaranteed income intervention did help improve the relationship of the parent with the child. And it did address specifically, presumably one of the reasons behind that participant's involvement with CFSA in the first place.
So that was one specific example that stands out to me.
[00:36:10] Speaker A: Terrific. Miguel, same question for you. Could be from the report or could be something that you just recall from going through these transcripts.
[00:36:17] Speaker D: I'm going to give two of the things I found the most interesting because I think that they speak to different things that the study is bringing out. So the first one is just, I would call it a more Clear cut kind of evidence from the interview transcripts that I analyzed and the ones that I went in depth into. One of the things that I find most interesting is that a lot of participants spoke of what I called a catalyst for their child welfare agency involvement.
And what I mean by catalyst is that while the situation that maybe child welfare, you know, investigated them for could have been happening for a long time prior to child welfare's involvement, there were in a lot of interview transcript, this one moment where the family got reported. And a lot of people talked about how frustrating that one moment was because it painted them in a unfavorable light, or they felt it was very unfair, or it was very discriminatory. And I thought that was very interesting because these catalyst moments are very preventable in many circumstances. Like, a lot of them have to do with, like, children not having clean clothes, people weaponizing reports of the child welfare agency in a way that was used to, you know, pay back for different things. And I found that really interesting because it speaks to how child welfare agency reports aren't being made in a systematic or in a way that really makes sense. At the end of the day, they're. They're quite inefficient if we're just waiting for this catalyst to like, quote, unquote, surveil people.
And I think the second one that I found really interesting when doing the inductive interviews is just how confused participants were, like, how difficult it was for them to make sense of their own situations in life and how they felt about it.
This one was really important to me because in the inductive coding, one of the things that I was most curious about is how participants talked about themselves. So we're talking to mothers, so how they talked about themselves or how they talked about their children.
And it was very interesting when people invoked religion or when people invoked a sense of fatalism or a sense of, you know, resilience to try and just like, say that they could make it out of this.
But what this was, they couldn't really say either. It was. It was very interesting to see, like, how financial hardship isn't just a set of problems that you need to pay or that you need money to resolve.
It's also just the inability to know exactly what's going on in your life. You're so overwhelmed, you're so confused, that even just knowing what the problem is becomes so, so difficult.
[00:39:05] Speaker A: So at this point, I'd like to just share some of the qualitative findings that we were able to make. Again, just to Emphas, we weren't able to make definitive conclusions from this phase two pre pilot. The number of participants was too small and the follow up period was too short.
The phase 2 pre pilot was supposed to be an operational pre pilot. To see if there was the larger study is feasible. It is feasible.
Nevertheless, there are some striking quotes and some striking conclusions from the interviews. And I'm just going to share a couple. First theme is to what extent was the CFSA involvement of the participants? To what extent did that, did that involvement stem from financial issues or could be remedied by financial assistance? So this is the idea, you know, how effective could a guaranteed income be in remedying CFSA involvement or in preventing CFSA involvement or child welfare system involvement?
And basically what we found at the top level just again on this underpowered study. So again, these aren't definitive conclusions, but what it's suggesting from the interviews is that looks like there's a fair degree of promise for cash assistance to avoid child welfare involvement or to solve it to resolve it when it does occur.
So approximately half the participants identified a direct link between financial problems and the CFSA case.
So for example, Participant 1 narrated that a lack of funds caused her child to misbehave, which led to the CFSA involvement. And here's the quote.
A lot of it came from the child's being stressed out because we couldn't afford to do a lot of things. And it also came from the child's being frustrated because they wanted to do something and I couldn't afford to do it.
And then so they would lash out or there were a couple of times where they would steal. And I'm like, you can't do that.
So again, there's a direct connection between the financial circumstances.
A common theme was also an inability to exit a domestic violence situation or an intimate partner violence situation.
Again, finances preventing an exit and that leading to CFSA involvement. So Here's a quote, for example, from Participant 16.
I was in a domestic violence situation with my child's father and I just kept going back to this situation, not because I had feelings for the person, but because that was the only person I had to help with the kids.
And I couldn't pay nobody to help with the kids babysit.
And so there again is a direct link. And about half of the participants in their interviews recognized or identified that direct link. A smaller fraction, less than half, said there was no link whatsoever.
And they just simply said, for example, it didn't at all, it being the lack, the financial challenges. And they didn't referring to whether it affected or caused the CFSA involvement.
Then others initially or actually did deny that there was an involvement, but then explained what they led to the CFSA case and it turned out that it probably was financially related.
And so I'll read you this quote and allow listeners to draw their own conclusions about whether you think we're right. The Access Justice Lab is right in concluding that there was a link. Here's Participant 17's response went to a question asking about whether finances had a role in the CFSA case.
No, I don't think so, if I'm understanding the question right.
So this is like whenever cfsa, I believe it's called whenever I had a case open with my child.
Yeah, well, I guess you could say it perhaps was because I couldn't get to the doctor's office for the child to get seen and stuff.
But health also played a part because I had surgery, so as I couldn't move after the surgery.
So the point there, I think, is that the participant is recognizing that the funds were insufficient to get the child to the doctor, but then is also saying but it was also health and surgery so that they couldn't move.
But of course, finances could have purchased childcare while the participant was down for the count, was recovering from surgery. And so this is an example, we think, where a link is probably there, even though it wasn't the first thing that participants always thought of.
Second thing was just the impact of the financial relief.
Of the 11 participants who received the guaranteed income, seven described using the money to spend more quality time with their children. 6 described using it to resolve issues that explicitly contributed to the initial CFSA case.
And five described using it to pay off bills and debts.
And so these are the sorts of things that relieve financial stress and financial pressure in ways that Miguel mentioned earlier could cause a sort of overall fog on one's ability to deal with one's life situation in general.
In addition, almost all the participants discussed how the additional income gave them more independence.
Some center that independence in ability to exit or ability not to continue with abusive past or abusive relationships and allowed them to feel more safe and secure.
Here's an example from Participant 6.
The money has really helped me a lot. Like it helped me to the point where I can still pay my bills and have some money left over. I can put money away from my kids and like I said, get them, you know, like a new pair of shoes. If they can't fit their old ones, I don't have to worry about, oh my Goodness, where am I going to have money after I get shoes? And I was able to get myself a pair of shoes last weekend and I have not bought anything for myself in like two years, years since I got pregnant.
So shoes were a necessity that basically participants kept putting off and because funds were short. And this is the sort of thing that I didn't think of at least would be, I just take for granted, for example, that I get, I get shoes, new shoes when I need them or when my old shoes are worn out. And that's not then the experience of the participants and their families in this step.
Another thing that the kids did, excuse me, that the participants did with the money was that they allowed them to focus more on the parent child relationship and finances affected this.
So for example, Participant 4, who was in the compensated research group, was not getting the cash transfers, had the following quote, right now like it's my son's birthday is tomorrow and I don't have anything and that's not fair to him.
And Participant 15 referenced quote, the joy that the children are missing out on because I'm not financially stable is, it's like that's a major problem.
And to further the theme of the, the stabilizing the parent child relationship, it actually extends to the extended family as well. So for example, particip 17, who did get direct cash, told us, quote, I did take my children to see their grandparents in our hometown and I was able to take the money that I did have left over from the mother up, you know, the money they gave us, unquote. And then they described shortly thereafter how they used it for hotel and travel expenses that they needed in order to be able to see the extended family.
So this is an, an additional theme final, final quote here, and we're going to end on this, is that one criticism of guaranteed income is that it may disincentivize work, that it may reduce the incentive to work or the, the reason why recipients, they may just get to rely on that, you know, presidential candidate in the 2000s referred to the need to end the nanny state.
And some people refer to government dependence to a desire. And sometimes even among politicians, there was an argument about an alleged desire to induce government dependence.
Whether that's true or we'll have to await a larger study.
And this little pre pilot produced some preliminary evidence that the problem may be the reverse, that actually that insufficient funds prevent people from entering the workforce.
And so here's the, or at least entering the workforce in a way that they would otherwise be able to do so. So we'll conclude here with this segment on by a quote from Participant 3.
And she and Participant 3 is describing how insufficient child care prevented her from completing the education she needed needed in order to qualify for jobs in her in a particular position. Here's the quote.
The field that I came from, which is accounting, they need me to have my. I don't have my bachelor's degree. I was about a credit and a half away, but that then I needed to stop because I had my kids.
So you can see there that perhaps an investment in child care and funds to allow for child care might have facilitated the entry of Participant 3 into the workforce.
So Renee, we're going to conclude this recording with you and ask because AMAN and the Access to Justice Lab are continuing their relationship, we're going to do what we're calling a phase three pre pilot.
And could you just give us a broad outline because you're the lead researcher on that. Could you just give us a broad outline of what the phase 3 pre pilot consists of and what its goals are are.
[00:49:50] Speaker F: Sure.
[00:49:51] Speaker C: So a couple differences from phase two. The one of the differences is the number of participants who are going to enroll 30 instead of 20. But I think a larger difference is the length of time.
So for both the intervention as well as the research activities.
So the intervention, the compensated research group group in this instance will receive the same $500 a month but for three years as opposed to one year.
And the compensated research group will also get the other services that MON provides. So one thing we didn't discuss yet today is that MON is not just a guaranteed income program and that's one of the things that they work on, but they really are a community service provider of many different things. And so they will also extend to that compensated research group the other services they provide, which include things like advocacy, organizing, empowerment of the mothers to be a part of this movement and then to encourage other mothers to be a part of this movement to kind of reduce, reduce the impact of the child welfare system. And then they also provide legal services. And I think that that is probably one of the more one of the things that we're really focused on in as one of their services in phase three, legal services to resolve some of these other state court issues that might crop up for folks. So not just legal services on their child welfare case, which they certainly can help with, but also housing issues and, and helping with, you know, their benefits benefits issues and things like that.
So the compensated research group will add that additional component of the MONS Services and again, excuse me, the Guaranteed Income
[00:51:57] Speaker A: group will add Direct Cash Transfers group. Direct cash transfers.
[00:51:59] Speaker C: Direct Cash transfers group, Excuse me. So the direct Cash transfers group will add that component of the MON services.
The compensated research group will continue to just be the compensated research.
No MON services will be added to that.
With the theory of change being that it is not just money that mothers need. The money is helpful to alleviate that initial burden. But then there's some follow on issues that they could use help with as well. And MON will work to help them with those.
We also are doing more interviews and more surveys. And so I mentioned we're extending the length of time for the research activities. So we're going to do research, we're going to do surveys, the same enrollment survey and interview that we did in phase two. And then every participant will have an opportunity to take a set of surveys for a longer period of time. And this spans over a period of almost four years at various intervals. And then the first seven individuals in each of the groups, both the direct Cash transfer and the Compensated Research group, will also have the opportunity to take an interview or to, excuse me, to do another interview. And we'll do more of those as well, again at intervals throughout that four year period that also stagger as it relates to the survey administration. So you won't have someone doing both an interview and a survey at the same interval, but for, I think one instance. So those will stagger and that increases the potential compensation for folks in the compensated research group, especially if they are also one of the first seven that will take interviews or that will do interviews.
[00:53:52] Speaker A: Then I think there's one final difference in terms of the eligibility requirements for phase three versus phase two. And phase three, the target population has greater degree of involvement with cfsa, is that correct?
[00:54:06] Speaker C: Yes. So there's actually two other differences, one in the eligibility and one in the data that we'll be hoping to collect. And so the eligibility criteria has changed in the sense that this phase is not looking at mothers who had an investigation, but no subsequent activity with CFSA or the child Welfare Agency. This is looking at mothers who have had a child removed either recently or at present the child is removed from the home, which is a very different posture than the mothers in the prior phase. And then in this evaluation or in this piece of work, we're also hoping to collect information about how mothers are spending that money. So we actually expect to have that data for our phase raised to mothers as well, but it won't be included in this first report.
But, but this is to kind of address some of the narrative about guaranteed income being used in ways that we that, you know, popular discourse does not value.
[00:55:21] Speaker A: So I think we'll leave it there today for this episode of Proof Over Precedent. Just want to give everyone big thanks. So big thanks to everyone and we'll look forward to seeing listeners or to engage listeners in the next episode. Proof Over Precedent is a production of the Access to Justice Lab at Harvard Law School.
Views expressed in student podcasts are not necessarily those of the A J Lab.
Thanks for listening. If we piqued your interest, please subscribe wherever you get your podcasts. Even better, leave us a rating or share an episode with a friend or on social media.
Here's a sneak preview of what we'll bring you next week.
[00:55:59] Speaker C: Week yes, hi, my name is Melody Webb and I came to this work as a family defense and economic justice lawyer.
Just watching Black mothers in D.C. being investigated and primarily because they were poor and not because they were really posing
[00:56:19] Speaker A: any danger to their children.
[00:56:21] Speaker C: The the work was just what I saw was just so compelling that I used it to build a mission around mother's outre.
And you know, again, just research shows
[00:56:34] Speaker A: that black families are disproportionately pulled into child welfare and that the poverty poverty is often confused with neglect and the
[00:56:42] Speaker C: basis for them getting identified and.
