Episode Transcript
[00:00:00] Speaker A: Imagine a justice system built on rigorous evidence, not gut instincts or educated guesses about what works and what doesn't.
More people could access the civil justice they deserve.
The criminal justice system could be smaller, more effective and more humane.
The Access to Justice Lab here at Harvard Law School is producing that needed evidence. And this podcast is about the challenge of transforming law into an evidence based field.
I'm your host, Jim Greiner, and this is Proof Over Precedent.
So we're back with Shannon Sewards. Welcome back to Proof Over Precedent. Shannon, thank you so much for being here. Why don't you just remind everyone one more time who you are?
[00:00:45] Speaker B: Yeah, so I have been in the IRB world for quite some time. I'm currently the director of the Human Research Protection Program at a large New England medical center, previous to that at bay, various universities across the nation. But I've been doing IRB work for over 20 years now.
[00:01:05] Speaker A: Terrific. And for folks for whom this happens to be the first Proof Over Precedent podcast they've encountered, this is the third in a series of conversations that Shannon has been kind enough to have with me about the ethics of research, especially research that is randomized and that is non medical and that is in the field. So, so we're talking about non Medical Field, RCTs in particular.
And the reason why this is a topic that we think should belongs in Proof over precedent is because there is a whole ethical structure that is encapsulated within a set of regulations called the Common rule.
And those are codified at 45 CFR Part 4.
And our previous conversations, Shannon, have gone over some of the overall provisions of that of the Common rule, which, which governs all research involving what's called human subjects.
And we're going to continue that conversation again because some IRBs and some lawyers don't know about each other, some legal researchers don't know about each other. So IRBs don't know about legal research and legal researchers don't know about IRBs. And so we're attempting to bridge that gap.
So thank you so much again for having the conversation.
[00:02:32] Speaker B: Pleasure.
[00:02:33] Speaker A: We're going to talk in particular about today about this, this term, human subjects.
I wish it weren't called human subjects. I wish, and this is something you taught me, I wish it were called participants, human participants, because subjects sounds a little ghoulish, but we'll just use the term human subjects, which is what the regulations use, the common rule uses.
And the reason that this is so important is because if the, if people in a study, in some way or involved or affected by a study, or touched upon by a study or related to a study in some way are human subjects, then certain consent requirements kick in. The common rule imposes certain consent requirements or, and you must get their consent for the research to go forward, or you must obtain a waiver of consent from the irb. And the common rule regulations govern when an IRB can provide a waiver.
And so a lot turns on who or what human subjects are.
And so, Shannon, I'm going to hope to receive the benefit of your wisdom on this. First of all, for the lawyers in the room, let's start out.
The human subjects definition, I have it in front of me, is codified at 45 CFR 46.102 E, no less. Actually, E1, 102 E1, no less.
So there's a deep, deep in the weeds.
And let's just start with the definition. And then I'm going to pose a set of hypothetical studies, some of which are not actually hypothetical studies, real studies, and we can see how this provision plays out. Does that sound like something we could do?
[00:04:27] Speaker B: I think that sounds like definitely something we could do.
[00:04:30] Speaker A: Terrific. Okay, let's do that then. So this definition says a human sub human subject means a living individual. So right there we have something that not everybody knows. When you're doing research on people who have passed away, do you need IRB clearance?
No, no. Right away. Because you're not doing human subjects research. Right?
[00:04:51] Speaker B: Exactly, exactly. And that's looking at it from a strict, you know, regulatory, federal regulatory perspective. There may be institutional, state or other local policies that may come to play as well, but the federal regs are. Are the floor.
[00:05:07] Speaker A: Right.
[00:05:07] Speaker B: And that's where you start.
[00:05:09] Speaker A: So, yeah, I actually should have been more accurate. Do you need common does. Do you trigger the common rule? And you don't trigger the common rule. Right.
[00:05:15] Speaker B: You do not.
[00:05:16] Speaker A: Right.
Okay, so a living individual, that's a human subject. Human subject is a living individual about whom an investigator conducting research. And then we have one and two.
Okay, now one to me demonstrates something we've talked about before, Shannon, that which is it presumes that all research is involving human subjects is medical because the very first thing you see is obtains information or biospecimens as though you're going to be doing, you know, biospecimens as often as information, which, you know, maybe we are. Who knows? But anyway, so for our purposes, since we don't do a lot of biospecimens in the law or in social Science research, generally we're going to say obtains information through intervention or interaction with the individual and uses, studies or analyzes the information. So you have to obtain it through an intervention or an interaction with the individual and then you have to use, study or analyze it.
That's part one.
[00:06:20] Speaker B: Yep.
[00:06:21] Speaker A: Or part two. Okay, so this is an or obtains, uses, studies, analyze or generates identifiable private information.
Okay, so there we have the first part of the definition and then we have a series of other parts which I'm not going to read out because it just get too boring. But we probably are going to have to touch on because the regulations go on to define what intervention means, what interaction means, what private innovation information is, what identifiable private information is. So we have definitions for all of those. So we may come, we may touch on those. Okay.
[00:06:55] Speaker B: Okay, sounds good.
[00:06:57] Speaker A: All right, so let's go over a couple of non hypotheticals, put it that way, we're going to keep it hypothetical. But these are real studies or these are so close to real studies that they might have, they might as well be real studies. So first, first non hypothetical, there are a series of investigations into whether a providing legal assistance of some kind could be a lawyer, could be self help materials, a lawyer, meaning a traditional attorney client relationship, could be, could be a lawyer for the day program, could be, you know, a legally adept chatbot, who knows. Right. Something that is in some sense access to justice promoting okay.
To a pro se individual, meaning an individual without a, without a law otherwise, without a lawyer, without legal assistance in a piece of lit. In a piece of traditional court litigation, which means it's adversarial, there's an adversary on the other side.
So suppose we were, we wanted to randomize the provision of such assistance versus, I don't know, no such assistance. We just wanted to have, when it was straight up to, so that we could measure the effectiveness, the effectiveness in the sense of the outcome of the case, the adjudicatory output of the court.
We wanted to measure that and we wanted to randomize that.
And so we know that we're going to need to get, we know we're going to need to get consent. In other words, a participant, a human subject is the person who would receive the particip, the assistance.
The person receiving the assistance is unquestionably a human subject because we are intervening.
Other words, we are going to obtain information about the individual in the form of the case output through an intervention by randomizing the type of assistance that they get. So that's easy. That's easy. When we know that that's going to trigger the consent or the waiver of consent regulations, we're either going to need to get consent or we're going to need to get. Or we get a waiver. Ordinarily, in the Access to Justice Lab studies, we get the consent.
Right? The question, though is, is the adversary in the litigation a participant? So, for example, if this is landlord tenant in a summary eviction context, and we're randomizing the level of legal assistance that we give to tenants who are typically unrepresented, is the landlord a human subject within the meaning of this definition, do you think? Obviously you're not speaking officially. I'm just trying to figure out how to think about these regulations.
[00:09:36] Speaker B: Well, I would ask, you know, are you intervening or interacting with that individual, or are you obtaining private, identifiable information from that individual?
[00:09:48] Speaker A: So the easy part is we're not going to obtain any kind of private information because all of the. And this is something that a lot of IRBs don't know, at least in most jurisdictions, not all the adjudicatory output of courts, as those are public records. And so anyone can go and watch court proceedings that they have nothing to do with, Right? And anyone can go request records on court proceedings that they have no relationship to. They can just ask about them. So there's no private information. We're just going to try to get the outcome of the court case, right? That's just, that's going to be our outcome variable.
The question is, and, and this is. This is it. Sometimes I remember getting, having these conversations with you and also having conversations with current IRBs. The question is, are we intervening somehow with the, with the landlords in this hypothetical, for example, and this is where we as scientists will say, well, wait a minute, irb, we'll tell you what we do.
You have to tell us whether we're intervening. And then you sometimes remember having conversations with you saying, well, sometimes you have to. It's a little more to it than that. So tell me what's going on. Tell me what you need to know to make a judgment here.
[00:10:59] Speaker B: So in regard to intervening or interacting, it's directly with that individual, okay? And that's how I interpret this. And I think that most IRBs do. Every IRB has a scope of authority, and that's why the regulations were written in such a way, so that you know who is the subject, what is the research, what is not the research, what is not the subject, so that you have a boundary to provide your review.
And what you're describing is where that boundary is getting a little bit blurred. Because without that clear boundary, the entirety of the world might be subject to oversight.
So it's important to distinguish what are the boundaries of any research study.
And I think that it's a slippery slope when those are blurred.
So in this instance, my question would still be based in the definition. Are you directly intervening or interacting with this individual? Are you obtaining private identifiable information from this individual? If the answer is no, then they're not a human subject.
[00:12:10] Speaker A: Right. So let's see. The regulations do define the term intervention.
Let's just take a quick look. E2. So 102. E2 defines an intervention to include a physical procedure which we're not using here. Right.
Or it also includes a manipulation of the subject which I don't think we're doing because we're not manipulating the landlord here or the subject's environment that are performed for research purposes. And so this again, this again gets, goes back to your idea. This is a little slippery in this application.
[00:12:45] Speaker B: Yeah.
[00:12:45] Speaker A: Right.
So it does seem a little odd to a lawyer, put it this way, to a lawyer to trained in adversarial processes to give a landlord a veto over whether a tenant, tenant in eviction litigation would receive legal assistance or what level of legal assistance that tenant would provide. That does seem a little odd from a lawyer's perspective.
[00:13:12] Speaker B: Agreed. And you know, I think the thing is, is that maybe the IRB's assertions that somehow the tenant is involved is predicated on these self help materials work.
[00:13:24] Speaker A: Right? Well, you mean the landlord is involved. Right. Because we know we're going to get in.
[00:13:28] Speaker B: We know we're going to get.
[00:13:29] Speaker A: Yeah, exactly. Yeah. So you're saying that maybe the self help materials actually do work, which of course we don't know because that's why we're doing the study.
[00:13:37] Speaker B: Exactly. And so, you know, let's say the self help materials just don't work at all.
Would the, would an IRB think that there's somewhat of an impact on the landlord?
[00:13:49] Speaker A: Right.
[00:13:49] Speaker B: Or is it only if the self help materials might work?
There shouldn't be a difference.
[00:13:57] Speaker A: Right. In other words, if what you need to know is the outcome of the study in order to figure out whether there is an intervention, then probably there isn't an intervention within the meaning of the common rule. Is that a fair statement?
[00:14:10] Speaker B: That is a fair statement.
[00:14:11] Speaker A: Okay, that's, that's actually a really good helpful piece of guidance by the way. That's something we can really operationalize here. Because that's actually, I Think going to answer an awful lot of these questions. So let's, let's see, let's see how much work that can do. Let's see how much work that can do. So we have there, by the, by the way, just as an aside, and you know this literature better than I do, but I believe there is, there are some published articles that have been expressed a concern about mission creep among IRBs that yeah, the mission is expand, has expanded too much. Is that correct?
[00:14:43] Speaker B: That's correct. That's correct.
You know, I mean, if, if you approach the federal regulations, you know, there is a lot of, I think, credibility to thinking about the larger good of research.
But to look at it through a very black and white, excuse me, black and white lens insofar as, you know, what are the regulations actually say, what is actually our scope of review, I think it makes some of these more challenging questions more clear if you just think about it very simply.
[00:15:15] Speaker A: Yep, yep. By the way, what you have just done is summarized at least one and, or maybe the first year or the first year and a half of law school, by the way, which is, you know, what does the law actually say and what, and in doing so you have, you have, in my view, properly identified IRBs as a sort of quasi court system and raise the question does, should the court apply the law as written or should it use its sense of justice to inform what the law is? Or something like that, which are debates. We have an awful lot in law school.
[00:15:50] Speaker B: Yeah. And that I think is a good description of what's going on.
[00:15:55] Speaker A: Right.
[00:15:56] Speaker B: Yeah.
[00:15:56] Speaker A: Although I find that many folks who work with IRBs with whom I have this conversation resist the characterization that they are sort of courts or quasi courts. They actually don't, don't like that at all. For whatever reason.
[00:16:10] Speaker B: I wouldn't bring that up.
[00:16:14] Speaker A: I'm sorry. They find it so insulting. I'll have to remember that actually you're lawyers. How about that? And then they don't, they won't have lunch with me. But anyway, okay, so let's look at another non hypothetical study. This is a study that the Access to Justice Lab or a series of studies the Access to Justice Lab has been pursuing for, for a long time.
So the, the idea here is that we have something called a risk assessment instrument, which is a quantitative score that purports to measure the likelihood in some sense or to classify cases in some way by likelihood that an undesired outcome will occur unless a decision maker intervenes.
And so the idea here in the law might be for example, that the, it's a judge, the decision maker is a jud, deciding whether to impose bail upon a recently arrested individual. And they're going to receive a risk assessment instrument that says, well, if you let this person get released, what's called or own recognizance release, meaning they don't have to, they don't have to post any bail.
Here is a risk classification for whether that person will do something undesired. The undesired might be get rearrested for a potential new crime. It might be fail to appear at a court proceeding. Right.
Or another instance might be a judge deciding whether to impose a sentence of incarceration.
And the idea is that the risk assessment instrument says, well, if you don't impose a sentence of incarceration, this person is likely to commit another crime or something like that. Right. Whatever it is. And what the Access to Justice Lab is doing at the bail stage is randomizing whether the judge receives the risk assessment instrument. So in some cases they do, in some cases they don't. Again, we have an easy question which is, is the judge a human subject? And that's easy because we are clearly manipulating the environment of the judge. Right. The decisional environment of the judge. Now we have a separate question about whether we could still go forward because it's an official purpose if the judge refuses. Right. That's that we may cover in a different session, but for now we're just going through the basics here. Is the judge a human subject?
And so the easy question is the judge is clearly a human subject. We manipulate the judge's environment. The harder question is, is the arrested individual a human subject within, within the, the scope of the regulations? And so why don't we walk through that one? Could you, could you give us the benefit of your wisdom on that one?
[00:18:44] Speaker B: Well, thank you. And, and I do have to say this is, this is a very challenging situation. Right. Because it, it does, it's, you find it connects that more logical decision making with more of a this doesn't feel right feeling.
And that's where I think IRBs tend to get a little bit misguided, is that you're bringing up sort of the human feeling aspect of things.
[00:19:14] Speaker A: And, and so if you, if, and so your, your instinct then is to stick to the regulations, is that correct? All right, so, and by the way, spoken like a true advocate of this, of the Antonin Scalia interpret, preferred interpretation of, of law, by the way, that let's just apply the law as written.
And so again, my lawyer friends will Recognize this and maybe giggle a little bit and you're shuddering.
[00:19:42] Speaker B: Oh, no, I know I am. I am.
[00:19:45] Speaker A: Actually, Jim, I meant it as a compliment. But anyway, let's go ahead.
So, anyway, so let's walk. Let's look at what the regulations say. So the regulations say again, it says there's a definition of intervention.
Right. Which focuses the key.
The most slippery phrase of the broadest phrase is manipulations of the subject's environment that are performed for research purposes. There's also interaction that might be a little less obviously applicable because it says includes communication or interpersonal contact between the investigator and the subject. There is no communication or interpersonal contact unless you extend that phrase to be quite indirect.
Encompass things that are extremely remote in time and space.
Right. So the question is, are we manipulating the subject's environment?
And that means are we. By giving the judge a risk assessment instrument, which, by the way, the arrestee may not even know that the judge is looking at it. They have no idea it's even there.
Are we. Are we manipulating the subject's environment for research purposes? And tell me how you would. How you would think about that.
[00:20:57] Speaker B: I would want to know more about the process and what would be considered the standard process and how this process might differ from the standard process.
You know, just in regard to the landlord situation. Right. Do we feel differently about it if it has a more favorable outcome versus an unfavorable outcome?
And if so, why shouldn't that be the same?
[00:21:24] Speaker A: Right.
[00:21:26] Speaker B: So, you know, I think that thinking about this from that perspective, we have to go back to the definition of a human subject and, you know, intervention, interacting, private, identifiable information.
Does it meet that threshold?
[00:21:41] Speaker A: Right.
[00:21:43] Speaker B: You know, would we. And getting back to favorable or unfavorable, would we mind if we found that, you know, creating this score creates a more equal playing field and reduces a judge's bias.
[00:21:57] Speaker A: Great.
[00:21:58] Speaker B: That sounds wonderful.
[00:21:59] Speaker A: Which is the intention of those who created it, by the way. That's what they're hoping to do, right?
[00:22:03] Speaker B: Yep.
[00:22:04] Speaker A: We just don't know Exactly.
[00:22:06] Speaker B: And I think too, I mean, they're the regulations.
They're somewhat simple in a way, but they.
They encapsulate a lot of different thoughts. And one also is not only who is a human subject, but what's the risk benefit ratio here? Are you, you know, putting people at greater risk than what would normally encounter, what is that risk, identifying that risk, et cetera. So those are all decision points that come into this as well, insofar as, you know, whether that individual, whether their outcome has been Manipulated or not.
That's a tough one, Jim.
[00:22:53] Speaker A: So let's explore this a little bit further. Right. Because we learned from tough, from tough cases. Right. Not risky ones. Right, we do. Let's explore this a little bit further. The way a risk assessment instrument works, at least we think scientifically, if it works at all.
The way it works is not by changing the behavior of the arrested individual, it works by changes the decision of the judge.
And so what that means is that the only way it can have any effect is if the judge's decisions and therefore the environment that the arrested individual experiences improve in some sense for.
Well, the judge's decisions improve. Right. They get better. But what that means is they change.
And so some individuals who would not have been incarcerated, say, or had bail imposed, more accurately had bail imposed, which does lead to some incarceration, not directly because perhaps they can post the bail, but does lead to some incarceration.
Right. Some individuals who, absent the risk assessment instrument, would have achieved a war release will now have bail imposed and thus end up incarcerated. Other individuals who would have had bail imposed and thus ended up incarcerated will now be given or release and thus walk free.
We don't know who those are, by the way. We don't know whether this will happen because it may be that the risk assessment instrument just has no effect at all. The judges don't pay any attention to it. There's no legal or require requirement that they do. In other words, we're in equipoise about all this. But if the risk assessment instrument works, if it works, it is not as though we're intervening by providing information to the arrested individual in some way and changing their behavior. We're instead changing a decision that they then experience.
And again, the likely that is going to mean if it works, some people who would have walked free are now going to be incarcerated, others are going to have the opposite. They're going to walk free when they would otherwise have been incarcerated and therefore we don't know. But it is redistributing that highly consequential decision, even though it doesn't. And it doesn't affect the behavior directly, it affects the environment that the arrested individual responds to. Does that change at all whether you think the arrested individuals are human subjects?
[00:25:29] Speaker B: You know, I, I think that it gives you more information to work from.
It doesn't necessarily change the definition of a human subject.
[00:25:39] Speaker A: Sure, sure. But how does it change how we might apply to, how we might apply that definition to the, to the arrested individual?
[00:25:46] Speaker B: You know, I, I, so I'M I'm, I'm contemplating also there's, you know, there's been various guidance.
[00:25:54] Speaker A: Right.
[00:25:55] Speaker B: About this and, and, and I think I, I'd mentioned some time ago there used to be this very ancient thing called this IRB guidebook that OHRP put out years ago. And it's actually, you know, there's a, there's a quote in it. So I was like, well, before you.
[00:26:12] Speaker A: Give us the quote, remind us what OHRP is.
[00:26:15] Speaker B: Office of Human Research Protection.
[00:26:17] Speaker A: There we go. Super. And that, that lives within with it. With where or at least it used to before Doge.
[00:26:24] Speaker B: It's kind of still there.
[00:26:25] Speaker A: We just don't know if anybody, if it has any, it has any staff. But anyway.
[00:26:28] Speaker B: Yeah, we don't know.
[00:26:29] Speaker A: Yeah, we don't actually know whether you. Wait, where does it, where does it live exactly?
[00:26:34] Speaker B: In Health and Human Services.
[00:26:35] Speaker A: In hhs. Right, exactly. Go ahead. Right, right.
[00:26:38] Speaker B: So there's this, you know, concept about the IRB really. This is again, boundaries. Right. What is the IRB's turf?
[00:26:46] Speaker A: Right.
[00:26:48] Speaker B: In regard to. The IRB needs to focus in on what are the risks to the individual, to the human participant imposed by the research.
So we still have to get back to who is the human subject.
[00:27:04] Speaker A: Exactly. Yep.
Which is what we're struggling with here.
[00:27:09] Speaker B: It is. And you know, the thing is as well is that dovetails with, you know, you shouldn't think about the long term outcomes of the research and what that might have on other individuals.
[00:27:23] Speaker A: Right, right.
[00:27:25] Speaker B: So, you know, one could say that, you know, use of this could have a long term outcome on other incarcerated individuals, but that shouldn't be thought of in the context of this particular research study.
One thing, there could be a medium, sort of a middle path that is achieved by just letting people know that are the part of that court system. This is what we're doing here in this court.
And you know, it's interesting because courts do that, institutions do that, they do a lot of activities such as this, but when it's labeled research, you have much more scrutiny to how it plays out, who's a subject, what rights they have and if it could even be done right.
[00:28:23] Speaker A: And for the lawyers approaching this, this is one of the rather hair pulling out. This is a sort of hair pulling out of your head moment. Pulling hair out of your head moment, I should say.
Because what they might point out is that there appears to be no problem with the federal regulations if the, if a municipality or county, say, just simply adopts the risk assessment instrument for everybody or decides not to adopt the risk assessment instrument for everybody.
Either way there would be, we would still be an equipoise about the risk assessment instrument works because we can't learn anything from when that or at least we can't learn as much. Put it that way. We can't learn anywhere near as much from a wholesale adoption versus a wholesale non adoption as if we run a randomized experiment. If we ran the randomized experiment, we'd actually would know. Right now we're in equipoise. And yet there is greater scrutiny on the effort to try to find out whether these things do all the great things that their proponents say or impose all of the harms that their detractors say. There's greater re. There's greater scrutiny on that action as opposed to proceeding blindly without knowledge.
Is that accurate?
[00:29:32] Speaker B: That's very accurate. And I have to say that, I mean this is I think just a challenge in this area to conduct research because there hasn't been research with a capital R.
There hasn't been much done in the legal arena because it's just been done by, hey, let's try this out.
[00:29:58] Speaker A: And this goes, that goes actually to. There are two points I wanted to bring up about something you said earlier. Here's one of them right here. You said you would, at one point you said I would want to know more about what the standard practice is. Right. That might help you decide whether the arrested individuals here are human subjects within the regulations. And here's where this is a conversation that we frequently have with Harvard's irb.
Our problem is we don't, we. There is no standard practice.
In other words, some municipalities, some counties are using risk assessment instruments without knowing whether they, whether they, whether they provide the benefits or impose harms.
Some are choosing not to again blind to the consequences.
At least there is no credible evidence one way or the other. There's supposition, there's theory, there's before and after studies, but there's no RCTs.
[00:30:55] Speaker B: Yep.
[00:30:55] Speaker A: Right.
There's no randomized research.
Some counties are in the process of changing one way or the other.
Right. Some are using risk assessment instruments and contemplating not using them anymore because they're expensive to generate and they're not sure they're worth it. Some are saying we want to rationalize our pre trial process and this is a step that you can take to rationalize it, to make it quote, unquote more scientific, which always, I always get a little chuckle about that because of course we don't have any clue whether these things Work.
[00:31:29] Speaker B: Exactly.
[00:31:29] Speaker A: It gives every mirror of science, but we have no idea whether, whether, whether they in fact do any good.
And so when, sometimes when IRBs ask us for a standard practice in the law, we have to go back to them and say there isn't one. And they actually, I'm not sure they believe us.
I, I really am not sure. They probably.
[00:31:49] Speaker B: Right, right, so you're probably right.
[00:31:51] Speaker A: If I were to tell you, well, Shannon, would you believe me if I were to tell you that with respect to say, risk assessment instruments, there isn't.
[00:31:57] Speaker B: A standard practice, you know, I, I, well, I believe you.
[00:32:01] Speaker A: Right.
[00:32:02] Speaker B: Because for quite a while.
[00:32:05] Speaker A: Right.
[00:32:05] Speaker B: And I've, I've come to learn this as, as truth.
[00:32:09] Speaker A: Right, right.
[00:32:11] Speaker B: Which is, I think, shocking, you know, and you know, it's, it's interesting in that, you know, being an IRB professional, being in this world and seeing it through this IRB lens, it doesn't really match reality at times.
And you know, as we've discussed and as you know, this entire set of regulations is based on biomedical research principles.
[00:32:37] Speaker A: Right.
[00:32:38] Speaker B: Where you used, you know, evidence based practices, you, you do tightly control trials, you control your variables, you know, you.
[00:32:48] Speaker A: Have an identifiable standard of care.
[00:32:51] Speaker B: Exactly, exactly. But it has been challenging because even in that arena, standard of care can vary from institution to institution.
[00:33:01] Speaker A: Sure.
[00:33:02] Speaker B: So it's, it's not that great of an, of an excuse as well.
But you know, I would say in this instance what's, what's challenging and what you've been trying to do is, is standardize is to really look at this from a scientific viewpoint rather than just doing, you know, observational research where, you know, things aren't controlled. There could be some explanation coming out of left field that provides, you know, the basis for why this works or why it doesn't work, et cetera.
And I think that a liar, a lot of IRBs have not encountered that before and they, you know, IRBs sometimes work very well in the black and.
[00:33:46] Speaker A: White, whereas we go straight for the line in between them and try to sit on it for a little while.
[00:33:53] Speaker B: Exactly.
[00:33:54] Speaker A: One other point that you brought up earlier was the idea that maybe the research that it becomes hard to say when, when the, what the stopping point is for classifying human subjects, which as you said, well, maybe there might be effects of having a risk assessment instrument on certain individual, with certain individuals, bail decisions on other incarcerated individuals. And then, you know, are we going to say people who weren't even, didn't even have their hearings or didn't even go through the bail process because maybe they were already incarcerated or maybe they were there before the study started or something like that, or maybe they were sitting in that jail because another jurisdiction got overcrowded.
And so they're actually transferred to that jail for a little while just because they could be housed there until the other jurisdiction solved its overcrowding problem. There might be an effect of this risk assessment instrument on them, but it becomes very difficult to say that they're human subjects. Furthermore, you might say, well, if this risk assessment instrument works, then there might be an effect on people who are not before the court at all.
Right. It could, for instance, benefit or if it backfires, harm them because they might be victims of the, of crimes that either will be committed because of the risk assessment instrument or won't be committed because of the risk assessment instrument.
You know, getting. If it, if it doesn't work, the risk assessment doesn't work and it botches up judges, releases, release decisions, then there will be potentially more crime and more victims. And are they, are they human subjects? And so I think at that point, you sort of see the ripples.
Right. And, and what is your thought? What are your thoughts on all this? Right.
[00:35:41] Speaker B: My thought is we're going to a place that's very difficult to.
[00:35:46] Speaker A: Right.
In other words, you have to stop. Right?
[00:35:49] Speaker B: You do, you do. And I, you know, I think that you could easily just continue with that thought and everyone would be a human subject.
[00:36:00] Speaker A: You're a human subject in this, and you just didn't know it. You're a human subject. And in this, in this, in this.
[00:36:05] Speaker B: Risk assessment race, where's my consent process?
Where's this, Jim?
[00:36:09] Speaker A: Right, exactly.
[00:36:12] Speaker B: Exactly. But I think that, you know, it is challenging in the work that you and the HOJ lab are doing, in that you're really trying to, you know, create evidence for these practices.
And it's very interesting. I think this is really challenging for a lot of researchers. Just within, just social, behavioral in general, is that these are human behaviors.
Right. And we're, we're all humans. We see each other, we see how people react.
You know, you know that if you're going to put a $10 bill on the street, it's. Someone's probably going to pick it up more than if it's a penny. Right.
But there's no scientific evidence for that.
[00:36:58] Speaker A: Right.
[00:37:00] Speaker B: So it's, it's, I think it's, it's going against what people's beliefs are, and it really almost questions people's, like, code of how they live in some ways. And what their thought patterns are and put that in an IR in a room with 10 IRB members and it's, it's really challenging.
[00:37:28] Speaker A: Let's go over one more. We're going to conclude with this one. This comes from one more hypothetical, non hypothetical or sort of quasi hypothetical situation, put it that way.
[00:37:37] Speaker B: What a softball one?
[00:37:38] Speaker A: Yeah, well, no, but sorry we didn't bring any of those today.
This comes from a friend of mine named Christopher Robertson and he wrote an article called the Ethics of Research that May Disadvantage Others. And by the way, for those who would like to read it, we'll of course put the link in the Access to Justice Lab's Roof Over Precedent website. But it's 43 ethics and human research two published in 2021. And one of the hypotheticals Chris posits is that a law school is, let's just make it simple. A law school is considering a pre, and a pre start of law school, so 0l type training program and they're going to offer it, they think only to first generation students. So students who that have the first in their immediate families to go to college and they've also decided to go to law school. And the idea is that the law school is going to randomize which of the first generation students get this pre law school program that sort of gives them a bunch of basics.
It tells them things like what do you do if a court system has no majority, appellate court system has no majority? How do you figure out what the, what the law of the case, the law coming out of that case is? Or what is an appellant? What is a, what is a respondent? Right. You know, just these basic definitional terms that eventually will be covered in law school. It's just the idea that folks who are first generation might face challenges or headwinds that, that others might not.
And so the, the idea is that the, the law school wants to randomize whether the, whether the first generation folks, which first generation goes, folks get this training program which don't because for all they know all it does is occupy their time that they could be using say to earn money, to work and earn money.
And, and, and it may have no effect, may have no benefit. Right. So it wants to randomize it.
And, and, and the, and, and IRB looks at that study and says well I know that the, the folks who are randomized are going to be, are, are human subjects. If the idea is if this otherwise qualifies as research, I know they're human subjects.
Now whether they, that will trigger the consent analysis Whether they have to get consent is another issue. It may be they'll fall within a waiver.
In my view, it might well fall within a waiver, but they're particip, they're human subjects under the definition. The question is, are the non first generation students human subjects within the, within the definition of the, of the, of the regulations under the theory that grades are curved? And if one of the outcomes of this study is that the, to see whether the training program, free law school training program, improves grades, the only way for it to improve grades under this theory is quote unquote, at the expense of the non first gen students because there's only a certain amount of top grades to go around.
And so Chris in this article calls these rival risk goods where there is a shortage of goods and there is in some sense some sort of rivalry which may be in, you know, a provocative term. I think Chris may mean it to me. Provocative term, but it's a rivalrous good. So what do you think about that situation? Are the, are the folks who are not first generation in this hypothetical, are they human subjects?
[00:41:14] Speaker B: I would say no.
[00:41:16] Speaker A: Tell me about that.
[00:41:17] Speaker B: You know, just going back to the definition of a human subject and we could even apply that to those that are incarcerated in the judge condition as well.
[00:41:26] Speaker A: Right? Yeah, right.
[00:41:28] Speaker B: And you know, I think in this, this situation it's, it's the human factor that, that confounds a lot of these outcomes.
[00:41:40] Speaker A: Right.
[00:41:41] Speaker B: And how people feel about it, not necessarily what the CFR says.
[00:41:49] Speaker A: Got it. And going back to your. Yeah, sorry, I'm going back to your earlier statement. If you have to know the outcome of the study in order to decide whether somebody's a human subject, then you may be going down the wrong path.
[00:41:58] Speaker B: Right, exactly.
[00:41:59] Speaker A: Yeah. So tell me how that would work here.
[00:42:02] Speaker B: Well, I mean, the thing is, is that.
So let's say this is something that works, right? You know, at the outset that this is going to improve grades and that other, you know, group is going to have to compete harder to get that A. Okay, well, if you already know that, why are you even doing the research?
[00:42:22] Speaker A: We wouldn't do the study. Right. We don't know that. Right, of course, that's the issue.
[00:42:25] Speaker B: Exactly, exactly.
So is it that if this is favorable then we have to be concerned, but if it's not favorable, then it's fine.
[00:42:36] Speaker A: It may be inflicting a harm. Right. It may be preventing them from working and earning money and instead they have to go through this training process program that does them no good whatsoever.
[00:42:45] Speaker B: So I think, you know, there's if. If one stays true to the regulations, takes the human factor out of it. I think you have a much clearer perspective of, you know, who is the individual, who are you trying to understand to protect and what is the purpose of the research? What benefit, maybe not to the individual, but overall would this research contribute to, which is really what the heart of the research regulations are.
[00:43:24] Speaker A: Super, super. So in other words, we might be engaging in a manipulation of the subject's environment, which is what eat what 102e2 says.
And therefore we don't have an intervention and we don't have an interaction because we don't have a communication or an interpersonal contact between the investigator and the non first generation students. Is that fair?
[00:43:49] Speaker B: That is fair.
[00:43:50] Speaker A: Right?
[00:43:51] Speaker B: And this is why IRB work is so fun, Jim, because you get to think about these things.
[00:43:59] Speaker A: And why lawyers should find it so. Because basically it's all law, right? It's just. It's all reading complex regulations and applying it to situations, which is what we do for a living.
[00:44:09] Speaker B: There you go.
[00:44:10] Speaker A: So once again, Shannon, I'm going to close by accusing you of being a lawyer. Ha ha.
And by saying we should stay true. Right? Which is what you said, right? We should stay true.
[00:44:21] Speaker B: Stay true.
[00:44:22] Speaker A: Terrific, Shannon, thank you so much as always. It's been spectacular and I hope you'll join us again on Proof Over Precedent sometimes.
[00:44:28] Speaker B: It would be my pleasure, Jim. Always a pleasure.
[00:44:30] Speaker A: Proof Over Precedent is a production of the Access to Justice Lab at Harvard Law School. Goal views expressed in student podcasts are not necessarily those of the A J Lab.
Thanks for listening. If we piqued your interest, please subscribe wherever you get your podcasts. Even better, leave us a rating or share an episode with a friend or on social media.
Here's a sneak preview of what we'll bring you next week.
[00:44:56] Speaker B: I read about pro se litigants being sanctioned for failing to comply with rules.
[00:45:03] Speaker A: Of civil procedure, both at the state and federal level.
[00:45:07] Speaker B: And my initial assumption was that they.
[00:45:11] Speaker A: Failed to comply, perhaps because the rules.
[00:45:14] Speaker B: Are convoluted and not accessible to laymen, so that the issue was a matter of interpretation.
And I went in with that assumption.
[00:45:24] Speaker A: Looking for research that maybe would confirm.
[00:45:27] Speaker B: That, but I actually found the opposite.
[00:45:30] Speaker A: Or some suggestions that the opposite could be true.
