Measuring Offender Recidivism-The Urban Institute<\/strong><\/p>\n DC Public Safety Radio<\/b><\/p>\n http:\/\/media.csosa.gov<\/b><\/a><\/span><\/p>\n Radio show at\u00a0http:\/\/media.csosa.gov\/podcast\/audio\/2014\/10\/measuring-offender-recidivism-urban-institute\/<\/a><\/p>\n LEONARD SIPES:<\/b> From the nation\u2019s capital this is DC Public Safety. I am your host Leonard Sipes. Ladies and gentlemen our show today is Measuring Recidivism. We have folks from the Urban Institute Ryan King. He is a senior fellow with the Justice Policy Center again at Urban Institute. Brian Elderbroom a Senior Research Associate with the Justice Policy Center at the Urban Institute. This is all in preparation for a Wednesday, October 15th<\/sup> webinar at 2\u2019o clock in the afternoon with the Urban Institute and the Bureau of Justice Assistance we encourage everybody to go to the website www.urban.org<\/span><\/a> for the document I am about to mention and the webinar and I will bring up the webinar a couple more times throughout the course of the program. Ryan and Brian welcome to DC Public Safety.<\/p>\n RYAN KING: <\/b>Thanks for having us.<\/p>\n BRIAN ELDERBROOM:<\/b> Thanks for having us.<\/p>\n LEONARD SIPES:<\/b> Like I said that before we hit the record button it\u2019s the Ryan and Brian show. It sounds like a morning drive time radio show, so you know it should be an interesting program. You did a report called Improving Recidivism as a performance measure and I just want to read from it ever so briefly at the very beginning to kick of the conversation. Performance measurement establishing metrics for its excess and assessing results is crucial first top in informed decisions made by all areas of government including criminal justice policy, understanding the outcome, the funding and policy decisions is critical to improving government performance and providing the best return on tax payers investments. If that\u2019s true Ryan or Brian why don\u2019t we do it? Why are we in the criminal justice system, why are we so reluctant to being involved in measuring recidivism.<\/p>\n RYAN KING: <\/b>Well I would say it\u2019s not a reluctance to be involved but it\u2019s a little bit more about the way that we are doing it. The reason, the impetus behind doing this brief was a recognition that for better or for worse recidivism is a primary performance measure that is being used in the fields of corrections and there are a lot of reasons why that is troubling practitioners and policy makers and researchers a like I think have a range of criticism but recidivism as sort of an end measure basically a measure of, when somebody comes in the system is that intervention while they are in the system, having some sort of positive outcome and result that is leading to a reduction in future offending and so if we have and imperfect measure. I think Brian and I and other folks at the Urban Institute felt it was really important for us to try to sketch out ways that we could improve it because it is not going away and I think one of the key things is sort of flagged right from the get go is that different folks have different criticisms about it and I think the biggest issue is that every individual who comes into the system has a different story, they have a different history and they are touched by a number of different parts of the system and so recidivism traditionally is looked at as a measure of success for prisons. People coming into prison do they succeed when they come back out again?<\/p>\n LEONARD SIPES:<\/b> Well I am going to go over to Brian on this question. When we say and I talked to a couple of practitioners from around the field saying I was going to do at this radio show today with Urban and they are saying, you know Leonard we have individuals that come to us in the Criminal Justice System that come to us from Community Supervision. They have histories of substance abuse, histories of mental health problems and they have a lousy job record. They spend a lot of time, in many cases in the American Prison System, women offenders in particular come out and they have to deal with kids, really the odds are so stacked against them and it is not as if we have the money and the wherewithal and the resources to remediate so many of the social problems that they bring to us, why should we be held responsible for that persons success when we don\u2019t control all the variables that go into that persons success or failure. Answer that question for me Brian.<\/p>\n BRIAN ELDERBROOM: <\/b>I think for those very dedicated professionals who are working with people involved in the Criminal Justice System, they care deeply about those outcomes that you are describing, not just whether they desist from future criminal behavior but whether they find stable housing, whether they find a job or that they stop using drugs and alcohol and those are important measures by which they should judge their success as well. At the end of the day what we outline in the brief is that a single state wide rate of recidivism obscures a lot of what you are describing in that comment and we need to incorporate other success measure in addition to the ones that you described. How long is someone successful, how many people are successfully not just how many people fail. What is our success rate at changing people\u2019s behavior and that is a big part of what we outlined in the brief.<\/p>\n LEONARD SIPES:<\/b> Well you are basically supporting what I\u2019m hearing from the field is that you want those variables measured, that maybe you shouldn\u2019t be held responsible for everything that the person did, if they did not receive drug treatment, if that person did not receive mental health interventions then you compare them against those who did and check the outcome and if those people who didn\u2019t do worse than those people who did, that is a measurement that is very useful in terms of funding sources.<\/p>\n BRIAN ELDERBROOM: <\/b>Absolutely, that is largely what the report is about, is comparing policy interventions, comparing the impacts of policies and looking at people who receive certain treatments or received certain changes to the ways that they are supervised on Community Supervision or the intervention that they get through prison to measure different populations to see what the impact is of policy and practice.<\/p>\n LEONARD SIPES:<\/b> Ryan not many states do this well, as far as I can see. I mean I am up on the criminal justice literature as much as anyone else. I\u2019m aware of the Washington Institutes for public policy, they have been before our microphones. I am aware of the Illinois Criminal Justice Information Authority that goes back decades in terms of doing a good job, in terms of measurement. You gave a couple of examples here specifically, Delaware and Pennsylvania was in your report but most states do not do a great job of measure the success of people coming out the prison system or measuring the success of people on parole and probation and why is that.<\/p>\n RYAN KING: <\/b>I think that we haven\u2019t really sat down and outlined the steps that states need to put in place to measure and so what the brief does is give us the four key areas that a state should be defining recidivism in a broad way. I think that was the point the Brian was just making, that we make it pretty clear and central point of this is that we are talking about multiple measures of recidivism there is no single \u201cright\u201d measure of recidivism. How you measure recidivism depends very much on what type of questions you are trying to answer. There are improvements to collection, to types of analysis and then to dissemination and I would say that there are a lot of states that are doing good things but in terms of a state that is pulling all of these different pieces together and being able to answer these key questions in a really rich and deep way, that\u2019s what I think is lacking and that is what we are trying to point out with the brief.<\/p>\n LEONARD SIPES:<\/b> Most states I would imagine are not doing it for one particular reason. This is an expensive proposition. You need qualifying people, you need money, you need time, you need data resources, you need to have the sanctity of the collection process to be unimpeachable. There are so many variables that go into good evaluations but this is not an inexpensive proposition, this is something that states are going to say to themselves, okay fine I understand the need, I understand the desire but you know, you are going to ask us to spend another three to four million dollars a year to pull this off, we don\u2019t have three or four million dollars a year to pull this off , so I would imagine expense and technical expertise is one of the reasons why states don\u2019t do this?<\/p>\n RYAN KING: <\/b>Yeah, I think we have talked about the fact that there are a number of different agencies that get involved and responsible for recidivism, I think that is one of the biggest obstacles is that you can\u2019t just go, traditionally what we see for recidivism comes out of the Departments of Corrections, from prisons because that is usually where the data come from. We don\u2019t have that. The collections across Law Enforcements, Across Community Supervision Agencies, across state lines. I would say that the Bureau of Justice Statistics who has been working on a major initiative to improve national collection of recidivism, has laid out some groundwork that I would like to think could help improve states collection within their own borders. So there are definitely are opportunities to engage. I think there is a lot of assistance particularly through some of the broader national efforts but there is no question that is going to take a serious commitment and it is going to take a serious upgrading of databases. I mean you know, and listeners will know that you know, in states many of these agencies the computers and databases don\u2019t speak with one another and so it is very difficult to measure performance, to measure reoffending performance on any of the types of our indicators that we have discussed, if you are not able to follow somebody through all the different systems that they touch.<\/p>\n LEONARD SIPES:<\/b> To pick up on one of your points that you just mentioned is that you are looking for a variety of agencies and provide data, that way it is just not the division of Correction or Parole and Probation data it is Law Enforcement, it is Treatment Provision. You are looking for as many data points as humanly possible with an offender who has one unique identifier so everybody knows what they are talking about. So you\u2019re not talking about sharing privacy information because you are not connecting that to a particular name, you are connecting it to a number and it is just an aggregate, it is big data, IBM, it is what Google, it is what Yahoo, it is what everybody else out there is doing, it is what Apple is doing. They are using big data to drive policy decisions. That is exactly what you are talking about, doing nothing more besides bringing the big data concept to the criminal justice system?<\/p>\n BRIAN ELDERBROOM: <\/b>Exactly and unlike Google which has access to large amounts of data through its own services. Each of the different Criminal Justice Actors has a different piece of the puzzle and that is the challenge with recidivism is that as Ryan said we need data sets that speak to each other so that the agencies that have responsibility for supervision and incarceration have the data to measure reoffending that comes from law enforcement and other agencies that are collecting and housing that.<\/p>\n LEONARD SIPES:<\/b> Okay what we have in essence, if we have it at all is an emulation of the Bureau of Justice statistic data on recidivism, the report just came out, oh I forget about it, little less than a year ago they updated data in essences talking about two thirds being arrested and 50% being re-incarcerated which is pretty much mimicking the two earlier data sets put out by the Bureau of Justice Assistance, but most states don\u2019t even have that, most states don\u2019t even have that level of survey, that level of following to go three years out and take a look at broader recidivism so you\u2019re saying that number one that is inadequate, number two you need to go much deeper than that, you know first of all most states don\u2019t do that. Correct?<\/p>\n BRIAN ELDERBROOM: <\/b>It is improving and I think Ryan could probably talk to the National Corrections Reporting Program which is where a large portion of the Bureau of Justice statistics data comes from.<\/p>\n RYAN KING:<\/b> Yeah there has been a significant improvement in the recidivism measurement. There has also been a really significant improvement in the NCRP, The National Corrections Reporting Program which has now been, they are getting very close to fifty states reporting. They have been filling in back data and this is an individual level data set that is actually structured in such a way that they are what are called Term Record Files. So you can actually follow an individual over time from year to year, it is an annual survey. From year to year and see how they flow through the system. So this gets back to an earlier point about if I\u2019m in a state and I say alright you have sold me on this brief but, and where am I going to get all the money to get this data together, we don\u2019t have the money to build to this capacity. I would like to see some exploration of some opportunities between some of the collection work that is going on within the Bureau of Justice statistics so there is not replication but it is important that you get this relationship where you are reporting data up for national reporting but you are getting something out of it. That is going to get obviously more buy-in from the state locals.<\/p>\n LEONARD SIPES:<\/b> We are getting better at doing and it is the larger of recidivism data. The second point I wanted to bring up is do we really want to know. I mean the individual discrete variables in terms of recidivism requires us to measure how well we are doing in terms of our drug treatment programs, in terms of our employment programs, in terms of mental health programs, in terms of how we supervise, what we supervise, a lot of folks are scared to know how well they are doing because I mean look national recidivism data is pretty consistent in terms of 50% coming back to the prison system, two thirds being rearrested. The question is do we really want to know. Do state administrators really want to know how well they are doing because that, if there is an indication of failure in terms of how well they are doing and you gave a wonderful example about Pennsylvania figuring out that their Community Corrections Center were not doing nearly as well as they could be, should be, so they rearranged the perimeters of their contracts with what I am assuming private providers saying you know, look you are going to have to do better for us to continue funding. That is exactly the sort of smart big data approach that you guys are talking about, but do we at the Criminal Justice really want to know how well we are doing and how well we are not doing?<\/p>\n RYAN KING: <\/b>Well the fact is the decisions are being made with the existing recidivism data that we are using right now. So, people are making decisions about specific policies based on broad measures. So if you\u2019re dealing with, let\u2019s say you have got a program or a policy that is dealing with drug offenders a specific slice of the drug offender in your state and you\u2019re looking at the recidivism rates for all drug offenders in your state and say when they have gone up this has clearly been a failure. Well we don\u2019t necessarily know that a huge part of the populations was not participating in this program but you can very quickly see how policy makers will make decisions they will shift funding around. We are very much seeing that in the current budget crisis and the current tightness in many states, moving dollars from spot to spot looking for things that work. So we are all using this data as is, and that is why we felt really strongly that we want to get a brief out there talking about improving it. It is not going away. It is warts and all the best performance measure that we have or at least the most frequently used one. Let\u2019s improve it. There are ways that we can do it in meaning full ways without major over hall. Some of the stuff that we are talking about here is really big picture and it is an ideal and I don\u2019t want to give the impression to listeners that you need to do all of these things or nothing. It is not all or nothing and I think the brief lays out some really small things as well as some really big more ambitious things but each step along the way is an improvement over the status quo because it is important to know policy makers now are making decisions about policies using excising data and in many cases this data is telling misleading stories.<\/p>\n LEONARD SIPES:<\/b> But we need to be held accountable do we not? Those of us in the Criminal Justice System don\u2019t we need to be held accountable?<\/p>\n BRIAN ELDERBROOM: <\/b>Absolutely, and I think for Community Supervision Practitioners in particular who have the first hand of experience of working with people who have been released from prison in particular and feel like they have a better opportunity to change their behavior through the intervention of Community Supervision then that person would have received, while incarcerated or if they had not gone to prison at all and I think it is important for in particular, people who provide those services whether its supervision or treatment or otherwise to be able to compare the effectiveness of what they are doing to prison in particular, as an alternative and one of the things that is laid out in the brief is that in order to do that you need to statically control for the population that you are studying and that is something that almost no states are doing right now and it would be great if we got to the point where we could compare probation as an intervention, to prison as an intervention using some of the statistical techniques that we talk about in the brief.<\/p>\n LEONARD SIPES:<\/b> It could save states tens of millions of dollars but we are more than half way through the program let me reintroduce you, Ladies and gentlemen we are doing a program today on Measuring Recidivism with the Urban Institute. We are here with Ryan King senior fellow with the Justice Policy Center at Urban and Brian Elderbroom Senior Research Associate at the Justice Policy Center at Urban and to remind everybody that this coming Wednesday, October 15th<\/sup> there is a webinar on this topic with Urban and the Bureau of Justice Assistance. You can find information on Urban website www.urban.org<\/span><\/a> Let me get back to the program gentlemen, we discussed before the program [PH 0:16:37.4] Joan and Peter Stillia who I interviewed before these microphones a couple of times and hopefully will interview in 2015 once again, who simply said that we within the Criminal Justice System overpromise and under-deliver when it comes to interventions in terms of the populations that we are dealing with. We really have not lived up to our promise in terms of really lowering recidivism by numbers that we would like to see and she is suggesting that we are promising too much, delivering too little and that may bite us, cause us problems in the near future. So to do that the only way of figuring that out is by doing good recidivism research as to whether or not we are over promising and under delivering correct?<\/p>\n RYAN KING: <\/b>Doing good research and making it targeted to answer specific policy relevant questions and I would flag there is some really new and interesting work that I encourage your listeners to go check out. There is a new journal article out using some of these NCRP data that I mentioned earlier. So they are able to actually following individuals through the system and it sort of flips recidivism on its head a little bit and I think it is important for us to think about that in our conversation here and looking at the proportion of individuals who go into prison for a first time, come out and never reoffend again basically people who desist and they find that in upwards of 60% of those individuals who go in on the first time don\u2019t come back again. And so the take away point from this is one that I think is not necessarily revelatory but it is worth raising again which is that the recidivism rates that are the 40% or 50% that we see are generally driven by a smaller portion of the population who churned through the system either through new crimes or revocations from Community Supervision. And so one of the pieces is that, there are things that do work and we certainly need to ask questions for these six and ten who don\u2019t return about why are they not returning and more importantly did they ever need to go to prison to begin with or would they have desisted on their own through some other sanctions or no criminal justice involvement. But for the four in ten the people who are churning through who Al Bloomstein and Alan Beck refer to as a reentry as this transient moment between confinement and the community essentially for individuals, this becomes their life. We don\u2019t know enough about that population. We have not asked the right questions about that population to find the right sort of intervention and so I do think that part of what we talk about here is multiple recidivism measure and fine tuning them so that lets talk about the six in ten who desist, lets also look at this four in ten, different types of individuals, how many times have they been in. We have to have the type of recidivism data that can be that granular to be able to really understand these sub populations and I do think certainly that Joan is correct that right now if we are looking around and saying it four in ten it is a failure, it is always been a four in ten. We need to unpack that four in ten a little bit and see what is under there. Some people are failing some people might be succeeding lets learn from the successes and lets fine tune the failures.<\/p>\n