Measuring Offender Recidivism-The Urban Institute
DC Public Safety Radio
Radio show at http://media.csosa.gov/podcast/audio/2014/10/measuring-offender-recidivism-urban-institute/
LEONARD SIPES: From the nation’s capital this is DC Public Safety. I am your host Leonard Sipes. Ladies and gentlemen our show today is Measuring Recidivism. We have folks from the Urban Institute Ryan King. He is a senior fellow with the Justice Policy Center again at Urban Institute. Brian Elderbroom a Senior Research Associate with the Justice Policy Center at the Urban Institute. This is all in preparation for a Wednesday, October 15th webinar at 2’o clock in the afternoon with the Urban Institute and the Bureau of Justice Assistance we encourage everybody to go to the website www.urban.org for the document I am about to mention and the webinar and I will bring up the webinar a couple more times throughout the course of the program. Ryan and Brian welcome to DC Public Safety.
RYAN KING: Thanks for having us.
BRIAN ELDERBROOM: Thanks for having us.
LEONARD SIPES: Like I said that before we hit the record button it’s the Ryan and Brian show. It sounds like a morning drive time radio show, so you know it should be an interesting program. You did a report called Improving Recidivism as a performance measure and I just want to read from it ever so briefly at the very beginning to kick of the conversation. Performance measurement establishing metrics for its excess and assessing results is crucial first top in informed decisions made by all areas of government including criminal justice policy, understanding the outcome, the funding and policy decisions is critical to improving government performance and providing the best return on tax payers investments. If that’s true Ryan or Brian why don’t we do it? Why are we in the criminal justice system, why are we so reluctant to being involved in measuring recidivism.
RYAN KING: Well I would say it’s not a reluctance to be involved but it’s a little bit more about the way that we are doing it. The reason, the impetus behind doing this brief was a recognition that for better or for worse recidivism is a primary performance measure that is being used in the fields of corrections and there are a lot of reasons why that is troubling practitioners and policy makers and researchers a like I think have a range of criticism but recidivism as sort of an end measure basically a measure of, when somebody comes in the system is that intervention while they are in the system, having some sort of positive outcome and result that is leading to a reduction in future offending and so if we have and imperfect measure. I think Brian and I and other folks at the Urban Institute felt it was really important for us to try to sketch out ways that we could improve it because it is not going away and I think one of the key things is sort of flagged right from the get go is that different folks have different criticisms about it and I think the biggest issue is that every individual who comes into the system has a different story, they have a different history and they are touched by a number of different parts of the system and so recidivism traditionally is looked at as a measure of success for prisons. People coming into prison do they succeed when they come back out again?
LEONARD SIPES: Well I am going to go over to Brian on this question. When we say and I talked to a couple of practitioners from around the field saying I was going to do at this radio show today with Urban and they are saying, you know Leonard we have individuals that come to us in the Criminal Justice System that come to us from Community Supervision. They have histories of substance abuse, histories of mental health problems and they have a lousy job record. They spend a lot of time, in many cases in the American Prison System, women offenders in particular come out and they have to deal with kids, really the odds are so stacked against them and it is not as if we have the money and the wherewithal and the resources to remediate so many of the social problems that they bring to us, why should we be held responsible for that persons success when we don’t control all the variables that go into that persons success or failure. Answer that question for me Brian.
BRIAN ELDERBROOM: I think for those very dedicated professionals who are working with people involved in the Criminal Justice System, they care deeply about those outcomes that you are describing, not just whether they desist from future criminal behavior but whether they find stable housing, whether they find a job or that they stop using drugs and alcohol and those are important measures by which they should judge their success as well. At the end of the day what we outline in the brief is that a single state wide rate of recidivism obscures a lot of what you are describing in that comment and we need to incorporate other success measure in addition to the ones that you described. How long is someone successful, how many people are successfully not just how many people fail. What is our success rate at changing people’s behavior and that is a big part of what we outlined in the brief.
LEONARD SIPES: Well you are basically supporting what I’m hearing from the field is that you want those variables measured, that maybe you shouldn’t be held responsible for everything that the person did, if they did not receive drug treatment, if that person did not receive mental health interventions then you compare them against those who did and check the outcome and if those people who didn’t do worse than those people who did, that is a measurement that is very useful in terms of funding sources.
BRIAN ELDERBROOM: Absolutely, that is largely what the report is about, is comparing policy interventions, comparing the impacts of policies and looking at people who receive certain treatments or received certain changes to the ways that they are supervised on Community Supervision or the intervention that they get through prison to measure different populations to see what the impact is of policy and practice.
LEONARD SIPES: Ryan not many states do this well, as far as I can see. I mean I am up on the criminal justice literature as much as anyone else. I’m aware of the Washington Institutes for public policy, they have been before our microphones. I am aware of the Illinois Criminal Justice Information Authority that goes back decades in terms of doing a good job, in terms of measurement. You gave a couple of examples here specifically, Delaware and Pennsylvania was in your report but most states do not do a great job of measure the success of people coming out the prison system or measuring the success of people on parole and probation and why is that.
RYAN KING: I think that we haven’t really sat down and outlined the steps that states need to put in place to measure and so what the brief does is give us the four key areas that a state should be defining recidivism in a broad way. I think that was the point the Brian was just making, that we make it pretty clear and central point of this is that we are talking about multiple measures of recidivism there is no single “right” measure of recidivism. How you measure recidivism depends very much on what type of questions you are trying to answer. There are improvements to collection, to types of analysis and then to dissemination and I would say that there are a lot of states that are doing good things but in terms of a state that is pulling all of these different pieces together and being able to answer these key questions in a really rich and deep way, that’s what I think is lacking and that is what we are trying to point out with the brief.
LEONARD SIPES: Most states I would imagine are not doing it for one particular reason. This is an expensive proposition. You need qualifying people, you need money, you need time, you need data resources, you need to have the sanctity of the collection process to be unimpeachable. There are so many variables that go into good evaluations but this is not an inexpensive proposition, this is something that states are going to say to themselves, okay fine I understand the need, I understand the desire but you know, you are going to ask us to spend another three to four million dollars a year to pull this off, we don’t have three or four million dollars a year to pull this off , so I would imagine expense and technical expertise is one of the reasons why states don’t do this?
RYAN KING: Yeah, I think we have talked about the fact that there are a number of different agencies that get involved and responsible for recidivism, I think that is one of the biggest obstacles is that you can’t just go, traditionally what we see for recidivism comes out of the Departments of Corrections, from prisons because that is usually where the data come from. We don’t have that. The collections across Law Enforcements, Across Community Supervision Agencies, across state lines. I would say that the Bureau of Justice Statistics who has been working on a major initiative to improve national collection of recidivism, has laid out some groundwork that I would like to think could help improve states collection within their own borders. So there are definitely are opportunities to engage. I think there is a lot of assistance particularly through some of the broader national efforts but there is no question that is going to take a serious commitment and it is going to take a serious upgrading of databases. I mean you know, and listeners will know that you know, in states many of these agencies the computers and databases don’t speak with one another and so it is very difficult to measure performance, to measure reoffending performance on any of the types of our indicators that we have discussed, if you are not able to follow somebody through all the different systems that they touch.
LEONARD SIPES: To pick up on one of your points that you just mentioned is that you are looking for a variety of agencies and provide data, that way it is just not the division of Correction or Parole and Probation data it is Law Enforcement, it is Treatment Provision. You are looking for as many data points as humanly possible with an offender who has one unique identifier so everybody knows what they are talking about. So you’re not talking about sharing privacy information because you are not connecting that to a particular name, you are connecting it to a number and it is just an aggregate, it is big data, IBM, it is what Google, it is what Yahoo, it is what everybody else out there is doing, it is what Apple is doing. They are using big data to drive policy decisions. That is exactly what you are talking about, doing nothing more besides bringing the big data concept to the criminal justice system?
BRIAN ELDERBROOM: Exactly and unlike Google which has access to large amounts of data through its own services. Each of the different Criminal Justice Actors has a different piece of the puzzle and that is the challenge with recidivism is that as Ryan said we need data sets that speak to each other so that the agencies that have responsibility for supervision and incarceration have the data to measure reoffending that comes from law enforcement and other agencies that are collecting and housing that.
LEONARD SIPES: Okay what we have in essence, if we have it at all is an emulation of the Bureau of Justice statistic data on recidivism, the report just came out, oh I forget about it, little less than a year ago they updated data in essences talking about two thirds being arrested and 50% being re-incarcerated which is pretty much mimicking the two earlier data sets put out by the Bureau of Justice Assistance, but most states don’t even have that, most states don’t even have that level of survey, that level of following to go three years out and take a look at broader recidivism so you’re saying that number one that is inadequate, number two you need to go much deeper than that, you know first of all most states don’t do that. Correct?
BRIAN ELDERBROOM: It is improving and I think Ryan could probably talk to the National Corrections Reporting Program which is where a large portion of the Bureau of Justice statistics data comes from.
RYAN KING: Yeah there has been a significant improvement in the recidivism measurement. There has also been a really significant improvement in the NCRP, The National Corrections Reporting Program which has now been, they are getting very close to fifty states reporting. They have been filling in back data and this is an individual level data set that is actually structured in such a way that they are what are called Term Record Files. So you can actually follow an individual over time from year to year, it is an annual survey. From year to year and see how they flow through the system. So this gets back to an earlier point about if I’m in a state and I say alright you have sold me on this brief but, and where am I going to get all the money to get this data together, we don’t have the money to build to this capacity. I would like to see some exploration of some opportunities between some of the collection work that is going on within the Bureau of Justice statistics so there is not replication but it is important that you get this relationship where you are reporting data up for national reporting but you are getting something out of it. That is going to get obviously more buy-in from the state locals.
LEONARD SIPES: We are getting better at doing and it is the larger of recidivism data. The second point I wanted to bring up is do we really want to know. I mean the individual discrete variables in terms of recidivism requires us to measure how well we are doing in terms of our drug treatment programs, in terms of our employment programs, in terms of mental health programs, in terms of how we supervise, what we supervise, a lot of folks are scared to know how well they are doing because I mean look national recidivism data is pretty consistent in terms of 50% coming back to the prison system, two thirds being rearrested. The question is do we really want to know. Do state administrators really want to know how well they are doing because that, if there is an indication of failure in terms of how well they are doing and you gave a wonderful example about Pennsylvania figuring out that their Community Corrections Center were not doing nearly as well as they could be, should be, so they rearranged the perimeters of their contracts with what I am assuming private providers saying you know, look you are going to have to do better for us to continue funding. That is exactly the sort of smart big data approach that you guys are talking about, but do we at the Criminal Justice really want to know how well we are doing and how well we are not doing?
RYAN KING: Well the fact is the decisions are being made with the existing recidivism data that we are using right now. So, people are making decisions about specific policies based on broad measures. So if you’re dealing with, let’s say you have got a program or a policy that is dealing with drug offenders a specific slice of the drug offender in your state and you’re looking at the recidivism rates for all drug offenders in your state and say when they have gone up this has clearly been a failure. Well we don’t necessarily know that a huge part of the populations was not participating in this program but you can very quickly see how policy makers will make decisions they will shift funding around. We are very much seeing that in the current budget crisis and the current tightness in many states, moving dollars from spot to spot looking for things that work. So we are all using this data as is, and that is why we felt really strongly that we want to get a brief out there talking about improving it. It is not going away. It is warts and all the best performance measure that we have or at least the most frequently used one. Let’s improve it. There are ways that we can do it in meaning full ways without major over hall. Some of the stuff that we are talking about here is really big picture and it is an ideal and I don’t want to give the impression to listeners that you need to do all of these things or nothing. It is not all or nothing and I think the brief lays out some really small things as well as some really big more ambitious things but each step along the way is an improvement over the status quo because it is important to know policy makers now are making decisions about policies using excising data and in many cases this data is telling misleading stories.
LEONARD SIPES: But we need to be held accountable do we not? Those of us in the Criminal Justice System don’t we need to be held accountable?
BRIAN ELDERBROOM: Absolutely, and I think for Community Supervision Practitioners in particular who have the first hand of experience of working with people who have been released from prison in particular and feel like they have a better opportunity to change their behavior through the intervention of Community Supervision then that person would have received, while incarcerated or if they had not gone to prison at all and I think it is important for in particular, people who provide those services whether its supervision or treatment or otherwise to be able to compare the effectiveness of what they are doing to prison in particular, as an alternative and one of the things that is laid out in the brief is that in order to do that you need to statically control for the population that you are studying and that is something that almost no states are doing right now and it would be great if we got to the point where we could compare probation as an intervention, to prison as an intervention using some of the statistical techniques that we talk about in the brief.
LEONARD SIPES: It could save states tens of millions of dollars but we are more than half way through the program let me reintroduce you, Ladies and gentlemen we are doing a program today on Measuring Recidivism with the Urban Institute. We are here with Ryan King senior fellow with the Justice Policy Center at Urban and Brian Elderbroom Senior Research Associate at the Justice Policy Center at Urban and to remind everybody that this coming Wednesday, October 15th there is a webinar on this topic with Urban and the Bureau of Justice Assistance. You can find information on Urban website www.urban.org Let me get back to the program gentlemen, we discussed before the program [PH 0:16:37.4] Joan and Peter Stillia who I interviewed before these microphones a couple of times and hopefully will interview in 2015 once again, who simply said that we within the Criminal Justice System overpromise and under-deliver when it comes to interventions in terms of the populations that we are dealing with. We really have not lived up to our promise in terms of really lowering recidivism by numbers that we would like to see and she is suggesting that we are promising too much, delivering too little and that may bite us, cause us problems in the near future. So to do that the only way of figuring that out is by doing good recidivism research as to whether or not we are over promising and under delivering correct?
RYAN KING: Doing good research and making it targeted to answer specific policy relevant questions and I would flag there is some really new and interesting work that I encourage your listeners to go check out. There is a new journal article out using some of these NCRP data that I mentioned earlier. So they are able to actually following individuals through the system and it sort of flips recidivism on its head a little bit and I think it is important for us to think about that in our conversation here and looking at the proportion of individuals who go into prison for a first time, come out and never reoffend again basically people who desist and they find that in upwards of 60% of those individuals who go in on the first time don’t come back again. And so the take away point from this is one that I think is not necessarily revelatory but it is worth raising again which is that the recidivism rates that are the 40% or 50% that we see are generally driven by a smaller portion of the population who churned through the system either through new crimes or revocations from Community Supervision. And so one of the pieces is that, there are things that do work and we certainly need to ask questions for these six and ten who don’t return about why are they not returning and more importantly did they ever need to go to prison to begin with or would they have desisted on their own through some other sanctions or no criminal justice involvement. But for the four in ten the people who are churning through who Al Bloomstein and Alan Beck refer to as a reentry as this transient moment between confinement and the community essentially for individuals, this becomes their life. We don’t know enough about that population. We have not asked the right questions about that population to find the right sort of intervention and so I do think that part of what we talk about here is multiple recidivism measure and fine tuning them so that lets talk about the six in ten who desist, lets also look at this four in ten, different types of individuals, how many times have they been in. We have to have the type of recidivism data that can be that granular to be able to really understand these sub populations and I do think certainly that Joan is correct that right now if we are looking around and saying it four in ten it is a failure, it is always been a four in ten. We need to unpack that four in ten a little bit and see what is under there. Some people are failing some people might be succeeding lets learn from the successes and lets fine tune the failures.
LEONARD SIPES: The report again is Improving Recidivism as a performance measure again available through the website at Urban www.urban.org. Right where do we go to from here? Now you are going to be doing this seminar and you are going to be talking to a lot of people in the system, their cynical. Oh god here we go. I had this discussion with my folks upstairs before doing this radio show. Probably my biggest criticism of my people in the field about what it is that I do on the radio and the television side is Leonard, you bring these folks in from NIJ and Urban and Pew and Justice and they talk about these wonderful grandiose ideas for the rest of us, wow don’t they understand what it is that we are dealing with out here. I mean with huge case loads and no money for treatment and no money…, I mean and so we are going to measure what, our ineptitude, our lack of resources, and our lack of public confidence as to why we don’t have the money to really impact the lives of the people on our case loads. How do you respond to them? They are frustrated. What you are talking about is a very detailed big data policy. They are talking about, for the love of god Leonard I have a case load of 150-1, 250-1, how am I suppose to meaningfully intervene in the lives of these people when I have these large case loads. When are the folks from Urban and Pew and everybody else going to deal with my reality? So their reality is, I am going to guess as what I said before. They don’t have the money nor the wherewithal to do the sort of measurement that you are talking about but if they don’t, policy mistakes are going to continue to be made. Tens of millions of dollars or even billions of dollars could be wasted, correct?
RYAN KING: Here is the thing that I think and some of that frustration and hopelessness comes from the fact that we really are using these blunts and imprecise recidivism measures and so you are right, you are doing good work and you may actually, let’s put a side case on, let’s say you are actually doing good work and having a positive impact but an overall say why recidivism measure is not going to point that out, so you’re not going to be able to draw attention on that. What I think is important about this is that the types of things and steps that we point out in the report and again its important to note that there is a lot of things that is big and ambitious but there are also small steps that can be taken. Those actually can help individuals and the corrections and they are designed in a way so that the idea that I shouldn’t be held accountable for the ten different systems that have touched the person’s life. Why should I be the one that is held accountable? We are talking about doing a level of analysis that controls and identifies and is precise enough so that your people are being held accountable for their particular steps, their particular interventions in the system and it is important to note that a lot of the stuff is already going on in states. So we have examples in the report there are other states that are having this webinar. It is primarily focused on practitioners and individuals working in the states to get an opportunity. So some of the stuff it sounds big and the way we are talking about here it sounds ominous and there is no way I can do that but I think you will find that there are states out there that are able to make some really positive impacts and we are going to share some of those stories on the webinar. Of states that are able to use their excising capacity. Some of it is not necessary they may already have the data collection capacity they just haven’t thought about doing an analysis in this way. They just done it one way forever and they just keep doing it. But with some minor modifications and trying to get some people from states to talk with one another we can make some of these improvements.
BRIAN ELDERBROOM: Let me talk a little bit about how one of the measures we outline in the report can actually feed into a policy conversation in a way that helps practitioners. So one of the measures we talk about is time of failure and the research from many people shows that people are most likely to fail in the days, weeks, and month shortly after release and the vast majority and people who go on to commit another crime after being released from prison do so in the first year after they have been released. That is an important finding, it is an important recidivism finding and what we can learn from that is supervision terms of 3, 5,10 years are not affectively deterring criminal reoffending and so what we need is to reduce case loads to the point where we are really focusing on those people who are highest risk and the people who are going to need the most help when they first leave prison. And that is how using a new recidivism measure like time to failure can inform policy in a way that helps practitioners focus on the people who need supervisions and interventions the most.
LEONARD SIPES: You know it is intriguing. Is there the possibility and one of you brought it up before of the idea of Community Supervision as being as effective or more effective than incarceration itself and incarceration rates have gone back up after a loss since 2009 according to the Bureau of Justice Statistics. Some people have suggested well we are not doing the job we need to do at the state and local and county level in terms of proving how viable we are. It is just not convincing people that we say we over incarcerate, we say we incarcerate for too long, we hold people for too long. That the fiscal burden on the states is enormous, I mean virtually every governor in the country has had a discussion with his or her Corrections Administrator that you have got to bring down the budget. You have got to do something about recidivism. The only way we are going to do any of this is to do good research. I mean how else are suppose to manage the Criminal Justice System and relieve states of this enormous burden that they are crying about. How else are we going to do that unless we do good research?
RYAN KING: And we need to know what works. There states, recidivism rates can go up and they can go down completely independent of any policy changes. That is the important things that we really want to point out in there. As a profile of a prison population becomes more or less risky then you are going to have recidivism rates and so the question is did my recidivism rate go down because I am releasing individuals who are at very low risk of reoffending versus prior release cohorts or they go down because we are putting in place appropriate policy interventions and evidence based practices to adjust recidivism.
LEONARD SIPES: The Washington state’s Institute for Public Policy found that because they were saying it is not a measure of not doing it well but that we were releasing people at a higher risk thereby those people at a higher risk are going to recidivate more. It is not the policy it is the people correct?
RYAN KING: Exactly, that’s exactly we talk about that example in our report. That is a perfect example and another in some prior work looking at recidivism rates in Oregon and Oklahoma or two states that had recidivism rates down in the mid 20% some of the lowest in the country. One of them, Oregon had taken very deliberate steps to try and address recidivism so you would talk to people in the states and they would point to some of those practices. They identified recidivism as a problem and they wanted solutions. Oklahoma on the other hand, we talked to individuals who worked with data there and they said they were filling their prisons up with very low risk people and so naturally their recidivism rates were low and so those are two very different policy conversations. If you had controlling for the underlying population and don’t know that how can you possibly be making informed decisions about directions of your recidivism or to invest your resources.
LEONARD SIPES: You can save literally millions, tens of millions, billions of dollars over time if you had the data to guide you. You can protect public safety if you have the data to guide you.
RYAN KING: Without question. I think we are seeing that now and that’s why it is an exciting time, that’s why I think it’s a good time for this brief because in a time when we have more research and more data and better guides about what works, the question now is how can states invest there resource and one of the first steps they need to be able to do is identify drivers of the population and put in place some of these practices. You do need these data but there are many different steps and as I said there is sort of an ideal perfect system that nobody has but there are a lot of steps that states can take now with a limited amount of additional resources that can really improve their understanding of recidivism in their state.
LEONARD SIPES: This is a terribly unfair question to ask either one of you but does Office of Justice Programs, BJS, Bureau Justice Assistance is anyone prepared. I know that the Federal Government has put tons of money into state analytical centers throughout the years to improve the quality of their data but we have the technical assistance and funding at hand to help states come to grips with what you are suggesting.
RYAN KING: First of all I would say a BJA supported this brief and they are in support of this work so I think that is a great sign right there and certainly in conversations we have there is an acknowledgement and understanding from folks in BJA about the importance of this issue and we will certainly continue to carry that message over there so that is a great area where we love to support and I think again the work that BJS has been doing to improve recidivism in partnership with [PH 0:28:17] Apton and NCRP data collection has been absolutely fantastic. There is enormous improvements that are not just for national level but can have benefits for the states so there is a lot of leadership and understanding this issue is not news to anybody, working here on this issue here in Washington and what we are hoping to accomplish at this webinar is to get folks in the state aware of it and hopefully there can be opportunities down the road for additional leadership and guides and support from the federal government for states.
LEONARD SIPES: Brian, 30 seconds left do you have anything to add?
BRIAN ELDERBROOM: No I think one of your first comments was that these are professionals that are for the most part overworked and under resourced and I think that is an important point to make and part of what we are trying to do with this brief is to help practitioners communicate their success.
LEONARD SIPES: This has been a fascinating conversation and I really do want to encourage everybody this Wednesday, October 15th the webinar on this issue on Measuring Recidivism with the Urban Institute and the Bureau of Justice Assistance at 2’o clock. Go to the website www.urban.org. Our guests today have been Ryan King and Brian Elderbroom from the wonderful Urban Institute and I really appreciate both of you being here. Ladies and gentlemen this is DC public safety. We do appreciate your comments. We even appreciate you criticism and we want everybody to have yourselves a very, very pleasant day.
