RSD Achievement Growth as Compared to the State: A Response to Louisiana Educator

A recent post by Michael Deshotels on the Louisiana Educator blog criticized the Cowen Institute for ignoring two measures used by the state to assess school and district effectiveness in our 2010 State of Public Education in New Orleans report.  We thought this presented a good opportunity to explain, over a couple of posts, the different ways the state measures student achievement and school performance.  The School Performance Score, Assessment Index, and the rates of standardized test passage for a single school or type of school can present very different pictures of student achievement, and its growth or decline, at a particular school.  These posts should make clear how each of these measures is calculated and which is best to consider when interested in a school’s (or district’s) long-term or short-term performance or growth, in addition to addressing Deshotel’s concerns with achievement in RSD schools.

SPS Growth Among RSD Schools Versus All Louisiana Public Schools
One of the measures we did not highlight is the School Performance Score (SPS), a number given to every public school in Louisiana and calculated using student standardized test scores, attendance and drop-out data.  The SPS is used as a measure of absolute school performance and as a means of calculating the growth schools need to achieve in order to make Adequate Yearly Progress under the No Child Left Behind Act.  It is based on students’ test scores on all of the state tests (LEAP, iLEAP, and GEE), drop-out rates, and attendance.

A school’s Baseline SPS is calculated by averaging the previous two years of school performance data and is used to give a school a performance rating label.  Because two years of data are used, significant growth or decline in any given year (on any of the metrics used to calculate the SPS) is balanced somewhat by the prior year’s performance.  This provides a more accurate estimate of a school’s performance over time and evens out dramatic increases or declines over one year.

Deshotel’s post notes that the stated purpose of the Recovery School District (RSD) is to close the achievement gap between the low-performing schools it takes over and other schools across the state.  However, in 2008-2009, RSD schools’ SPS grew by an average of 2.6 points, compared to 4.7 points for all schools statewide.  He believes that this is a problem because the RSD will need to show a greater rate of growth in order to close this achievement gap.

An analysis of the RSD’s school and district performance scores reveals levels of system-wide growth that fall short of the state average; however, these numbers might not accurately reflect the extent of a school’s achievement growth over the course of a school year or the rate at which the achievement gap is closing or expanding.  While monitoring changes in a school’s SPS may be valuable over the long term, this method of calculation can mask true and significant short-term gains or declines.  This problem is particularly acute in the case of schools in New Orleans who just two years ago were still struggling to re-establish themselves after Hurricane Katrina.  Graduation rates, attendance records, and student test scores from even two years before may no longer be reflective of current realities at some schools.  In addition, the Department of Education notes in a press release that the RSD’s School Performance Scores suffered due to the difficulty in obtaining accurate student attendance data because of schools’ highly mobile populations, continual enrollment throughout the school year, and a previously unreliable information system.

Growth and Decline in Standardized Test Passage Rates
Another common, and in this case, more valid, way of measuring absolute performance and growth is to consider student proficiency rates on the six high stakes promotional tests given to Louisiana 4^th, 8^th (LEAP) and 10^th graders (GEE) in English language arts and math as well as the 10 other standardized tests given to 3^rd, 5^th, 6^th, 7^th, and 9^th graders as part of the iLEAP.  Proficiency rates are typically expressed as the percentage of students who score “Basic” or above on each exam.  This is an even more straightforward measure of a school’s academic performance, and can identify particular grades and subjects in which schools are excelling or struggling.

An analysis of standardized test passage rates reveals that in most grades and subjects, both RSD charter and direct-run schools demonstrated greater growth than the state average, sometimes by significant amounts.  Among the six high stakes tests administered, RSD-run schools saw less growth than the state in two exams.  In other subjects and grades, RSD-run school growth exceeded that of the state by up to 5.9 percentage points.  RSD charters showed even greater growth than the state in every high-stakes test subject and grade.  In some areas, this growth was particularly great.  While the state saw passage rates on the math portion of the GEE grow by 8.0 points, among RSD charters the rate grew by 31.5 points.  In English, growth across Louisiana was 3.2 points compared to 17.1 points among RSD charters.

Click here for a detailed table that shows the growth or decline of standardized test passage rates among each type of school compared with the state average.  Numbers highlighted in green show where RSD-run or RSD charter schools in New Orleans out-grew the state’s average performance.  Numbers highlighted in red show where they did not.

Though using the SPS as a measure of school performance would indicate that RSD schools are not closing the achievement gap, these numbers tell a different story.  In only a very few cases did RSD schools under-perform the state’s growth; in the vast majority, they out-performed.  The Cowen Institute chose to highlight the growth in test score passage rates in the 2010 State of Public Education in New Orleans report because we believe it gives a more accurate depiction of a school or district’s short term progress than the SPS score, especially in the case of RSD schools.  As school populations stabilize and data collection improves, SPS scores may become more useful for measuring one year’s gains in New Orleans’ public schools.

The GEE and Graduation Rates in RSD Schools
Deshotels also claims that the Cowen Institute ignored poor GEE passage rates and poor graduation rates in RSD schools.  He is correct that the GEE scores and graduation rates are very low in RSD high schools and should be a matter of concern to everyone.  However, as he also points out, it is important to think about rates of growth when trying to gauge the RSD’s progress.  RSD schools were taken over precisely because they were low-performing; as a result, their graduation rates should be expected to be well below average rates, at least for a while.  More important is how quickly the RSD schools can catch up.

As this chart shows, RSD schools improved much faster than the state average on the GEE tests between 2008 and 2009.  This held for charters and traditional schools in both English and Math.  While their passage rates are still unacceptably low, their progress is nonetheless promising.  Ultimately, if the state tasks the RSD with turning around low-performing schools, it should not be surprising or a strike against the RSD if those schools continue to perform below the state average as long as they are improving at a faster rate than the state.  In this case, they are.

The question of graduation rates is more complicated because of a lack of data.  We are unsure how Deshotels calculated his estimated graduation rates.  The state has only just begun using a 9^th – 12^th grade cohort graduation rate to measure graduation by high school, school district, and across the state.  However, because full cohorts have not yet gone through the post-Katrina RSD schools, no cohort rate is yet available.  According to the state’s most recent listing of cohort graduation rates:

“Districts that were heavily impacted by the hurricanes of 05-06 will not receive graduation cohort results until 2011 based on 2010 data. These districts are Cameron, City of Bogalusa, Orleans, Plaquemines, RSD, and St. Bernard.”

To our knowledge, there is no reliable way yet to estimate RSD cohort graduation rates at this time, especially beginning in the 7^th grade, since no New Orleans student who entered an RSD school in 7^th grade would yet be eligible to graduate.  Thus, we question the author’s ability to create a 7^th – 12^th grade cohort graduation rate for RSD schools that have only been fully open since the 06 – 07 school year.

That said, RSD high school graduation rates, however they are measured, are no doubt very low and need to be higher.  The question remains, are they improving and how quickly?  In Deshotel’s words, graduation rates “need to show a greater rate of growth [that the state] to catch up.”  We do not have the data at this point to answer this question, though it is one that we would like to know more about.  We certainly agree that graduation rates must improve.

It is important for all education stakeholders to put pressure on the state and the Recovery School District in order to ensure that its schools continue to improve.  While the evidence presented above shows that the RSD schools in New Orleans have out-paced the state in terms of improvements on standardized tests, it is also the case that RSD schools still have a long way to go.  In addition, there has been recent evidence that RSD high schools are not implementing their redesign plans faithfully, endangering the education of many RSD high school students.

However, while pressure should be applied, it should be applied responsibly.  Criticizing the state for the low performance of schools that were taken over because they were low-performing in the first place is dishonest.  Under the best of circumstances, the RSD will need time to improve its schools.  The process cannot happen overnight.  The Cowen Institute believes that ongoing accountability and responsible public pressure are essential to ensuring that the state remains committed to improve low-performing schools.  However, we also believe that knee-jerk and ill-informed attacks on the RSD only endanger an already difficult task.