Basket of measures defines new student performance index

Basket of measures defines new student performance index

(Calif.) For more than two years, a state commission has struggled to find valid and reliable student performance indicators that could reduce the influence of the almighty test score.

Meanwhile, a group of bellwether districts operating independently under a federal waiver are less than a year away from being formally judged under just that sort of system.

Members of the California Office to Reform Education – CORE – have defined and are in the process of implementing an accountability program where test scores and student academic measures account for 60 percent of the evaluation and other factors such as a school’s culture, climate and social settings account for the other 40 percent.

“We wanted to move away from a one-dimensional accountability system and replace it with a basket of measures we think will give a more holistic picture of how schools and kids are doing,” said Noah Bookman, CORE’s chief accountability officer.

It is noteworthy that CORE’s School Quality Index would seem to fulfill the mandate imposed on the Public Schools Accountability Act Advisory Committee by SB 1458, a 2012 bill by former Senate leader Darrell Steinberg aimed at reducing reliance on test scores for measuring school success.

But the advisory committee, which has met almost monthly on the question since 2012, has also been forced to do its work in a more politically-complex arena as the Brown administration, the California Department of Education and the rest of the state’s schools not affiliated with CORE have been focused on implementing Common Core State Standards and preparing for new assessments aligned to the new content goals.

By virtue of winning a federal No Child Left Behind waiver, the participating CORE districts – Fresno, Long Beach, Los Angeles, Oakland, San Francisco, Sanger and Santa Ana (Sacramento City Unified has dropped out ) – have had far more freedom to consider options and make decisions.

Still, the CORE index offers some important differences to the state’s current accountability system – the Academic Performance Index – and may prove an important model for what is eventually established for the rest of the state.

For one, the CORE index measures performance changes of individual students year-over-year, as opposed to the API, which measures changes experienced by an entire school population from one year to the next.

The group has also agreed on the non-test score measures representing 40 percent of the evaluation – the same percentage called for in the Steinberg bill.

And unlike some state officials working on the API update, the CORE districts have dispatched the idea of trying to find a single measure that would reflect college and career readiness – arguing instead that the entire index provides that valuation when taken as a whole.

“The genesis of the index came from this set of districts examining what they thought was important around education,” said Bookman. “We wanted a system that wouldn’t be used as a hammer but instead would support meaningful adjustments in needs and strengths.”

Dissatisfaction with existing models for judging school performance has largely focused on the federal system, defined by the Adequate Yearly Progress measure – which set hard performance targets for schools to meet and a 2014 deadline for when all students needed to be proficient.

Although the AYP system is universally understood to be flawed, it remains the law of the land except for those states that – like the CORE districts – have been granted the NCLB waiver. California is one of only a handful of states that have not received the waiver.

California’s API, established in 2000 a year ahead of NCLB, was considered by many educators at the time a better approach because it measured growth instead of assigning hard targets. Over time, however, because the API is solely based on test scores, interest in broadening the state system has grown.

The Steinberg bill requires the state to include graduation rates as a component of the API by 2016 and limits the value of test scores to no more than 60 percent. The work of the accountability committee at the direction of the state board has expanded to include other measures including college-career readiness.

CORE’s index also restricts test scores and academic measures to 60 percent, based on the following:

  • The percentage of a school’s students that meet grade level standards in English language arts and math as measured by state standardized testing makes up 20 percent of the score for high and middle schools and 30 percent for elementary schools.
  • The year-over-year growth of individual students in comparison to other similar students. This growth measure accounts for 30 percent of the score for elementary schools and 20 percent for middle and high schools.
  • There is also a 20 percent score for middle and high schools tied to the number of in-coming 9th graders that are held back – a measure that is still under review.

The CORE districts are using the same state tests that are in use elsewhere – the computer-based Smarter Balanced assessments that are aligned to the Common Core.  

Using individual student scores to measure growth – instead of scores for the entire school – is an important distinction, said Bookman, who noted the API doesn’t take into account the change in students coming into a school one year versus the next. “We are not precisely examining how a school did in taking the students who started the school year to the end of the school year.”

CORE’s index gives 40 percent value to non-test indicators:

  • School absentee rates
  • School suspensions/expulsions
  • Student social-emotional skills
  • Surveys from parents, staff and students
  • Disproportionality in special education
  • English learner re-designation rates

Some of these indicators all schools in California track because of reporting requirements under the state’s Local Control Funding Formula – student absentee rates and suspensions/expulsions, for instance. Districts also report on the race and ethnicity of students being placed into special education as well as the number of English learners that promote out of that designation.

What is new with the CORE index, however, is the size of a qualifying subgroup – just 20 students, compared to the API’s subgroup of 100. “The intent here is to make more students visible,” said Bookman, “so that we can work on the identifiable gaps.”

The survey section is intended to draw input in four areas – teaching and learning, interpersonal relationships, safety, and community engagement. Students in grades five to 12 are included in the school climate surveys as are teachers, parents and other caregivers.

Perhaps the most complicated non-test measure is on a student’s social and emotional skills. Much of this indicator has subjective elements – such as a teacher’s sense of a student’s mindset to accept challenges, learn from criticism and persist in the face of setbacks.

There’s also an element around self-management such as regulating emotions, managing stress; and social awareness, which centers on the ability to empathize with others despite differences in background and culture.

“These are skills that are really meaningful in helping us become successful in life,” Bookman explained. “We’re also looking at what’s called the ‘growth mindset,’ which is the sense or belief that you can get smarter – by putting out effort I can get better at things.”

CORE districts have already been measured on most of the academic elements and schools identified as needing help have started on improvement plans. Next summer or fall, schools will receive their first formal scoring under the CORE index that will include most of the non-test measures, although disproportionality in special education will be for information only.

It is important to note that non-CORE schools are lobbying the Brown administration for another year before the scores from the Smarter Balanced system will be used for accountability purposes. But CORE districts have already been told by federal officials that they will be held accountable for the student performances next spring.

...read more