The recent spotlight on circumstances surrounding our nation’s education system has highlighted the need for positive school reform. As part of this effort to bring about positive change, organizations are being asked, or required by legislative or other mandate, to study the validity, reliability, quality, and impact of various educational assessment programs. As testimony to the value of our services, HumRRO has been selected by competitive bid to be the independent evaluator of the California High School Exit Examination (CAHSEE) for the California Department of Education (CDE) four times since 1999. Our independent evaluation of the CAHSEE includes four interrelated phases:
- Planning the evaluation by formulating the primary research questions mandated by the legislature as well as questions to address special studies that will assist in the formation of state educational policy regarding this graduation requirement.
- Collecting data using a variety of methods such as focus groups, workshops, customized surveys, observations, and web research.
- Generating key statistics and analyzing the statistical outcomes to inform recommendations for policy decisions and future research.
- Communicating results in reports and presentations to inform key audiences (e.g., members of the Board of Education, the public) of the evaluation process and its fi ndings.
Over the past ten years, our research has addressed core questions such as (a) what effect has the CAHSEE requirement had on high school completion and college attendance? (b) how are students performing on the examination over time? and (c) does the CAHSEE requirement have differential effects across demographic groups?
With annual and biennial reports, HumRRO sheds light on answers to these questions and points the way for further research. The credibility and clarity of both the evaluation process and the results are critical to ensuring that fi ndings are well understood and acted upon post-evaluation. For example, comparisons of outcomes before and after the CAHSEE requirement took effect have refuted the concern that the exit examination alone would greatly reduce graduation rates. Analyses of trends in passing rates over time have indicated steady improvement in student competency in the content standards measured by the CAHSEE. Additionally, future research was recommended to investigate the persistent though slightly narrowing gaps in test results among some subgroups.
Our Research And Reports Have Notable Impact
In 2003, HumRRO recommended a delay in implementation of the CAHSEE requirement, until adequate advances were made in the alignment of instruction to the content standards measured by the CAHSEE. The State Board of Education subsequently delayed implementation of the requirement for two years for all students and longer for students in special education programs, based largely on evidence from our instruction study that more time was needed to align instruction to the CAHSEE content, particularly for students in special education programs.
Our Special Studies Advance Knowledge Of Emerging Issues
In 2010 HumRRO conducted a two-part study to analyze a state panel’s recommended alternative means to the CAHSEE for students with disabilities. First, we analyzed other standardized test scores of students with disabilities who had not passed the CAHSEE. Next, we developed and administered to special education teachers and coordinators a web-based survey to investigate the feasibility and comparability of the proposed alternative means. Key results from the special study included two fi ndings: (a) a small number of students with disabilities have mastered the required skills but cannot pass the CAHSEE and (b) teachers are generally supportive of an alternative means but concerned about the reliability of the evaluation of student work samples. HumRRO recommended CDE conduct a pilot study to explore the alternative means criteria. This pilot study is now in progress.