Education Evaluation at the School/Program Level
HumRRO's approach to evaluating educational programs involves gathering quantitative and qualitative information to ensure a comprehensive assessment of key elements. We traditionally seek formative and summative input from relevant stakeholders (e.g., students, staff, and parents) using a variety of methods. Following data collection, we conduct all appropriate analyses and triangulate the data, offer recommendations to enhance the program, and present findings in writing and briefings.
HumRRO has worked with the Prince George's County Public Schools in Maryland to evaluate four educational programs. Two studies, the Biotechnology Magnet Program and the Science and Mathematics Magnet Program, involved evaluations at the program level. The Middle and High School Alternative Programs were at the school level.
The purposes of the magnet program evaluations were to review the enacted practices associated with each, and to assess the extent to which each program delivers its promised policies and practices. Each evaluation included a review of the documents describing the relevant magnet programs. To collect the requisite information, we conducted focus groups with students, parents, and teachers. We administered Web- and paper-based surveys to these groups. We also conducted interviews with internship sponsors and observed classroom practices.
We consolidated results into a score or grade for each enacted practice, as well as for the program as a whole, to indicate the quality and extent of program implementation. Overall, each magnet program received an average or better grade. Recommendations made for the Biotechnology Program included placing more emphasis on students' applied experiences, increasing the number of mini-research projects completed, and updating the equipment. For the Science and Math Program, we recommended institutionalizing selection requirements to increase the overall caliber of the program.
We gathered information about key Middle School Alternative Program elements and intervention strategies through a review of program materials and interviews with key stakeholders. Researchers determined changes in students' academic achievement school records, scores on the Woodcock-Johnson II (a commercially-available norm-based achievement test), course grades (by quarter and semester), and grade point average. To assess changes in school behavior, we administered the Behavior Assessment System for Children (BASC) and the School Climate Survey to students.
The BASC is a commercially available system of instruments that evaluates the behaviors, thoughts, and emotions of adolescents from multiple perspectives (parent, teacher, and self). The School Climate Survey is a commercially available instrument to assess perceptions of and satisfaction with the school climate from student, teacher, and parent perspectives. We gathered more information about the students' school behavior from a review of school records and by conducting focus groups with teachers and parents.
Parents, teachers, and students participated in focus groups and completed surveys that provided information about the students' knowledge of social and behavioral skills. Results indicated that students in the program performed somewhat better in the core academic subjects as evidenced by their grades. Teachers handled most discipline problems in the classrooms, and reported few suspensions. Teachers and students believed that the program had a positive effect.
A similar research design was followed to evaluate the High School Alternative Program. Through a variety of methods, formative and summative information about the program's implementation processes and outcomes was obtained from all relevant stakeholders. Efforts are currently focused on documenting and reporting changes in students' academic achievement, school behavior, and knowledge of social and behavioral skills.
For more information, contact:
Monica Gribben, or Sheila Schultz or Research Notes