Research on Middle School Renewal

Peter F. Mulhall, Nancy Flowers, & Steven B. Mertens

Understanding Indicators Related to Academic Performance

The recent enactment of the No Child Left Behind Act of 2001 (NCLB) makes it unequivocally clear that the academic success of students, subgroups of students, schools, and states will be assessed through annual student achievement tests for grades 3-8 (NCLB, 2002). Although many educators and advocacy groups raise serious concerns regarding the use of achievement scores as the single defining measure of success and failure for education, most citizens believe that schools need to be held to a higher standard of accountability (Rose & Gallup, 2001). Moreover, the consequences of not meeting NCLB requirements are only beginning to be understood at the state and local level. The middle grades are of particular concern to policymakers because many state, national, and international tests report significantly low performance and decreases in academic achievement, although some improvements were made more recently (Illinois State Board of Education, 2002; U.S. Department of Education, 2001).

Academic success in achievement scores in the middle grades can be attributed to a myriad of individual, family, school, and community-level factors. These mitigating factors may either promote or inhibit academic success, and often began much earlier in life. For example, research has found that students with high levels of support and expectations regarding academics have a greater likelihood for academic success compared to students not receiving the support and expectations (Felner et al., 1997).

It will become increasingly critical that educators understand these mitigating factors to attain high levels of academic success. This understanding does not suggest there is only one pathway to academic success, but in fact, there are likely to be multiple ways of meeting high academic standards. Standardized achievement tests are relatively easy to understand and interpret. There is much less known, however, about how students' backgrounds, motivation, interests, instruction, school climate, and numerous other factors influence academic achievement (Alexander, 2000).

To help broaden the understanding of the factors contributing to student achievement, the Center for Prevention Research and Development (CPRD) at the University of Illinois has been working with middle level schools throughout the country providing a series of Self-Study questionnaires. The School Improvement Self-Study deconstructs middle grade experiences from the perspective of parents, students, teachers, and administrators by providing a deeper understanding of the student backgrounds, school structures, student characteristics, and educational practices guided by comprehensive reform models and personal experiences in middle level schools (Jackson & Davis, 2000; Middle Start, 2001; National Forum to Accelerate Middle Grades Reform, 2001). The Self-Study provides data to schools for needs assessment, goal setting, program planning, implementation, and data-based decision making for school improvement. Self-Study data are also linked to local and state achievement scores to ascertain academic gains and to understand the relationships between student and school factors and standardized achievement scores.

Disaggregating the Data

Under the new NCLB legislation, academically low performing schools will now be required to disaggregate data for nine subgroups of students (e.g., by ethnicity and gender). The goal of disaggregating student achievement data is to understand differences between and within groups of students in the school that are not meeting predetermined standards of performance. Our experience has been that by disaggregating student achievement data, differences in subgroups of students are more easily detected, leaving little insight as to what educators should be doing to address these disparities. Academic-related indicators from the Self-Study can also be disaggregated by different groups of students (grade level, gender, lunch status, ethnicity). These disaggregations provide schools with information as to what factors need to change, what contributes to these changes, where to begin change efforts, and how to monitor the change process. Therefore, disaggregations of achievement scores combined with other relevant data are likely to provide better precision regarding why some student groups are performing differently and the educational strategies necessary to reach and enhance student learning.

The purpose of this paper is to provide educators with several examples of academic-related indicators reflecting student characteristics, qualities and trends that contribute to educational success or interfere with student achievement in sixth through eighth grades. The data presented here are from a large sample of nearly 32,000 sixth, seventh, and eighth grade students attending middle grade schools in Mississippi, Louisiana, and Arkansas during the 2000-01 school year. These schools are participating in the Mid South Middle Start Initiative under the direction and leadership of the Foundation for the Mid South, based in Jackson, Mississippi, and funded by the W. K. Kellogg Foundation.

Academic-Related Indicators

To demonstrate how academic-related indicators can assist the school improvement process, we provide five Self-Study disaggregations that have been associated with academic success and should help educators understand the causes of student success and failure.

Educational expectations
A well-known indicator for student success is the sense of future the child has regarding his or her educational expectations, which are rapidly changing in the middle school years. Although schools and teachers are only part of developing the value of education, they do have a fundamental role in helping students understand abilities, options, and opportunities for their future.

The Self-Study contains a series of questions that ask students their personal, parental, and teachers' academic expectations for them. Figure 1 shows student self-reported responses to the statement regarding the definite and probable likelihood of attending college. The overall trend across grade levels is relatively high; however, if we compare middle grade boys receiving a free or reduced-price lunch with boys that do not, it is apparent that the overall, or average, trend is not shared by all boys. Across all grade levels, boys receiving a free or reduced-price lunch have significantly lower expectations of attending college than boys not receiving a free or reduced-price lunch (75% vs. 85% in sixth grade). Over time, both groups show slight declines in expectations of college attendance. This type of disaggregation helps schools understand the importance of educational expectations, how they vary among groups of students, and may indicate the need for different improvement strategies.

Figure 1
Student Reports of Expected College Attendance

Figure 2 shows the overall percentage of students who report reading three or more books in the past three months. As students move through the middle grades there is a dramatic decrease (17%) in the percentage of students indicating they have read three or more books (from 52% in sixth grade to 35% in eighth grade). Examining the data by lunch status we found that both groups (free-reduced and full paid lunch) appear to begin at approximately the same level at sixth grade (52% vs. 53%), but by eighth grade students receiving free lunch are reading slightly more books (36%) than students not receiving free lunch status (33%). Although this pattern only represents small percentage point differences between groups, middle level educators, students, and parents may want to investigate this issue in greater depth to see if these differences are significant. One might argue that older students (eighth graders) are reading fewer books because the books they are reading are longer and more sophisticated. Or, some eighth grade students receiving free or reduced-price lunch may be participating in community groups or activities that require them to read books and literacy materials. However, it is only by reviewing Self-Study-like data, asking critical questions, and discovering unique school and community elements, that an issue like this could be identified and addressed.

Figure 2
Student Reports of books Read in Past Three Months

Academic efficacy
The degree to which students have a strong sense they can be successful in meeting academic and school demands has been referred to as academic self-efficacy (Jinks & Morgan, 1999). Academic self-efficacy reflects the level of confidence or competencies a student reports for completing or succeeding with academically related tasks and achievement (Roeser, van der Wolf, & Strobel, 2001). Figure 3 contains the overall student responses to the Self-Study question, "even if the work in class is hard, I can learn it." Here we can see the decline of academic efficacy during the middle grade years. Moreover, comparisons between boys and girls show that girls' sense of efficacy drops by 8% from sixth to eighth grade, compared to 2% for boys. Some could argue that this difference says more about how boys and girls perceive their sense of self at this age; however, this is a pattern that schools should examine in an attempt to understand underlying factors.

Figure 3
Student Reports of Academic Efficacy

An additional analysis, comparing boys and girls by free lunch status, found that girls not receiving free lunch had the highest level of efficacy in sixth grade. However, by the time these same girls reached the eighth grade, their academic self-efficacy had dropped by 14% to the lowest level for any student group.

Self-reported grades
Another indicator of academic success through the middle grades is students' self-report of their past year's grades. Although student self-reports of grades are likely to be less accurate as compared to their actual report cards, it does appear to provide an assessment that matches the pattern of other academic indicators, including student achievement scores.

Figure 4 shows that students report declining levels in their academic grades over the three-year middle school period. The results for all students show an 11% decrease in student reports of receiving mostly A or B grades through the middle school years. In addition, girls not receiving free or reduced-price lunch report substantially higher (72%) grades at sixth, and dropped significantly (14%) by the eighth grade. Similarly, girls receiving free lunch also dropped by 14% over the three years, despite reporting a lower level (47%) in sixth grade. This pattern further reflects how girls enter middle grade schools with high levels of grades, self-efficacy, and educational expectations that, over time, appear to decline more rapidly than these factors do for boys, regardless of lunch status.

Figure 4
Students Self-Reports of Grades

Parent involvement
The relationship between parent involvement and student achievement has been well delineated in the research literature (Hoover-Dempsey & Dadler, 1995). Parents may engage in the educational lives of middle grades students in numerous ways, but this involvement almost always involves conveying their values, attitudes, and support for education and acting in ways that reinforce that support. The Self-Study asks students 13 questions regarding the way their parents participate in their education, ranging from helping with homework to talking with them regarding future careers and jobs.

Figure 5 presents the results of student reports of how frequently their parents talk to teachers regarding their progress. Again, the overall analysis shows a 5% drop (from 23% to 18%) from sixth to eighth grade, which suggests a modest disengagement of the parent-teacher relationship. A further breakdown by gender finds that boys report a 4% decrease from sixth to eighth grades regarding parent-teacher communication, whereas girls report a 5% decrease.

Figure 5
Student Reports of Parent Communication with Teachers About Progress

The fact that boys report slightly higher teacher-parent contact over the three years may reflect teachers needing more contact and communication with parents of boys to address personal, social (behavior), or academic problems. By contrast, it may also reflect that girls do a better job of communicating teacher issues to parents that, in turn, means teachers do not have to contact parents of girls as frequently. These two questions could be further explored to determine the reasons for these changes and the apparent differences between boys and girls.

Summary

With the new reporting requirements from the NCLB legislation, the use of data to assess student and school progress is now becoming standard practice for the school improvement process. Although student achievement scores will remain the primary, if not sole, indicators for most educators and policymakers, they will provide a limited understanding of potential causes, processes, and solutions for improving student achievement for differing student groups. Reform efforts must delve deeper into the underlying factors that lead to student success and failure, and to academic-related indicators discussed above that demonstrate variation by student groups and grade levels. Further, examining practices and their impact on these indicators is critical for implementing change. The data disaggregation process further directs the school improvement by understanding the "big picture" for middle grades reform as well as identifying targets for immediate action.

Finally, the academic indicators described here should be continuously evaluated and analyzed in relationship to student achievement data. This ongoing research and analysis will expand the understanding of the influence of academic indicators on student achievement, determine how these relationships may change, and how they vary in different contexts. Finally, data-based decision making for school improvement beyond achievement scores is a relatively new and unrefined science. Emerging research and new computer technology are likely to make collecting, organizing, and using data results easier and less costly, which will ultimately lead to a better understanding of how students learn and succeed in middle level schools.

References

Alexander, P. A. (2000). Toward a model of academic development: Schooling and the acquisition. Educational Researcher, 4, 28-33.

Felner, R. D., Jackson, A. W., Kasak, D., Mulhall, P., Brand, S., & Flowers, N. (1997). The impact of school reform for the middle years: Longitudinal study of a network engaged in Turning Points-based comprehensive school transformation. Phi Delta Kappan, 78, 528-532, 541-550.

Hoover-Dempsey, K. V., & Dadler, H. M. (1995). Parental involvement in children's education: Why does it make a difference? Teacher College Record, 97, 310-333.

Illinois State Board of Education. (2002). ISAT 2002 statewide results: Performance levels and achievement gaps for all students and grades. Retrieved August 17, 2002, from http://www.isbe.net/assessment/2002ISAT.htm

Jackson, A. W., & Davis, G. A. (2000). Turning Points 2000: Educating adolescents in the 21st century. New York: Teachers College Press.

Jinks, J. L., & Morgan, V. L. (1999). Children's perceived academic self-efficacy: An inventory scale. The Clearing House, 72(4), 224-230.

Middle Start. (2001). About Middle Start. Retrieved August 17, 2002, from http://www.middlestart.org/

National Center for Education Statistics. (1996). Reading proficiency and home support for literacy (NCES 96814). Washington, DC: Author. Retrieved August 17, 2002, from http://nces.ed.gov/pubs/96814.asp

National Forum to Accelerate Middle Grades Reform. (2001). Our vision statement. Retrieved August 17, 2002, from http://www.mgforum.org/vision.asp

No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002).

Roeser, R. W., van der Wolf, K., & Strobel, K. R. (2001). On the relations between social-emotional and school functioning during early adolescence: Preliminary finding from Dutch and American samples. Journal of School Psychology, 39(2), 111-144.

Rose, L., & Gallup, A. M. (2001). The 33rd Annual Phi Delta Kappan/Gallup Poll of the public's attitude toward public schools. Phi Delta Kappan, 83(1), 41-58.

U.S. Department of Education. (2001). Outcome of learning: Results from the 2000 program for international student assessment of 15-year olds in reading, mathematics, and science literacy (NCES 2002115). Washington, DC: National Center for Education Statistics.

Peter F. Mulhall (mulhall@uiuc.edu) is the director, Nancy Flowers (nflowers@uiuc.edu) is the coordinator of research programs, and Steven B. Mertens (mertens@uiuc.edu) is a senior research scientist at the Center for Prevention Research and Development at the University of Illinois, Champaign.