The purpose, need, and the road ahead
Nupur Hukmani
In this article, I attempt to present the literature on school effectiveness and how its definition and process have changed over the years; why it is so important to relook the lens through which we measure the effectiveness of schools and to assess some concrete ways of going about that.
The other day I was in a conversation with a mother of a student in the school I teach. I am the child’s class teacher. She wanted to speak to me about her child’s progress, which to me was proof of her investment in her child’s education. She started the conversation by letting me know how difficult it had been for her to put her child in a “good” school against a slew of religious, financial, and family constraints. She had stood up for her daughter in more ways than one so that her child could receive an education that would help her out of the cycle of poverty and dogma.
I updated her about her daughter’s academic progress in various subjects; I jubilantly informed her about the several extra-curricular activities the child was good at and I proudly exclaimed how her daughter exemplified values of respect, excellence, hard work, and collaboration. She seemed unmoved. “That’s great! But does she understand the struggles and restraints that exist in life despite which she is lucky enough to get an excellent education? Does she understand that only a good education will remove some of these constraints?”
Schools are important institutions in society. Right from the individual citizen to schools themselves, society, and at a larger level the government, all stakeholders want to know how effective they are. In most societies, schools are mainly assessed on the basis of whether they teach foundational numeracy and literacy to students in an environment that is structured and functional. For example, in India, we have the ASER survey, which is an education survey conducted to provide a glimpse of how India’s children are faring in literacy and numeracy and whether there is basic infrastructure in place, such as a playground, well-maintained toilets, or access to mid-day meals. Over the years, this national survey has provided essential school effectiveness data that subsequently informs education policy in India. At the global level, we have surveys like Program for International Student Assessment (PISA) that measures reading, math, and science to rank countries on these student achievement outcomes.
No doubt student achievement and school infrastructure data are significant to assess the effectiveness of schools, and in turn the effectiveness of the related education policy, but these are rudimentary data points or just performance indicators in measuring the development of an entire human being and the larger society. There are so many functions that schools must serve including but not limited to building a personal value system in students, providing them with ample exposure to build knowledge, skills and mind-sets to become purposeful, skilled, and socially conscious individuals. Schools are responsible spaces to build long-term outcomes like creating an employable force that will contribute to nation-building. They are responsible for constructing community consciousness. With such gigantic responsibility resting on schools, it is surprising that student achievement is often the only measure of school effectiveness that we focus on.
The conversation with the parent got me thinking about the purpose of education and the role of schools in our society. It made me realize how effective schools are not a norm but an anomaly. However, the bigger question that got me thinking was, how do we define “effective schools” and measure their effectiveness? A pioneering study on the effectiveness of schools called the Coleman Report, carried out in the 70s concluded that “schools make no difference”. The report established that the most important predictor of student outcomes was not the school or its resources, but the child’s home life and family. More recently, however, research everywhere tells us that schools do affect and build a child’s overall development. Hence, we now see observable patterns between effective schools and education policies all over the world that are being directionally informed through these patterns.
There are currently three types of school effectiveness research – School Effects Research (to analyze the relation between input and output measures of existing schools to study their effects), Effective Schools Research (to study effective schools and the nature of their effectiveness), and School Improvement Research (studying processes to improve existing schools).
All of us at some point have complained about the education system not doing its job of ensuring an engaging, fear-free environment, devoid of the pressures of competition, away from the shackles of bureaucracy, and one that pushes children to reach their full potential. We have innumerable stories of systemic failures and many reasons to blame this complex system which more often than not feels like it is resistant to any change. Even though literacy may have increased post-independence, the Indian education system has to grapple with essential questions like “Do students even like coming to school?” and “Is the classroom a place that connects them to life?” We all know the answer to these questions in most cases is a resounding no. However, we also have many schools and experiments in India as part of a growing alternative schooling movement that is trying to change this. They focus on ensuring that the child (even the most disadvantaged) relishes coming to school and not just cope with it.
Montessori’s work with orphaned and handicapped children in Italy was directly able to influence the rural program of the Centre for Working Children in Karnataka and is currently a preferred methodology in Early Childhood Care and Education (ECCE) all over the country. Pragat Shikshan Mandal (PSS) in Maharashtra adopted Sylvia Ashton-Warner’s reading methodology outlined in her classic book Teacher, which describes her trial and error to teach disadvantaged Maori students in New Zealand, to improve reading outcomes in government schools. Many others set up alternative schools or methods in different parts of the country and introduced radical ways of teaching, sometimes within existing schools. Jiddu Krishnamurthy and Sri Aurobindo were pioneers in this area. More recently, we have witnessed the work of alternate education in India be it the Hoshangabad Science Teaching Program in Madhya Pradesh, David Horsburgh’s Neel Bagh and Vikasana inspired from that in Bangalore, Digantar in Rajasthan, or the Jagriti School in Delhi. These are just some of the numerous experiments in alternate education in India that have shown us what is conceivable and have pushed the system to change, even if it is slight. Effective schools research is a type of school effectiveness research that shows us what is conceivable despite the system. Mainstream education research could learn from these schools as they tell us what is possible and how those possibilities may, if not completely, at least partially be incorporated into our system.
Even so, one question remains – How can we produce better schools for all students? We must be equally invested in not just describing great schools but also monitoring, evaluating, and improving existing ones so that we can create new effective ones. That is why school effects research and school improvement research are so essential. As already mentioned, there has been a considerable argument, and rightly so, over school effectiveness research only measuring output data such as test scores or outcomes like the pass percentage of grade 10 students. However, schools are a dynamic ground of knowledge-sharing, community-building, and conscience-provoking spaces that involve multiple stakeholders and their teaching and learning practices.
If we have to build effective schools for all, it only makes sense to monitor and measure the input provided by various stakeholders in building student lives and the impact of those efforts. Research in this area suggests that strong instructional leadership from the principal, strong focus on structured, contextual, and well-defined curricular instruction, a learning environment that is safe and promotes student well-being, having teachers who stress high expectations from ALL their students, are essential variables to monitor on an ongoing basis if we are to evaluate their correlation and causation with short and long-term school effectiveness outcomes. This must also include measuring student achievement test scores to continually monitor the effectiveness of these practices. If we are to assess all these factors, we must ensure that all stakeholders in the school community are involved in the measuring and evaluative process. School improvement and school effects research are a combination of input and output measurement as well as an analysis of the various systems and processes in a school that are assessed and correlated with other effective school variables.
Input and output variables include the impact of human and physical resources on the student outcomes. In the initial years, the inputs mostly included economic variables such as per pupil expenditure, school facilities, or the number of books in the library. Over time, with increasing education research, other input variables that have begun to be assessed include but are not limited to the students’ sense of control of their environment, the quality of teachers’ education, teachers’ high expectations for their students, perceptions of teachers, students, and principals, school climate, curriculum planning, and effects of the school board. This is done so that we can look at student development in a holistic manner and not just in terms of test scores. We can assess students in terms of their knowledge, skills, behaviour, attitudes and also look at long-term outcomes.
The impact can be measured as an effect on a student or an effect of a particular policy or practice. Outcomes such as scholastic attainment, classroom behaviour, absenteeism, attitudes toward learning, continuation in education, employment, and overall social functioning are essential to the development of an individual. Learning, and the love of learning; personal development and self-esteem; life skills, problem-solving, and learning how to learn; the development of independent thinkers and well-rounded, confident individuals, all rank as highly as variables to measure as opposed to a narrow outcome of academic achievement. There have been some studies conducted in this area that have focused on evaluating the impact of socio-emotional-factors such as attitude to learning or a child’s academic self-concept. The outcomes of these studies are inconclusive. However, there is no doubt that there is a correlation between socio-emotional development of a child and their learning outcomes. That is why, very recently, even the 2015 edition of PISA integrated a new instrument that will evaluate students on five socio-emotional domains such as psychological, cognitive, social, physical, and material. The results of the past year have been positive. The results of the cognitive dimension (that measured enjoyment of science, self-efficacy, instrumental motivation as well as test anxiety) showed a positive correlation with student performance across countries.
Another important aspect of measuring school effectiveness is to assess the systems and processes in the school. Processes related to classroom instruction, assessment, overall curriculum, planning at a school and teacher level, teaching and non-teaching staff-related processes, data collection and monitoring processes, community engagement, and partner-related processes are just some of the important processes which when conducted in a smooth, well thought out and orderly manner, will make a school effective. For the school to be effective, these processes should be planned well and evaluated and improved continuously.
A school co-ordinator working in a private school in Bombay that I spoke to, seconded this. She mentioned how the school that she works in evaluates the consistency of these processes and also seeks feedback on the effectiveness of these processes from the various stakeholders involved in that process. For e.g., it was decided to put in place a new process to promote student leadership and student voice. Students were encouraged to give suggestions through a suggestion box installed in each class and they led the way in reading suggestions and discussing the possible solutions. Before, during, and after this process was introduced, feedback was taken from the teachers and a few students to evaluate how the process could be made better. This was done during the regular staff meetings. The outcomes of such practices may include small changes in the processes conducted in the school or big changes which can lead to long-term outcomes.
Many schools monitor these inputs and outputs or process evaluations by creating a mixed stakeholder group including parents, teachers, administrative staff, students, and school leadership to conduct internal school audits. The mixed stakeholder groups are expected to go on school walks, execute classroom observations, conduct interviews with various stakeholders, and go through documentation including student work. The aim is to gather triangulated evidence to get an overall picture of school effectiveness.
Some schools also collaborate with other comparable schools to conduct similar school audits to provide a more objective view of their school’s effectiveness. I spoke to one such mixed stakeholder group at a private school serving underserved communities in Pune. All of them unanimously believed in the process being essential to understanding the processes and systems at school and eventually improving these to make the school more effective. One of the parent members said, “I am very amazed that the school is allowing transparency in this way. When we go for such a school walk, we know what is happening in every corner of the school.” This transparency is what makes the process effective. For this transparent process to lead to an equitable system, there needs to be clear communication throughout the process. Shared leadership is the foundation of this process, and all stakeholders must be involved in essential decision-making. There must be a willingness of all stakeholders to learn and ensure that they understand that educating children and effective schooling are a collective responsibility. Each stakeholder can act as a ‘critical friend’ to the other and ensure that they come with an attitude of questioning with empathy rather than inspecting.
Some schools also participate in a self-evaluative and reflective process. The Australian Education Council (AEO) started a Good School Strategy. Part of it was the Effective Schools Project in 1991 conducted by the Australian Council for Educational Research (ACER). More than 2300 Australian schools responded to a questionnaire on various aspects of school effectiveness. Things that made schools effective included staff, ethos, curriculum, and resources. The report also stressed that accountability of schools should be a local issue rather than centralized at a national or state level. The report commented on moving the focus of school effectiveness on maximizing academic achievement and instead shifting to a paradigm that supports students to build the love and pursuit of learning. The report stressed that personal development, problem-solving, life skills, learning how to learn, and self-esteem to develop independent thinkers and confident adults are what school effectiveness research should focus on.
The purpose of all school effectiveness work is to ensure equity of outcomes at all levels for all students. An equitable system serves all its stakeholders. The system ensures belief in the self-actualization of all students if they are given the right support at the right time. For that, the system ensures high expectations through high-performance standards. It has monitoring and evaluation processes in place to ensure the disaggregation of data to determine systemic and individual needs. This in turn ensures differentiation in instruction. An equitable system ensures access and inclusion and capitalizes on the diversity of stakeholders. It ensures a safe and positive environment for learning. It safeguards the involvement of the community and makes sure it connects learning to real-life experiences. Such a system is possible only if accountability can be built into the system at various levels and systems and processes are consistent. It is important to note that measuring school effectiveness just like measuring student achievement must be based on a system of collaboration rather than competition. Improving school effectiveness is a societal goal that all of us must pay attention to and in practice, must be willing to move away from the static outcome based process to a more holistic, journey based process.
Note: I spoke to two schools – the school in Pune is a K-10 state board affiliated school serving underserved communities. The school in Bombay is a K-10 IB board school.
References
• https://hub.jhu.edu/magazine/2016/winter/coleman-report-public-education/
• Handbook of School Research.pdf
• https://www.asercentre.org/p/134.html
• https://www.arvindguptatoys.com/arvindgupta/alternative-schooling.pdf
• https://www.frontiersin.org/articles/10.3389/fpsyg.2020.00431/full#B62
The author holds a Masters in psychology and has been an educator for the last nine years. She is currently working as Assistant School Leader at a school in Pune. When she is not teaching or writing, she loves reading, cooking, and long-distance running. She can be reached at nupurhukmani12@gmail.com.