The Folly of “Student Success”

Report card

“I want to adopt a policy on school success. I hope we’ll be able to have consultations this fall to identify the ways we can improve our graduation rate,”

—Sébastien Proulx, Quebec Minister of Education.

It really is no wonder we struggle in North America, and in a few other places, to reform or fine-tune our public education systems. There are two principal reasons for our perpetual failure. The first is a matter of bad science.

Good scientists prefer proofs based on controlled experimentation. They expect that some treatment will be applied to one randomly selected group of subjects while not being applied to another such group. The treatment should be double-blind, meaning that no one involved in the execution of the study knows which group is which. The results of the treatment should be measured in the same way for each group, using a reliable measuring instrument. Only when the two results are significantly different can we conclude that the treatment brought about some important change.

Every teacher is applying a treatment to her students. The problem is that it is the same teacher who then measures the resulting performance! When we use grades to evaluate success we are using a measure which no self-respecting scientist would accept.

Furthermore, we make the same mistake with derivatives of grades, pass rates for example. The number of students passing is simply a grouping of grade results.

This may sound like an appeal for standardized testing. Never!

Standardized testing is the other extreme. A standardized test undoubtedly measures something but the obvious problem is knowing what. Does it measure achievement in whatever the teachers set out to teach or the students tried to learn? Most classroom situations are microcosmic to the point where an elsewhere-devised test will not fairly measure what students were actually striving to learn.

And this points to the most fundamental impediment to educational advancement. We refuse to define what education is or to believe that we possibly could. The problem from the get-go is the popular belief that education is about “the transfer of knowledge (or worse, information)” from the full vessel to the empty ones. Real teachers understand this error. They know that the teacher’s task is to be a designer of learning activities. They also know that good teachers experiment, always looking for the best way. This freedom to experiment can cause havoc with any form of fixed testing because sometimes the experiments don’t work. In addition, some strategies may take longer to pay off or may introduce valuable side paths worth exploring.

We have a natural tendency to ensure accountability by measuring outcomes. Maybe we would have better success if we measured inputs. Why not measure the extent to which the necessary inputs are in place to make optimal learning possible? The relevant inputs are numerous and just defining them could lead to valuable insights. They would include an appropriate level of pedagogical expertise on the part of the teacher, and let’s throw in a minimal level of empathy, suited to the learning level. Facilities count: buildings, classrooms, teaching tools. Often support services are necessary, such as counseling, tutoring, special needs support. Maybe family involvement plays an important role and needs to be developed. Whatever the inputs are deemed to be they can all be easily measured. They are either there or they are not; they can either be improved or not. By maximizing the inputs we maximize the outputs. Even those who try to measure outputs end up coming back to the inputs to effect change, unless of course they have allowed themselves to be lulled into complacency by meaningless figures.

Finally, the focus on learning, the kernel process in public education, leaves out of the picture other expectations which are of no less importance. Funders of public education expect more than knowledge accumulation. They inevitably want students to become employable and, in some measure, good citizens, choosing healthy life styles. Sometimes the list is longer. They want graduates to be critical thinkers, creative thinkers, team players, good communicators, and problem-solvers. They may also want students to be strong in Science, Technology, Engineering, and Math (so-called STEM).

Whatever the list, it makes sense to know what’s on it. Any attempt to measure success needs to take into account all of the goals that constitute success for a public education system. Why is it so difficult to write the list? If we would undertake the task we would find ourselves less susceptible to the clichés and empty political promises of those who talk in vague terms like “student success”, “graduation rates”, and “retention”. They cannot deliver on these. As long as we believe them we are wasting our time and our money.