Back in June of 2011, Dirk Van Damme, Head of the OECD’s Innovation and Measuring Progress Division (IMEP), stunned a Toronto gathering of prominent international educators at OISE with a rather harsh assessment of the state of education research. “It’s mostly of low quality,” he said, “and we need to be more hygenic when using the word research.”
While Van Damme recognized that education research was improving, he claimed that much of the “research” lacked credibility because researchers began from “fixed ideological positions” and limited their work to “small scale” projects with limited broader applicability. He warned then, a decade ago, that we were not “preparing students for 21st century challenges.”
The most recent national study, “Children and Schools During COVID-19 and Beyond,” produced for the Royal Society of Canada by University of Ottawa’s Tracy Vaillancourt and a team of researchers, provides us with a rare opportunity to examine the state of the field. Surely, a team of widely-known university researchers could produce evidence of how the massive disruption and school closures have impacted the learning of 5.7 million Canadian students in the “pandemic generation.”
Studying the Royal Society Policy Briefing report does give you a pretty good sense of the current shape and quality of faculty of education-based research. Social and emotional well-being and children’s mental health are the clear priorities of the vast majority of researchers, mostly trained in child psychology and educational sociology. It’s little wonder, then, that the report emphasizes the social and emotional impacts and focuses, to a large extent, on “notable threats to children’s well-being, educational success, and healthy development” in that order.
Vaillancourt and her research team do convey a sense of urgency. “We are on the cusp of a ‘generational catastrophe’ that requires swift action to mitigate the harm,” they claim, and we now need to keep schools open. “Schools must be the first to open and the last to close” is the mantra repeated throughout the report. Why we need them open seems to revolve mostly around their 21st century mandate to ensure educational equity and provide social and emotional security for children. Judging from the report, the COVID-19 crisis may well have reinforced the commitment of researchers to transform schools into “therapeutic institutions” for all children.
What’s strange about the report is the absence of official Canadian data on learning impacts and a call for education gatekeepers to collect and disclose mission-critical data on student achievement, absenteeism, behaviour, and graduation standards. Closing the achievement gap and addressing “learning loss” do not figure in the general policy proposals. Buried among the ten recommendations is this revealing statement: “We need a precise account of who was impacted, how and for how long, so we can take appropriate steps toward providing systems and services that better support them moving forward.”
Lead author Vaillancourt’s cornerstone essay on the COVID-19 impact on children’s mental health, including school closures and social isolation, is original, reliable and evidence-based, and so is Jessica Whitley’s research summary on the impact on vulnerable children. Few would quibble with this assessment: “Many children and youth have experienced disengagement, chronic attendance problems, declines in academic achievement and decreased credit attainment during the pandemic, with the impact far deeper for those already at-risk.” Learning loss, we can infer, is only of real consequence when it applies to struggling students or those from marginalized communities.
One of the nine essay chapters, “Estimates of Student Learning During COVID-19 School Disruptions,” does cut to the heart of the matter. University of Toronto researcher Scott Davies and University of Waterloo professor Janice Aurini confront the problem squarely: “School disruptions over 2020 and 2021 have likely had a significant impact on children’s learning.” We know this from international research documenting significant “learning shortfalls” during March to June 2020 school shutdowns and more recent international studies showing “learning loss” during online instruction in the spring of 2020.
What we do know is worrisome. “Canada lacks high-quality and largescale data that can be used to directly measure any impacts of those disruptions on student achievement,” Davies and Aurini confirm. “Compounding this problem, provinces like Ontario cancelled their planned standardized testing in 2020 and 2021, precluding the possibility of comparing achievement shortly before and after the school closures. Available studies of achievement are limited to single school boards or handfuls of schools, or parent and teacher surveys that can only capture their perceptions of student learning.” (p. 52) With few exceptions, Canadian researchers have also ignored sound research on “the summer slide” which formed the basis for early estimates of COVID-19 school shutdown setbacks.
Forced to rely upon international studies and research data models, Davies and Aurini claim that the spring of 2020 disruptions alone resulted in “enduring 3-month learning shortfalls and gaps growing between the quartiles up to 1.5 years.” “Most Canadian students struggled, as did students elsewhere in the world,” they conclude, “gaining little ground and soon disengaging from schooling partially or fully.” While students resumed more normal patterns of learning during the interrupted 2020-21school year, the problem was compounded when students “reached a threshold of ‘pandemic fatigue’ and grew tired of online learning.” (pp. 59-60).
The long-term impacts of “learning shortfalls” are now surfacing, again documented first by international research (UNESCO 2021 and UNICEF 2021). The only significant Canadian research, so far, focuses on social and emotional impacts, often to the exclusion of academic achievement. Poor mental health obviously adversely affects student achievement, but – as Davies and Aurini, point out – the reverse is true because “students who feel they are not achieving will have their well-being compromised”
False binaries bedevil Canadian education research and are much in evidence throughout to RSC report. The whole idea that prioritizing academic achievement is at odds with priorities for student’s well-being is not really defensible. That faulty assumption was nicely laid to rest in 2020 by University of Cambridge researcher Tania Clarke in a research article exposing “the dangerous discourse of ‘trade-offs in education.” Academic achievement and student well-being are, more often than not, reciprocal and mutually reinforcing. Simply put, doing better academically improves your outlook and sense of well-being.
The COVID-19 school closures have demonstrated the need for better achievement data for guiding evidence-based policy making in Canadian education. That research gap is exemplified, once again, in the RSC report. Some of the RSC chapters, particularly those produced by University of Ottawa professors Andy Hargreaves and Joel Westheimer, attempt to steer public education away from measuring learning and setting clear expectations. Like much of the current school change literature, those essays privilege student-well being over academic achievement, denigrate the term “learning loss,” and seek to limit or eliminate completely system-wide student assessment.
Actual data from parent surveys, school district reports, and quantitative studies suggest a major disconnect between such educational experts and parents and classroom teachers. Surveys conducted by the Alberta Teachers Association (ATA 2020) Canadian Teachers Federation (CTF 2020) demonstrate the depth of parent and teacher concerns over erosions of children’s skills and mental health. After the first phase of COVID-19 shutdowns, parents in the Hamilton-Wentworth Board also expressed a strong desire for more teacher-led synchronous learning activities during regular school hours. The vast majority of parents, when given the choice, still opt for in-person schooling, with the possible exception of those who live in multigenerational households. Summer learning loss recovery programs have, according to Davies and Aurini, proven popular with parents who choose them for their children.
The identified “learning shortfalls” will not go away. Here again Davies and Aurini caution us not to brush the problem aside because the COVID-19 school disruptions may well “trigger a series of negative consequences” in the coming years. Taken together with the solid evidence of adverse mental health impacts, the soundest RSC essays simply cry out for high quality and timely data that can guide educational policy while also speaking to the legitimate concerns of parents, teachers and the public.
What’s missing in the current approach to combatting the “learning shortfalls” and psycho-social impacts of COVID-19 on children, teachers and families? Consistent, reliable, and evidence-based data. More specifically, we need a national educational body to support the ongoing creation of seasonal learning data in which sizeable numbers of students are tested biannually in fundamental literacy and numeracy skills, first in September and then again in June. Such research is already conducted in many jurisdictions around the world. Our leading experts, Davies and Aurini, could not identify a Canadian jurisdiction anywhere in Canada that routinely uses seasonal learning designs to generate the kind of data that can assess interventions aimed at developing students’ well-being and learning. The COVID-19 disruption has made such research more critical than ever. It would provide the kind of data that can assess interventions aimed at developing students’ learning and well-being. Seasonal learning designs that test students at the end and beginning of consecutive school years can also identify the kinds of students that need extra support and the times the year in which they need those supports.
The cumulative impact of school disruptions on proficiency in the foundational literacy and numeracy skills is visible for all to see. Focusing on student well-being in isolation is not the answer and an accumulating body of research, mostly generated outside of Canada, is demonstrating why. Davies and Aurini at least provide some “estimates” as to the likely learning shortfalls. Today’s short-term losses, they claim, “may amplify as children move up grade levels and fall farther behind their peers.” Being better prepared for the possibility of future closures is now a systemic priority and that simply won’t happen without better data that can track the challenges and successes of our students.
The August 2021 RSC study reveals both the strengths and weaknesses of faculty of education research, but all research is handicapped by the paucity of student data. “All policy decisions are made by leaping over the data,” Dr. Bernard Shapiro once said, twenty years ago. Now we know that the critical data is actually missing in K-12 education and it’s time to demand better from the gatekeepers.
Where is the data on the impact of COVID-19 shutdowns and disruptions in Canada on student learning and psycho-social development? What does the Royal Society of Canada report reveal about the preoccupations and implicit biases of education researchers? How many of the RSC research summaries reaffirm school change theories common before the pandemic? Why, in a collection of nine different studies, does only one confront squarely the lack of reliable student performance and well-being data?