Pandemic Achievement Slide: Is the Learning Loss worse than has been reported?

School’s out and the first reliable reports on pandemic learning loss are appearing in the United States and, far more slowly, from province-to-province across Canada. In some school systems, education leaders and regional superintendents are breathing a sigh of relief and far too many are acting like the disruptions of two-and-a-half years of pandemic learning are over. But the first wave of student assessment scores reveals many students — especially from kindergarten to Grade 6, but all the way to Grade 12 — are behind with school closures, remote learning, and irregular school schedules to blame.

During the COVID-19 pandemic America’s schoolchildren lost out on from 16 to 70 weeks in the classroom. Most pupils received some form of virtual schooling which varied greatly in quality and quantity. While many parents recognized the risk to health posed by keeping schools open, they—and teachers—were concerned that lessons taken at the kitchen table were less effective than those in a classroom. Weathering one wave after another of the pandemic, and particularly Omicron, led to repeated schedule disruptions and reversions to remote/home learning. Early student test results show just how much childrens’ education has suffered during the pandemic.

            Standardized student assessment tracking in the U.S. was far more extensive during the pandemic and the Brookings Institution has reported lower levels of achievement, with younger children hit the hardest. Graduation rates dropped and fewer kids were pursuing post-secondary studies. It’s doubly difficult to identify and assess learning loss in Canada because our education authorities simply suspended provincial testing and, in many cases, final examinations.

Wilfrid Laurier University professor and researcher Kelly Gallagher-Mackay pinpointed the nub of the problem in Ontario and elsewhere: “we don’t have public data on how Ontario students are doing, so we are a lot more in the dark.” That’s problematic because “the risk with educational issues is that they can multiply if they’re not addressed,” she told The Toronto Star. It also has compounded effects: if students’ confidence or sense of preparedness have taken a hit, they may be more inclined to opt for programs they feel are easier, rather than more challenging ones that down the line provide more post-secondary opportunities.

Canada’s largest school district, Toronto District School Board (TDSB), produced Grade 1 Reading data that raised some alarms. TDSB data from 2020-21 for in-person schooling compared with 2018-19, reported students were 3 percentage points behind, while those in virtual schooling were 9 percentage points behind. The board is tracking student well-being and achievement, as part of its COVID-19 Pandemic Recovery Plan, to identify groups most impacted and where interventions are needed

An authoritative November 2021 American study of pandemic education impact, produced by Clare Halloran and a research team for the National Bureau of Educational Research, demonstrated how the shift in schooling mode to home learning adversely affected test scores tracked over 2020-21 across 12 different U.S. states. Student pass rates declined compared to prior years and that these declines were larger in districts with less in-person instruction. Passing rates in math declined by 14.2 percentage points on average, but somewhat less (10.1 percentage points smaller) for districts fully in-person. Reported losses in English language arts scores were smaller, but were significantly larger in districts with larger populations of disadvantaged students who were Black, Hispanic or eligible for free and reduced-price lunch programs.

Studies in Britain also show that the longer kids were in remote learning, the worse they fared. That’s particularly worrying in Canadian provinces like Ontario, where students lost out on about 27 weeks or more of in-person learning from March 2020 to the end of June 2022. Judging from the June 2021 Ontario Science Table study, Canadian provinces lost more days, averaging about 20 weeks, than similar jurisdictions in the U.S., U.K. or the European Union.

            The Canadian province of Nova Scotia is, as usual, a reliable bell-weather for K-12 education. Province-wide assessment was suspended completely in 2020-21 and then reinstituted in 2021-22.  The latest test results were embargoed until the last week of school in June 2022, posted on an obscure Nova Scotia Education website under PLANS, then released without any notice or comment. Putting them out at the tail end of the year all but guarantees that they escape public notice.

            Studying the latest installment of Nova Scotia provincial student results, covering the 2018-19 to 2021-22 period, it is easy to see why they are buried on an obscure public website.  Nothing was reported covering Grade 3, the critical first step in monitoring the acquisition of student competencies in reading, writing and mathematics. Instead, the province released Grade 6 results showing, as predicted, a pronounced achievement decline, most acute in mathematics and writing, but also affecting reading competencies and comprehension. 

            What are education authorities attempting to hide?  Grade 6 Mathematics results (2021-22) dropped to 64% achieving expectations, down 6 % from before the pandemic. In the case of Grade 6 Reading, some 71% of students met the standard, down 4% since 2018-19. Going back ten years to 2012-13, the achievement slide is actually gradual and continuing, perhaps worsened by some 22 weeks of COVID-related school closures from March 2020 to June of 2021.

From school district to district, student achievement in 2021-22 was also highly irregular, ranging in Grade 6 Mathematics from Halifax RCE (67%, down 6%) to TriCounty RCE (50%, down 14%). In Grade 6 Reading, the comparable figures were Halifax RCE (74%, down 3%) to TriCounty RCE (61%, down 6 %).

Some marked progress has been made in addressing the problem of underperformance among marginalized and racialized students. In Grade 6 Mathematics, for example, African Nova Scotian students’ scores have risen from 36% (2013-14) to 55% (2016-17) and then held firm at 54% (2019-20) before the pandemic.  For Indigenous students, Grade 6 Reading has risen from 64% (2013-14) to 65% (2016-17) and then reached 74% (2019-20), just 2% below the provincial mean score. 

            The declines in Grade 6 Mathematics and Reading in Nova Scotia post-pandemic are perhaps predictable. What is more concerning is the longer-term trend toward an “achievement slide,’ revealed starkly on publicly- reported provincial assessment results over the past decade. Grade 6 Mathematics scores, for example, have plummeted from 73% (2012-13) to 71% (2018-19) to 64% (2021-22), a drop of 9 points.  In Grade 6 Reading, the slide is gentler from 76% (2012-13) to 74% (2018-19) to 71% (2021-22).  In short, somewhere between one-quarter to one-third of all students are not functionally literate or numerate at the end of elementary school.

One of Canada’s leading international education experts, Paul Cappon, warned ten years ago that Canada was becoming “a school that does not issue report cards.”  Suspending student assessment during the pandemic, then re-instating tests on a limited basis is bad enough.  Holding-off on releasing student results until everyone is on the way out for the summer holidays suggests that Dr. Cappon’s prophecy has come to pass, even after the biggest educational disruption in our lifetime.

What was the full extent of the learning loss experienced by K-12 students over the past two-and-a-half years? How reliable are the initial assessments coming out of the United States, the UK, and the European Union states?  Why is it next-to-impossible to assess the pandemic impact on Canadian students?  By limiting student assessment, rationing the results, then issuing partial sets of results are Canadian school authorities cushioning the blow or merely deferring the day of reckoning?  

Read Full Post »

Overhyped Psychological Theory: What does the “Growth Mindset” Controversy Teach Us?

The concept of a “growth mindset” is so wildly popular these days that it has spread into mass culture and creeps into many supposedly cutting-edge leadership development presentations.  Having a “growth mindset” means believing that you can improve your intelligence through effort and the use of effective strategies, whereas having a “fixed mindset” means accepting your limitations. It is now virtually the ‘New Age’ elixir for the ambitious in 21^st century times.

Since the publication of Stanford University psychologist Carol Dweck’s 2006 book Mindset: The New Psychology of Success, the whole notion gained widespread currency. Her TED Talk attracted 10 million views and the mindset approach spread from elementary and secondary education and was applied in stress and mental health research, in conflict resolution, and in corporate boardrooms. School systems in Canada and around the world began to promote the teaching of growth mindset as a learning technique, and educational companies jumped on the bandwagon, generating sets of mindset materials for teachers and parents.

Millions of dollars went into funding mindset research until the first studies appeared five years ago calling into question the legitimacy of the fashionable psychological theory. Dweck’s claims and those of her research collaborator, David Yeager of the University of Texas at Austin, were challenged in a March 2018 study by Case Western Reserve University researchers. Two meta-analyses, replicating Dweck’s most-cited papers, reported “little or no support for the idea that growth mindsets are beneficial for children’s responses to failure or school attainment.”

Overhyped educational panaceas tend to underdeliver when subjected to evidence-based analysis and mindset theory is no exception. While some mindset-based interventions produced good results, the Case Western Reserve team found others had no effect on student outcomes.  Aside from a few methodological quibbles, the biggest criticism was that mindset research fell well short of its promise.

Schools tend to be fertile ground for the latest psychological theories and learning experiments. From Brain Gym to learning styles, a succession of innovations promoted by curriculum and pedagogical consultants have been implemented by classroom teachers, only to be abandoned or simply disappear when shown to be largely a gimmick rather than a genuine breakthrough.

Unlike most educational ‘fads,’ Dweck’s ‘growth mindset’ did emerge out of some sound initial research into brain plasticity and was tested in actual case studies with students in the schools. University College London education researcher Dylan Wiliam, a renowned student assessment expert, even lent his support to the Growth Mindset movement when he embraced Dweck’s findings, codified the approach as  Talent = Hard Work + Persistence, and applied it to building ‘feedback’ into student assessment.

From 2015 to 2017, Dweck and her research associate Susan Mackie alerted researchers and education policy-makers to the spread of what was termed a “false growth mindset”  in schools and classrooms in Australia as well as in the U.S. and the UK. Too many teachers and parents, they pointed out in an influential 2016 article in The Atlantic, had either misinterpreted or debased the whole concept,.

Dweck discovered that in many classrooms it had been reduced to simple axioms like “Praise the effort, not the child (or the outcome).” In most cases, it was educational progressives, or parents, looking for alternatives to “drilling with standardized tests.” “Growth mindset disciples,” Dweck acknowledged, had reverted to praising students rather than taking “the long and difficult journey” and showing “how hard work, good strategies, and good use of resources lead to better learning.”

Defenders of mindset research now concede that the concept was disseminated far too fast. “Any popular idea in education gets spread way ahead of how ready the science is,” David Yeager told Scientific American in August 2019. Much like Dweck, he acknowledges that growth mindset is far more complex and subject to misinterpretation in schools and misapplication in classrooms.

Yeager, Dweck and members of their Mindset Scholars Network have fought back against the skeptics.  A massive study, based upon a randomized control trial of 12,000 students from across the United States, published in August 2019 in Nature demonstrated that mindset interventions can work in certain contexts. In this case, at the grade 9 level, and with lower-achieving students.  Exposure to two short, low cost online programs led to higher grades for lower-achieving Grade 9 students (an average improvement of 0.1 grade point) and many students chose more challenging math courses in the next grade. While showing positive signs, critics questioned whether, given the investment of resources, a 0.1 point boost was meaningful and whether the claims for such programs are inflated by the marketers.

Growth mindset may not have been debunked but the psychological theory has lost its lustre.  Successful implementation of mindset interventions appear to require finesse in the classroom. The national study showed that it could work with Grade 9 students supplied with study materials designed for that purpose. The latest 2022 research study on “Teacher Mindsets” in Psychological Science identifies where and why growth-mindset interventions do and do not work.  At the risk of oversimplifying, it essentially comes down to this: first year high school students supported by mathematics teachers with more highly-developed growth mindsets perform better. That is, to say the least, hardly earth-shaking.

What’s the litmus test for successful educational interventions? The bar, we now know, is set relatively low.  What is clear: Growth mindsets have proven very hard to instill and harder than its inventors ever imagined. It requires a laser-focused growth mindset to persevere and overcome the next set of obstacles. Even modest effects, Yeager confessed in March of 2018 in Wired Magazine, are “somewhat amazing” given the fact that “many, or even most very extensive and expensive educational programs have no effect at all.”

Why are school change theorists and system leaders so susceptible to the latest panacea?  How did “Growth Mindset” achieve its exalted status in North American K-12 education?  What happened to undercut its legitimacy?  How have lead proponents Carol Dweck and David Yeager responded to shore up support for the theory?  What does the whole controversy over “mindset theory:  teach us?

Read Full Post »