Starting next year, students from Kindergarten to Grade 12 in Canada’s largest province, Ontario, will be bringing home report cards that showcase six “transferable skills”: critical thinking, creativity, self-directed learning, collaboration, communication, and citizenship. It’s the latest example of the growing influence of education policy organizations, consultants and researchers promoting “broader measures of success” formerly known as “non-cognitive” domains of learning.
In announcing the latest provincial report card initiative in September 2017, Education Minister Mitzie Hunter sought to change the channel in the midst of a public outcry over continuing declines in province-wide testing results, particularly in Grade 3 and 6 mathematics. While Minister Hunter assured concerned parents that standardized testing was not threatened with elimination, she attempted to cast the whole reform as a move toward “measuring those things that really matter to how kids learn and how they apply that learning to the real world, after school.”
Her choice of words had a most familiar ring because it echoed the core message promoted assiduously since 2013 by Ontario’s most influential education lobby group, People for Education, and professionally-packaged in its well-funded ‘Measuring What Matters‘ assessment reform initiative. In this respect, it’s remarkably similar in its focus to the Boston-based organization Transforming Education. Never a supporter of Ontario’s highly-regarded provincial testing system, managed by the Education Quality and Accountability Office (EQAO), the Toronto-based group led by parent activist Annie Kidder has spent much of the past five years seeking to construct an alternative model that, in the usual P4E progressive education lexicon, “moves beyond the 3R’s.”
Kidder and her People for Education organization have always been explicit about their intentions and goals. The proposed framework for broader success appeared, almost fully formed, in its first 2013 policy paper. After referring, in passing, to the focus of policy-makers on “evidence-based decision making,” the project summary disputed the primacy of “narrow goals” such as “literacy and numeracy” and argued for the construction of (note the choice of words) a “broader set of goals” that would be “measurable so students, parents, educators, and the public can see how Canada is making progress” in education.
Five proposed “dimensions of learning” were proposed, in advance of any research being undertaken to confirm their validity or recognition that certain competing dimensions had been ruled out, including resilience and its attendant personal qualities “grit’/conscientiousness, character, and “growth mindset.” Those five dimensions, physical and mental health, social-emotional development, creativity and innovation, and school climate, reflected the socially-progressive orientation of People for Education rather than any evidence-based analysis of student assessment policy and practice.
Two years into the project, the Measuring What Matters (MWM) student success framework had hardened into what began to sound, more and more, like a ‘new catechism.’ The Research Director, Dr. David Hagen Cameron, a PhD in Education from the University of London, hired from the Ontario Ministry of Education, began to focus on how to implement the model with what he termed “MWM change theory.” His mandate was crystal clear – to take the theory and transform it into Ontario school practice in four years, then take it national in 2017-18. Five friendly education researchers were recruited to write papers making the case for including each of the domains, some 78 educators were appointed to advisory committees, and the proposed measures were “field-tested” in 26 different public and Catholic separate schools (20 elementary, 6 secondary), representing a cross-section of urban and rural Ontario.
As an educational sociologist who cut his research teeth studying the British New Labour educational “interventionist machine,” Dr. Cameron was acutely aware that educational initiatives usually flounder because of poorly executed implementation. Much of his focus, in project briefings and academic papers from 2014 onward was on how to “find congruence” between MWM priorities and Ministry mandates and how to tackle the tricky business of winning the concurrence of teachers, and particularly in overcoming their instinctive resistance to district “education consultants” who arrive promising support but end up extending more “institutional control over teachers in their classrooms.”
Stumbling blocks emerged when the MWM theory met up with the everyday reality of teaching and learning in the schools. Translating the proposed SEL domains into “a set of student competencies” and ensuring “supportive conditions” posed immediate difficulties. The MWM reform promoters came four square up against achieving “system coherence” with the existing EQAO assessment system and the challenge of bridging gaps between the system and local levels. Dr. Cameron and his MWM team were unable to effectively answer questions voicing concerns about increased teacher workload, the misuse of collected data, the mandate creep of schools, and the public’s desire for simple, easy to understand reports.
Three years into the project, the research base supporting the whole venture began to erode, as more critical independent academic studies appeared questioning the efficacy of assessing Social and Emotional Learning traits or attributes. Dr. Angela L. Duckworth, the University of Pennsylvania psychologist who championed SEL and introduced “grit” into the educational lexicon, produced a comprehensive 2015 research paper with University of Texas scholar David Scott Yeager that raised significant concerns about the wisdom of proceeding, without effective measures, to assess “personal qualities” other than cognitive ability for educational purposes.
Coming from the leading SEL researcher and author of the best-selling book, GRIT, the Duckworth and Yeager research report in Education Researcher, dealt a blow to all state and provincial initiatives attempting to implement SEL measures of assessment. While Duckworth and Yeager held that personal attributes can be powerful predictors of academic, social and physical “well-being,” they claimed “not that everything that counts can be counted or that that everything that can be counted counts.” The two prominent SEL researchers warned that it was premature to proceed with such school system accountability systems. “Our working title, ” she later revealed, “was all measures suck, and they all suck in their own way.”
The Duckworth-Yeager report provided the most in-depth analysis (to date) of the challenges and pitfalls involved in advancing a project like Ontario’s Measuring What Works. Assessing for cognitive knowledge was long-established and had proven reasonably reliable in measuring academic achievement, they pointed out, but constructing alternative measures remained in its infancy. They not only identified a number of serious limitations of Student Self-Report and Teacher Questionnaires and Performance Tasks (Table 1), but also provided a prescription for fixing what was wrong with system-wide implementation plans (Table 2).
Duckworth went public with her concerns in February of 2016. She revealed to The New York Times that she had resigned from a California advisory board fronting a SEL initiative spearheaded by the California Office to Reform Education (CORE), and no longer supported using such tests to evaluate school performance. University of Chicago researcher Camille A. Farrington found Duckworth’s findings credible, stating: “There are so many ways to do this wrong.” The California initiative, while focused on a different set of measures, including student attendance and expulsions, had much in common philosophically with the Ontario venture.
The wisdom of proceeding to adopt SEL system-wide and to recast student assessment in that mold remains contentious. Anya Kamenetz‘s recent National Public Radio commentary(August 16, 2017) explained, in some detail, why SEL is problematic because, so far, it’s proven impossible to assess what has yet to be properly defined as student outcomes. It would also seem unwise to overlook Carol Dweck’s recently expressed concerns about using her “Growth Mindset” research for other purposes, such as proposing a system-wide SEL assessment plan.
The Ontario Measuring What Matters initiative, undeterred by such research findings, continues to plow full steam ahead. The five “dimensions of learning” have now morphed into five “domains and competencies” making no reference whatsoever to the place of the cognitive domain in the overall scheme. It’s a classic example of three phenomena which bedevil contemporary education policy-making: tautology, bias confirmation and the sunk cost trap. Repeatedly affirming a concept in theory (as logically irrefutable truth) without much supporting research evidence, gathering evidence to support preconceived criteria and plans, and proceeding because its too late to take a pause, or turn back, may not be the best guarantor of long-term success in implementing a system-wide reform agenda.
The whole Ontario Measuring What Works student assessment initiative raises far more questions than it answers. Here are a few pointed questions to get the discussion started and spark some re-thinking.
On the Research Base: Does the whole MWM plan pass the research sniff test? Where does the cognitive domain and the acquisition of knowledge fit in the MWM scheme? If the venture focuses on Social and Emotional Learning(SEL), whatever happened to the whole student resilience domain, including grit, character and growth mindset? Is it sound to construct a theory and then commission studies to confirm your choice of SEL domains and competencies?
On Implementation: Will introducing the new Social Learning criteria on Ontario student reports do any real harm? Is it feasible to introduce the full MWM plan on top of the current testing regime without imposing totally unreasonable additional burdens on classroom teachers? Since the best practice research supports a rather costly “multivariate, multi-instrumental approach,” is any of this affordable or sustainable outside of education jurisdictions with significant and expandable capacity to fund such initiatives?
Perhaps work from Harvard’s Project Zero on Visible thinking both on John Hattie’s making learning visible and on seeing the IMPACT a teacher has in his/her classroom offers something for teachers.
Duckworth has argued that “measuring” the impact of SEL is done at the individual teacher and classroom level. While it is becoming increasingly clear that SEL “works”, precise measurements are still far away. Rather like the relationship between diets and health. We see differences though they are hard to explain precisely.
The history of assessment since the 1860s in the UK has demonstrated the difficulty of doing measurement big scale even for low-level factual accumulation.
These “transferable skills” are simply the “competencies” or soft skills, or social emotional learning (SEL) efforts that are being inserted into many schools, generally without much information or consent by parents. This is what I just posted into Facebook:
Parents As Displaced Persons In Schools
The SEL shove is part of a massive, orchestrated move by well-funded groups to even more profoundly displace parents from their children’s schooling. Social Emotional Learning, under many guises, basically injects competencies (all those “Cs”, collaboration, etc., and diversity, equity, etc., etc.) instead of academic pursuits.
Parents, who on the whole, still want the traditional goals to be accomplished, are being overwhelmed and overridden . Social justice and collective behavior are emphasized instead of individual accomplishments. Fortunately, there is a group of legal scholars (Justice Centre for Constitutional Freedoms) interested in supporting parent rights. Both Alberta and Ontario sex education curriculums are questioned.
John Carpay of the Justice Centre put it best in a recent video message on Parental Rights in Education: https://www.youtube.com/watch?v=j-UVUreq0fc
Parents want education in conformity with their own convictions, not “the convictions of politicians, bureaucrats, political activists.”
“So, today’s activists want to indoctrinate kids into the state ideology of these ambiguous and dangerous slogans of equality, diversity, tolerance, inclusion and so on. And today’s social warriors will not rest as long as some parents are teaching their own children . . . so the ideologues are out to sexualize our children as early as possible . . . any law which puts parents as the enemy, any law which drives a wedge between parents and children, is in its very nature a totalitarian law, contrary to the UN Declaration of Human Rights, the Covenant on Political and Civil Rights and contrary to the free society.”
Since when are equality and tolerance bad ideas? Since when are learning self-discipline and self-assessing one’s strengths and weaknesses not desirable for all of us? Since when is a collaboration with others on complex tasks an unworthy goal? These are attributes of evolving humans and help explain our success on the planet.
You certainly see this in black and white terms,Tunya. I’m more inclined, as you know, to see it differently. Society is evolving and so are societal values, some for the better and others problematic. Defenders of the Measuring What Matters initiative are socially progressive, but not really a threat to family values (which are evolving to become more tolerant and inclusive).
Promoting values in school really amounts to — whose values? Going back to the origins of the Ontario school system under Egerton Ryerson, schooling was shot through with a clear set of values – rooted in Glasgow Free Church beliefs and yet few today recognize or see that influence. In other words, public education since its inception conveyed and imparted social values, most visible in the Protestant work ethic.
I don’t see a conspiracy here, but rather a transformation because of changes in majoritarian values. My guess is that the advocates of MWM would not get any of this and simply see it as the natural course of events.
Well said, Paul. States (Marx, Fourier, Bakunin notwithstanding) are not going to wither away.
I see firsthand indoctrination of kids on a regular basis. Many truths are revealed by Tunya’s comment. My youngest child was horrified when her female teacher proclaimed that girls had a much tougher go based on the prejudiced views that males had against them My child thought that was being incredibly sexist, coming from a teacher. Another lesson began with the teacher proclaiming that our Canadian forefathers, and current day political leaders were just a bunch of old white guys who impressed their prejudiced views on today’s, and past citizens of Canada. A more enlightened approach may have just been to teach historical facts and content so that a lively discussion may have ensued, rather than blatantly proclaiming her own biases in a classroom setting, full of impressionable young minds.
The premise of social justice is just that…a premise. The reality is much different. We now have scholars proclaiming that Algebra is racist and shouldn’t be taught…even though the exact opposite has been proven to be true. Be very careful how this topic is packaged when dealing with impressionable young minds. If we didn’t see a declining trend of academic performance perhaps more parents might be more accepting of these initiatives. We don’t need social justice taught in math class…what we need is math to be taught properly to ensure social justice is obtained for all students.
The Ontario People for Education initiative is much more nuanced than what might be described as naked “indoctrination.” You are focusing on values, while my commentary raises questions about “which values”? Teaching mathematics with direct instruction is, in fact, a statement of values. All teaching, I have found, conveys value messages, even a traditional math classroom.
I only wish that you had engaged in debating the issue that I raised or attempted to raise. Isn’t it time for us to confront the implicit as well as explicit values being imparted in classrooms. Suggesting that we stick to teaching facts, in my view, may weaken the case for raising standards, developing resilience, and teaching responsibility in the classroom. Those are values, so why would we rule it out going forward?
Once again, Paul, you raise the valid point of the trend.
One advantage of direct instruction for teaching math is that it has strong evidence to back it.
There is increasing evidence of the value of teaching responsibility and resilience to students who otherwise struggle.
I do see major problems with the research base, cost, teacher – buy in, and teacher work-load; this sounds like one more top-down initiative.
However, my major concerns are the reactions and impact on more conservative or less affluent sectors. I am not sure if these parts of the population will see eye to eye with Annie Kidder or will be well served.
My major concern is how measurement of SEL will cope/conflict with the student populatiion that has legitimate, diagnosed or undiagnosed mental health problems. I have concerns about lack of knowledge re more serious mental health issues, the role teachers are increasingly expected to play and of course major gaps in the mental health services for children.
Great to see this provocation generating some insightful comments. Setting aside the philosophical differences, is MWM even implementable? Then the big one— is it cost prohibitive everywhere but in Kathleen Wynne’s Ontario?
My background research led me to read Dr David Hagen Cameron’s previous and concurrent studies. He’s an educational sociologist not a quantitative research specialist with no previous work on student assessment. His specialty: micro politics of schooling and overcoming resistance to consultants implementing policy mandates in the UK.
No, Ontario does not have the money. We have teachers and EAs wearing Kevlar to school as well as young children witnessing acts of violence in the classroom. Actually in many instances, these same children are being hurt themselves. This is a priority.
Not that I would want to show my bias, but I have been envisaging the scene when an expert educates the teachers on the need for teachers to teach the students to be responsible and develop resilience.
Finally, we must stop excluding the expertise of veteran and skilled teachers. Sure they may be cranky, a bit jaded and prone to saying ‘the wrong thing’, but they are weary of the exclusion. Until they are at the podium in large numbers, we will make little real progress. Generally they have the techniques down pat.
For my wanderings off topic, I beg forgiveness.
Everything People for Education does, including MWM, is predicated on the assumption that investing more in education is the answer to most problems. It’s been that way since its inception in the mind of Annie Kidder and a small group of Toronto parents.
Teacher knowledge and acquired expertise needs to be valued — and research, as you know, tends to support what regular teachers learn through their everyday experience. Not always, but to a large degree.
To a large degree, yes. Every presenter who speaks should have a teacher partner. When there is disagreement, that can be carefully explored. I agree with your thoughts.
What will harm education is the short term political goals that hamper long term educational improvement. The Ontario initative may be a non-starter with a provincial election looming. A new government will develop a new set of initiatives, especially from a party not in power for over 15 years.
Displaced Parents and Schools — Counting the Ways, Part I
Parents have legal rights and duties regarding their individual children’s development and education yet many feel squeezed out of school decision-making that affects them and their children. Outspoken critics of this squeeze-play might even be regarded as holding conspiracy theories! Here is some history:
1. The book, “Parents and Schools”, Wm W Cutler shows 150 years (to yr 2000) of the continuing struggle between parents, parent groups and public school systems. Bottom-line — “The school and the home need not be in conflict so long as the former controlled the latter, not the other way around.”(p83)
2. S Sarason’s book, “Parental Involvement and the Political Principle: Why the existing governance structure of schools should be abolished”, 1995, goes beyond the biological and legal rights of parents. There is the political democratic principle that Western nations espouse — “when you are going to be affected, directly or indirectly, by a decision, you should stand in some relationship to the decision-making process.” (p7) More parents want a say in their child’s individual growth outside the home.
3. Why is there a need for extraordinary rules for parent involvement in schools? The UN Declaration of Human Rights (1948) and the International Covenant on Civic and Political Rights (1976) assert parental primacy for education choice and support their right to expect such education to be in conformity with their own religious and moral convictions. John Carpay’s video on Parent Rights lays out the WHY — “because ideologues, throughout history, have always sought the hearts and minds of children . . . to indoctrinate (them) into the ‘correct’ ideas.” He mentions Hitler, Lenin, Stalin, Mao. He warns that anything that drives a “wedge” between parents and children is contrary to a “free society”. Informed parental consent is an important issue today. https://www.youtube.com/watch?v=j-UVUreq0fc
Displaced Parents and Schools — Counting the Ways, Part II
4. Nat Hentoff, well-known education and political writer (recently deceased), is known for his book, Does Anybody Give a Damn? (1977) about education neglect in NY City. He should be better known for his article “The Greatest Consumer Fraud Of All” (Social Policy, Nov/Dec77). After a scorching list of education transgressions against parents and students, he said, “Ultimately . . . the basic responsibility for keeping education professionals honest and competent will be that of parents.”
5. What can parents do to make public education more responsive and meeting expectations? I have a list of over two dozen books, all saying What You Can Do. Remember Flesch’s book, Why Johnny Can’t Read and what you can do about it, 1955? Well, that dubious reading program, Whole-Language, which was seen as the culprit is still causing countless reading problems, and phonics, a proven method, continues to be disparaged. Attendees at the upcoming researchED program in Toronto, Nov 10-11, will have a chance to talk to two authors and ask if THEIR books have made a difference: Michael Zwaagstra, What’s Wrong with Our Schools and How We Can Fix Them, 2010, and Tom Bennett, Teacher Proof: Why research in education doesn’t always mean what it claims, and what you can do about it, 2013.
6. But what do parents want? Can we presume there are some common wishes? Yes, there is a list, Effective Schools Checklist (1978), which would serve most parent purposes — IF it hadn’t been “disappeared”, presumably because the education establishment feared the accountability implications. Can you beat this? Leadership, Mission Statement, Orderly Environment, High Expectations, Basic Skills Mastered, Monitor results of student-teacher-school goals, Meaningful Parent Involvement, Avoidance of Pitfalls. The last item speaks directly to the researchED mission: Up-to-date awareness of good educational practice plus retaining currency in the field concerning promising and discredited practices.
7. Meanwhile, as my list of displacement of parents keeps growing, highly-funded, dubious, untested, 21st C, coercive, stealthy, unmeasurable, political, undemocratic, social-emotional, interventionist, global transformations of school systems are gaining momentum and fervency, behind our backs! Where are the 3Rs in all this?
You certainly took up my challenge, Tunya.. You seem to have cited a whole library of research sources. We have thirty speakers at researchED Toronto and many of them would support your claims with regard to what the largest body of research shows works for teachers and students. Many come at it from differing ideological persuasions, yet find common ground on “what works” in the classroom. .
Many Happy Eurekas !
I’m hoping that teachers attending researchED in Toronto gain great respect for skepticism regarding their profession. It’s a wonder in this day and age that pedagogy has become so porous as to allow so much that is questionable into practice. With hundreds (maybe 1,000s) of books about education reform flooding the market with little to show it’s probably groups like this that might finally make a difference in public confidence.