Stealth Assessment: Where is Ontario’s Student Well-Being Assessment Heading?

Ontario now aspires to global education leadership in the realm of student evaluation and reporting. The latest Ontario student assessment initiative, A Learning Province, announced in September 2017 and guided by OISE education  professor Dr. Carol Campbell, cast a wide net encompassing classroom assessments, large scale provincial tests, and national/international assessment programs.  That vision for “student-centred assessments” worked from the assumption that future assessments would capture the totality of “students’ experiences — their needs, learning, progress and well-being.”

The sheer scope whole project not only deserves much closer scrutiny, but needs to be carefully assessed for its potential impact on frontline teachers. A pithy statement by British teacher-researcher Daisy Christodoulou in January 2017 is germane to the point: “When government get their hands on anything involving the word ‘assessment’, they want it to be about high stakes monitoring and tracking, not about low-stakes diagnosis.”  In the case of  Ontario, pursuing the datafication of social-emotional-learning and the mining of data to produce personality profiles is clearly taking precedence over the creation of teacher-friendly assessment policy and practices.

One of the reasons Ontario has been recognized as a leading education system is because of its success over the past 20 years in establishing an independent Education Quality and Accountability Office  (EQAO) with an established and professionally-sound provincial testing program in Grades 3, 6, 9 and 10.  Whether you support the EQAO or not, most agree that is has succeeded in establishing reliable benchmark standards for student performance in literacy and mathematics.

The entire focus of Ontario student assessment is now changing. Heavily influenced by the Ontario People for Education Measuring What Matters project, the province is plunging ahead with Social and Emotional Learning (SEL) assessment embracing what Ben Williamson aptly describes as “stealth assessment” – a set of contested personality criteria utilizing SEL ‘datafication’ to measure “student well-being.” Proceeding to integrate SEL into student reports and province-wide assessments is also foolhardy when American experts Angela Duckworth and David Scott Yeager warn that the ‘generic skills’ are ill- defined and possibly unmeasureable.

Social and emotional learning is now at the very core of Ontario’s Achieving Excellence and Equity agenda and it fully embraces “supporting all students” and enabling them to achieve “a positive sense of well-being – the sense of self, identity, and belonging in the world that will help them to learn, grow and thrive.” The Ontario model is based upon a psycho-social theory that “well-being” has “four interconnected elements” critical to student development, with self/spirit at the centre. Promoting student well-being is about fostering learning environments exhibiting these elements:

Cognitive: Development of abilities and skills such as critical thinking, problem solving, creativity, and the ability to be flexible and innovative.

Emotional: Learning about experiencing emotions, and understanding how to recognize, manage, and cope with them.

Social: Development of self-awareness, including the sense of belonging, collaboration, relationships with others, and communication skills.

Physical: Development of the body, impacted by physical activity, sleep patterns, healthy eating, and healthy life choices.

Self/Spirit:  Recognizing the core of identity whieh has “different meanings for different people, and can include cultural heritage, language, community, religion or a broader spirituality.”

Ontario’s new student report cards, proposed for 2018-19 implementation, will incorporate an distinct SEL component with teacher evaluations on a set of “transferable skills” shifting the focus from organization and work habits to “well-being” and associated values, while retaining grades or marks for individual classes. The Ontario Education “Big Six” Transferable Skills are: critical thinking, innovation and creativity, self-directed learning, collaboration, communication, and citizenship.  Curiously absent from the Ontario list of preferred skills are those commonly found in American variations on the formula: grit, growth mindset, and character. 

The emerging Ontario student assessment strategy needs to be evaluated in relation to the latest research and best practice, exemplified in Dylan Wiliam’s student assessment research and Daisy Christodoulou’s 2017 book Making Good Progress: The Future of Assessment for Learning.  Viewed through that lens, the Ontario student assessment philosophy and practice falls short on a number of counts.

1. The Generic Skills Approach: Adopting this approach reflects a fundamental misunderstanding about how students learn and acquire meaningful skills. Tacking problem-solving at the outset, utilizing Project-Based Learning to “solve-real life problems” is misguided  because knowledge and skills are better acquired  through other means. The “deliberate practice method” has proven more effective. Far more is learned when students break down skills into a ‘progression of understanding’ — acquiring the knowledge and skill to progress on to bigger problems.
2. Generic Feedback: Generic or transferable skills prove to be unsound when used as a basis for student reporting and feedback on student progress. Skills are not taught in the abstract, so feedback has little meaning for students. Reading a story and making inferences, for example, is not a discrete skill; it is dependent upon knowledge of vocabulary and background context to achieve reading comprehension.
3. Hidden Bias of Teacher Assessment: Teacher classroom assessments are highly desirable, but do not prove as reliable as standardized measures administered under fair and objective conditions. Disadvantaged students, based upon reliable, peer-reviewed research, do better on tests than of regular teacher assessments. “Teacher assessment is biased not because they are carried out by teachers, but because it is carried out by humans.”
4. Unhelpful Prose Descriptors: Most verbal used in system-wide assessments and reports are unhelpful — tend to be jargon-ridden, unintelligible to students and parents, and prove particularly inaccessible to students struggling in school. Second generation descriptors are “pupil friendly” but still prove difficult to use in learning how to improve or correct errors.
5. Work-Generating Assessments: System-wide assessments, poorly constructed, generate unplanned and unexpected marking loads, particularly in the case of qualitative assessments with rubrics or longer marking time. In the U.K., for example, the use of grade descriptors for feedback proved much more time consuming than normal grading of written work Primary teachers who spent 5 hours a week on assessment in 2010, found that, by 2013, they were spending 10 hours a week.What’s wrong with the new Ontario Assessment Plan and needs rethinking?

1. The Generic Skills Approach – Teaching generic skills (SEL) doesn’t work and devalues domain-specific knowledge
2. Social and Emotional Learning (SEL) models — carry inherent biases and are unmeasurable
3. Breach of Student Security – Data mining and student surveys generate personality data without consent
4. Erosion of Teacher Autonomy – Student SEL data generated by algorithms, creates more record-keeping, more marking, cuts into classroom time.

The best evidence-based assessment research, applied in deconstructing the Ontario Assessment initiative, raises red flags.  Bad student assessment practices, as Wiliam and Christodoulou show, can lead to serious workload problems for classroom teachers. No education jurisdiction that lived up to the motto “Learning Province” would plow ahead when the light turns to amber.

A summary of the researchED Ontario presentation delivered April 14, 2018, at the Toronto Airport Westin Hotel. 

Where is the new Ontario student assessment initiative really heading? Is it a thinly-disguised attempt to create a counterweight to current large-scale student achievement assessments? Is it feasible to proceed with SEL assessment when leading researchers question its legitimacy and validity? Are we running the risk of opening the door to the wholesale mining of student personal information without consent and for questionable purposes? 

Read Full Post »

The British researchED Movement: What’s Causing the Buzz in Canadian K-12 Education?

“Asking the right questions” is what most of our best teachers encourage and expect from our students. It’s also what our leading education researchers do when trying to grapple with a particularly thorny or “wicked” problem besetting students and teachers in the schools. Yet far too many teachers across Canada remain reticent to do so because they are essentially trained to carry out provincial mandates. Raising the difficult questions is not always welcomed or appreciated where it counts — among those who set education policy, prepare teachers, and implement curriculum in our K-12 school system.

Working out what works in education is not as simple as it seems, particularly when it comes to improving student learning and deciding upon the most effective pedagogical approach for widely varying cohorts of students. Unfreezing fixed positions, both “progressive” and “traditionalist,” is what opens the door to more meaningful, productive conversations.  We see that in the recent success of Stephen Hurley’s VoicED radio conversations, introducing passionate educators representing differing perspectives to one another for the first time in living memory.

Since its inception in September 2013, researchED has championed creating space for regular classroom teachers in “working out what works” in their classrooms.  Posing those difficult questions can ruffle a few feathers, especially among curriculum leaders and in-house consultants. It’s not easy to venture outside the safe confines of social media “echo chambers” and to consider research generated outside the established “research bubbles.” It’s most encouraging to see Ontario Secondary School Teachers Federation president Harvey Bischof and his Provincial Executive actively supporting the movement.

Grassroots, teacher-led organizations can also, at times. be messy  Teachers are given a new platform to express not only their real life frustrations but also to share their discoveries during forays into the education research world. Independently-minded teachers are free to speak for themselves, but do not speak for researchED.  Debates can get overheated, especially on social media. We do need to be reminded that educators, whatever their persuasions, have to be prepared to listen, consider divergent viewpoints, and treat each other with respect.

The Internet and smart technology has changed the rules of engagement, bringing the latest research within a few keyboard clicks.  One would think that providing a forum for asking deeper questions would be more widely accepted in assessing province-wide and school board-wide initiatives before they are rolled out every September in our K-12 school system.  It can, however, be a little threatening to those promoting theory-based curriculum reform or pedagogical initiatives. Questioning such initiatives, most teachers sense– at least in some school systems –is not always conducive to career advancement.

We should all welcome the arrival of the latest book on Canadian education, Pushing the Limits, written by Kelly Gallagher-Mackay and Nancy Steinhauer and published August 29, 2017.  In many ways, it’s a hopeful and encouraging book because it identifies well-funded “lighthouse projects” in the GTA and a few other Canadian jurisdictions.  While the title is somewhat puzzling, the sub-title is far more indicative of the books real intent, i.e., explaining How Schools Can Prepare Our Children Today for the Challenges of Tomorrow. For Canadian educators and parents looking for a  popular, well-written, fairly persuasive brief for the defense of current policy directions, this is the book for you. For serious education researchers, it will be a goldmine of information on recent initiatives sparking further inquiry into the state of evidence-based teaching practice.

Teachers familiar with researchED will immediately spot a few contentious assertions in Pushing the Limits. Success stories abound and they serve to provide credence to provincial curriculum initiatives underway, particularly in Ontario, Alberta, and British Columbia. The overriding assumption is that schools exist to “prepare our students for the future” and to equip them with “21st century skills.”  Grade 7 teacher Aaron Warner, creator of the two-hour per week  “Genius Hour,” repeats a very familiar claim: “Sixty per cent of the jobs of the future haven’t been invented yet.” That buttresses the overall thesis that lies at the heart of the book.

As leading members of the Ontario People for Education research team, Gallagher-Mackay and Steinhauer, as expected,  do make a case for broadening provincial student assessments to include SEL, short for “Social and Emotional Learning.” That’s hardly surprising, given the Ontario Education- P4E partnership  driving that initiative across the province. Digging more deeply, it will be interesting to see what evidence the authors produce that it is either advisable or can be done successfully.

The wisdom of proceeding to adopt SEL system-wide and to recast student assessment in that mold remains contentious. On this particular subject, they might be well advised to consider Anya Kamenetz‘s recent National Public Radio commentary (August 16, 2017) explaining, in some detail, why SEL is problematic because, so far, it’s proven impossible to assess what has yet to be properly defined as student outcomes. They also seem to have overlooked Carol Dweck’s recently expressed concerns about using her “Growth Mindset” research for other purposes, such as proposing a system-wide SEL assessment plan.

Good books tackle big issues and raise fundamental questions, whether intended or not. Teachers imbued with the researchED spirit will be well equipped to not only tackle and effectively scrutinize Pushing the Limits, but to bring a broader and deeper understanding and far more scrutiny of the book’s premises, contentions, and prescriptions. That, in turn, will  hopefully spark a much better informed discussion within the Canadian K-12 educational community.

What’s causing all the buzz in the rather small Canadian teacher education research community? Is it the appearance of a new player committed to raising those difficult questions and to assessing initiatives, through a teachers’ lens? Is it our seeming aversion to considering or supporting evidence-based classroom practice? And is there room for a new voice in Canadian teacher-led education research and reform? 

 

Read Full Post »