Feeds:
Posts
Comments

Posts Tagged ‘Competency-Based Assessment’

The ongoing COVID-19 crisis may be claiming another victim in one of Canada’s leading education provinces – sound, reliable, standards-based and replicable summative student assessment. After thwarting a 2017-18 Learning Province plan to subvert the province’s Grade 3 provincial student assessment and broaden the ‘measures of success,’ the Ontario Doug Ford government and its education authorities appear to be falling into a similar trap.

What’s most unexpected is that the latest lubricant on the slippery slope toward ‘accountability-free’ education may well have been applied in Doug Ford’s Ontario under a government ostensibly committed to ‘back-to-basics’ and ‘measurable standards’ in the K-12 school system.


All K-12 provincial tests, administered by the Education Quality and Accountability Office (EQAO) were the first to go, rationalized as a response to the pandemic and its impact upon students, teachers, and families. More recently, Ontario’s education ministry opened the door to cancelling final exams by giving school boards the right to replace exam days with in-class instructional time.


Traditional examinations, the long-established benchmark for assessing student achievement, simply disappeared, for the second assessment cycle in a row, going back to the onset of the COVID-19 outbreak. Major metropolitan school districts, led by the Toronto District School Board, Peel District School Board and their coterminous Catholic boards, jumped in quickly to suspend exams in favour of what were loosely termed “culminating tasks” or “demonstrations of learning.”


Suspending exams was hailed in the Toronto Star news report as ‘a rare bright spot” for Ontario high school students. Elsewhere the decision to eliminate exams, once again, elicited barely a whimper, even from the universities. “Nobody’s missed standardized tests or final exams,” University of Ottawa professor Andy Hargreaves noted rather gleefully during the October 29-30 Canadian EdTech Summit.


Suspending examinations has hidden and longer-term consequences not only for students and teachers, but for what remains of school-system accountability. What’s most surprising, here in Canada, is that such decisions are rarely evidence-informed or predicated on the existence of viable, proven and sustainable alternatives.


Proposing to substitute culminating projects labelled as “demonstrations of learning” is based upon the fallacious assumption that teacher assessments are better than final exams. Cherry-picking a recent sympathetic research study, such as a May 2019 Journal of Child Psychology and Psychiatry article highlighting exam stress, may satisfy some, but it is no substitute for serious research into the effectiveness of previous competency-based “culminating activity” experiments.


Sound student evaluation is based upon a mix of assessment strategies, ranging from formative (daily interaction and feedback) assessment to standardized tests and examinations (summative assessment). It is highly desirable to base student assessment upon a suitable combination of reasonably objective testing instruments as well as teacher-driven subjective assessment. UK student assessment expert, Daisy Christodoulou, puts it this way: “Tests are inhuman – and that is what is good about them.”


Teacher-made and evaluated assessments appear, on the surface, to be more gentle and fairer than exams, but such assumptions can be misleading, given the weight of research supporting “level playing field” evaluations. The reality is that teacher assessments tend to be more impressionistic, not always reliable, and can produce outcomes less fair to students.


Eliminating provincial tests and examinations puts too much emphasis on teacher assessment, a form of student evaluation with identified biases. A rather extensive 2015 student assessment literature review, conducted by Professor Rob Coe at the Durham University Centre for Evaluation and Monitoring, identifies the typical biases. Compared to standardized tests, teacher assessment tends to exhibit biases against exceptional students, specifically those with special needs, challenging behaviour, language difficulties, or personality types different than their teacher. Teacher-marked evaluations also tend to reinforce stereotypes, such as boys are better at math or racialized students underperform in school.


Replacing final exams with teacher-graded ‘exhibitions’ or ‘demonstrations of learning mastery’ sounds attractive, but is fraught with potential problems, judging for their track record since their inception in the late 1980s. Dreamed up by the North American father of Outcome-Based Education, Dr. William Spady, assessing student competencies based upon ‘demonstrations of learning’ have a checkered history. Grappling with the OBE system and its time-consuming measurement of hundreds of competencies finished it off with classroom teachers.


A more successful version of DOLM (Demonstration of Learning Mastery), developed by Deborah Meier, Theodore Sizer and the Coalition of Essential Schools (1988 -2016), was piloted in small schools with highly-trained teachers. Such exhibitions were far from improvisational but rather “high stakes, standards aligned assessments” which aimed at securing “commitment, engagement and high-level intellectual achievement” and conceived as “a fulcrum for school transformation.” Systemic distrust, aggravated by testing and accountability, Meier conceded, “rendered attempts to create such contexts infertile.”


Constructing summative evaluation models to replace final exams is not easy and it has defeated waves of American assessment reformers. The Kentucky Commonwealth Accountability and Testing System (CATS) 2007-2008, and its predecessor, KRIS (1992-1998) serve as a case in point. Like most of these first generation reforms, the KRIS experiment was widely considered a failure. Its performance-based tools were found to be unreliable, professional development costs too high, and two elements of the program, Mathematics Portfolios and Performance Events, summarily abandoned. Writing portfolios continued under CATS but a 2008 audit revealed wide variations in marking standards and lengthy delays in returning the marked results of open answer questions.


Most of the recent generation of initiatives were sparked by a January 2015 white paper, “Performance Assessments: How State Policy Can Advance Assessments for 21st Century Learning,” produced by two leading American educators, Linda Darling-Hammond and Ace Parsi. Seven American states were granted a waiver under the Every Student Succeeds Act (ESSA) to experiment with such competency-based assessment alternatives.


Constructing a state model compliant with established national standards in New Hampshire proved to be an insurmountable challenge. While supported by Monty Neill and Fair Test Coalition advocacy forces, New Hampshire’s Performance Assessments for Competency Education (PACE) system ran into significant problems trying to integrate Classroom-Based Evidence (CBE) with state testing criteria and expectations. Establishing evaluation consistency and “comparability” across schools and districts ultimately sunk the experiment. It was anchored in state standards and required external moderation, including re-scoring of classroom-based work. Serving two masters created heavier teacher marking loads and made it unsustainable. Federal funding for such competency-based assessment experiments was cut in December 2019, effectively ending support for that initiative.


Provincial tests and exams exist for a reason and ensure that we do not fly blind into the future.. Replacing final exams with a patchwork solution is not really a wise option this school year. Simply throwing together culminating student activities to replace examinations is, judging from past experiments, most likely a recipe for inconsistency, confusion, and ultimate failure.


Teachers will, as always, do their best and especially so given the current turbulent circumstances. Knowing what we know about student assessment, let’s not pretend that the crisis measures are better than traditional and more rigorous systems that have stood the test of time.

What are the fundamental purposes of summative student assessment? Should provincial tests and final exams be suspended during the second year of the COVID-19 pandemic? Where’s the research to support the effectiveness of alternative ‘demonstration of learning’ strategies? Are we now on the slippery slope toward ‘accountability-free’ education?

Read Full Post »

University of Kentucky student assessment guru Thomas R. Guskey is back on the Canadian Professional Development circuit with a new version of what looks very much like Outcomes-Based Education.  It is clear that he has the ear of the current leadership in the Education Department of Prince Edward Island.  For two days in late November 2018, he dazzled a captive audience of over 200 senior Island school administrators with has stock presentations extolling the virtues of mastery learning and competency-based student assessment.

GuskeyThomasSpeakingP.E. I’s Coordinator of Leadership and Learning Jane Hastelow was effusive in her praise for Guskey and his assessment theories. Tweets by educators emanating from the Guskey sessions parroted the gist of his message. “Students don’t always learn at the same rate or in the same order,” Guskey told the audience. So, why do we teach them in grades, award marks, and promote them in batches?

Grading students and assigning marks, according to Guskey, can have detrimental effects on children. “No research,” he claims, “supports the idea that low grades prompt students to try harder. More often, low grades lead students to withdraw from learning.”

Professional learning, in Guskey’s world, should be focused not on cognitive or knowledge-based learning, but on introducing “mastery learning” as a way of advancing “differentiated instruction” classrooms. “High-quality corrective instruction,” he told P.E.I. educators, is not the same as ‘re-teaching.’” It is actually a means of training teachers to adopt new approaches that “accommodate differences in students’ learning styles, learning modalities, or types of intelligence.”.

Guskey is well-known in North American education as the chief proponent for the elimination of percentage grades.  For more than two decades, in countless PD presentations, he has promoted his own preferred brand of student assessment reform. “It’s time, “ he insists, “ to abandon grading scales that distort the accuracy, objectivity and reliability of students’ grades.”

Up and coming principals and curriculum leads, most without much knowledge of assessment, have proven to be putty in his hands. If so, what’s the problem?   Simply put, Dr. Guskey’s theories, when translated into student evaluation policy and reporting, generate resistance among engaged parents looking for something completely different – clearer, understandable, jargon-free student reports with real marks. Classroom teachers soon come to realize that the new strategies and rubrics are far more complicated and time-consuming, often leaving them buried in additional workload.

Guskey’s student assessment theories do appeal to school administrators who espouse progressive educational principles. He specializes in promoting competency-based education grafted onto student-centred pedagogy or teaching methods.

Most regular teachers today are only too familiar with top-down reform designed to promote “assessment for learning” (AfL) and see, first hand, how it has led to the steady erosion of teacher autonomy in the classroom.

While AfL is a sound assessment philosophy, pioneered by the leading U.K. researcher Dylan Wiliam since the mid-1990s, it has proven difficult to implement. Good ideas can become discredited by poor implementation, especially when formative assessment becomes just another vehicle for a new generation of summative assessment used to validate standards.

Education leaders entranced by Guskey’s theories rarely delve into where it all leads for classroom teachers.  In Canada, it took the “no zeros” controversy sparked in May 2012 by Alberta teacher Lynden Dorval to bring the whole dispute into sharper relief. As a veteran high school Physics teacher, Dorval resisted his Edmonton high school’s policy which prevented him from assigning zeros when students, after repeated reminders, failed to produce assignments or appear for make-up tests.

Teachers running smack up against such policies learn that the ‘research’ supporting “no zeros” policy can be traced back to an October 2004 Thomas Guskey article in the Principal Leadership magazine entitled “Zero Alternatives.”

Manitoba social studies teacher Michael Zwaagstra analyzed Guskey’s research and found it wanting.  His claim that awarding zeros was a questionable practice rested on a single 20-year-old opinion-based presentation by an Oregon English teacher to the 1993 National Middle School conference. Guskey’s subsequent books either repeat that reference or simply restate his hypothesis as an incontestable truth.

SpadyWilliamOBEGuskey’s theories are certainly not new. Much of the research dates back to the early 1990s and the work of William Spady, a Mastery Learning theorist known as the prime architect of the ill-fated Outcomes-Based Education (OBE) movement.  OBE was best exemplified by the infamous mind-boggling systematized report cards loaded with hundreds of learning outcomes, and it capsized in in the early 2000s. in the wake of a storm of public and professional opposition in Pennsylvania and a number of other states.

The litmus test for education reform initiatives is now set at a rather low bar – “do no harm” to teachers or students.  What Thomas Guskey is spouting begs for more serious investigation. One red flag is his continued reference to “learning styles” and “multiple intelligences,” two concepts that do not exist and are now considered abandoned theories.

Guskey’s student assessment theories fly mostly in the face of the weight of recent research, including that of Dylan Wiliam.  Much of the best research is synthesized in Daisy Christodoulou’s 2014 book, Making Good Progress. Such initiatives float on unproven theories, lack supporting evidence-based research, chip away at teacher autonomy, and leave classroom practitioners snowed under with heavier ‘new age’ marking loads.

A word to the wise for  P.E.I. Education leadership – look closely before you leap. Take a closer look at the latest research on teacher-driven student assessment and why OBE was rejected twenty years ago by classroom teachers and legions of skeptical parents.

What’s really new about Dr. Thomas Guskey’s latest project known as Competency-Based Assessment? What is its appeal for classroom teachers concerned about time-consuming, labour-intensive assessment schemes?  Will engaged and informed parents ever accept the elimination of student grades? Where’s the evidence-based research to support changes based upon such untested theories? 

Read Full Post »