Feeds:
Posts
Comments

Posts Tagged ‘CMEC’

“All that glitters is not gold” is a famous proverb plucked from William Shakespeare‘s play The Merchant of Venice that may well apply to recent international appraisals of K-12 education in Canada. Such rosy assessments tend to put a shiny lustre on what is essentially a sound and ‘pretty good’ school system that has lost ground to competing nations over the past decade.

Five years ago, the Organization for Economic Cooperation and Development(OECD) produced a rather rosy Education Policy Outlook for Canada as part of a series of reports offering comparative analysis of education policies and reforms across the world’s developed countries. Canada’s overall performance, aggregated from widely varied provincial assessment data, looked good, in comparison with the United States, the United Kingdom, and Australia. Most significantly, the OECD assessors brushed aside concerns about “plateaued student achievement” on the Programme of International Student Assessment (PISA) tests and the decline in the proportion of top performing students.

Emerging concerns were most clearly expressed in Dr. Paul Cappon’s final 2010 report for the Canadian Council on Learning. Student scores on the 2009 PISA test had revealed that Canadian 15-year-olds demonstrated relatively strong sets of skills in reading, math and science, but they were already slipping relative to high performing Asian countries and in some cases in absolute terms. “What I’m hoping,” Cappon said at the outset of his final cross-Canada tour, “is that when people realize that Canada is slipping down the international learning curve we’re not going to be able to compete in the future unless we get our act together.”

OECD Education Policy Outlook assessments and Country reports are based upon templates that tend to favour diverse and well-funded school systems like that of Canada. The six identified policy levers in 2015 were: 1) equity and quality of education; 2) preparing students for the future; 3) school improvement; 4) evaluation and assessment; 5) governance; and 6) funding.  Such public policy forecasts, based upon conventional criteria and historic trends, also tend to demonstrate “path dependency” which limits the capacity to capture radical shifts in context or dynamic changes in educational direction.

Fifteen-year-old students in Canada, based upon triennial PISA tests from 2000 to 2018, continue to perform above the OECD average in reading, mathematics and science. Our most economically and socially disadvantaged students, in aggregate, do relatively better than those in competing countries, demonstrating more equity than in most other countries.  A considerably higher proportion of Canadian K-12 students proceed to post-secondary education in universities and colleges. That much has not changed across time.

Three significant changes can be identified from the accumulating OECD student assessment and survey data and they deserve far more critical scrutiny:

Downward Trend in Student Performance:  The performance trends for Canadian fifteen-year-olds are consistently downward from 2000 to 2018 in READING,  from 2003 to 2018 in MATHEMATICS, and from 2006 to 2018 in SCIENCE.  While the OECD average scores are also in decline as more countries are included in PISA, the descent is more pronounced among students from Canada. Students in Canada’s top performing provinces of Alberta, Ontario, British Columbia and Quebec (Mathematics) tend to buoy-up the lagging results produced by students from New Brunswick, Newfoundland/Labrador, Saskatchewan, and Manitoba.

Deteriorating Classroom Disciplinary Climate:

The 2015 Education Policy Outlook for Canada flagged one measure, based upon student survey responses, where Canada simply met the OECD standard – the index of classrooms conducive to learning (Figure 5, OECD Canada, 2015).  That largely undiagnosed problem has worsened over the past three years.  Canada ranked 60th out of 77 participating nations and educational districts in the OECD’s 2018 index of disciplinary climate, released on December 4, 2019.  According to a global student survey conducted in the spring of 2018, one in five students, 15 years-of-age, report that learning time is lost to noise, distractions, and disorder, so much so that it detracts from learning in class. A relatively high proportion of Canadian students say the teacher is not listened to and it takes a long time for the class to settle down. In addition, students regularly skip school and report late to class.

High Incidence of Fear of Failure:

Personal anxieties may also run higher among Canadian students when they confront writing standardized tests and experience a fear of failing the test. In Canada, the OECD 2019 Education GPS report states, “15-year-old students have a strong fear of failure”ranking 6th among 77 national student groups participating in the survey.  Fear of failure runs highest among students in Chinese Taipei, Singapore, Macau, Japan, and Germany, but is less pronounced in high performing countries such as Korea. Estonia, and Finland.  Such fears are present to the same degree among students in the United Kingdom, but less so in the United States.  No analysis whatsoever is offered to explain why fears run so comparatively high among teens in Canada.

The initial report on the Canadian Results of the OECD PISA 2018 Study, released by the Council of Ministers of Education (CMEC) in early December 2019, are of little help in evaluating these rather striking trends.  Like previous reports in the CMEC series, the report puts a positive spin on the aggregate results by casting them within a broad, global context, lumping together countries with radically different commitments to education in terms of spending and resources. It is possible to ferret out anomalies and to conduct province-by-province comparisons, but only with time, effort, and attention to detail. That is sufficient to keep it either buried or accessible only to education assessment specialists.

Does the Canadian Education Policy Outlook ventured in 2015 stand up under close analysis. five years on?  What’s missing from the OECD and CMEC assessment reports for Canada over the past decade?  Should the Canadian public be concerned about the downward trend in the demonstration of core skills in reading, mathematics and science?  Is disciplinary climate now a real concern in Canadian classrooms? And why are Canadian students so afraid of failing in our schools when grade promotion and graduation rates are at record levels?

Read Full Post »

Millions of Facebook users were profiled by Cambridge Analytica without their knowledge and that public disclosure has heightened everyone’s awareness of not only the trend to “personality profiling,’ but the potential for massive invasion of privacy. These controversial actions have exposed the scope of Big Data and the wider aspirations of the data analytics industry to probe into the “hidden depths of people.” It has also, as U.K. expert Ben Williamson has reminded us, tipped us off about the growing trend toward personality measurement in K-12 and post-secondary education.

Williamson’s 2017 book, Big Data in Education, sounded the alert that the collection and analysis of more personal information from schoolchildren will be a defining feature of education in coming years. And just as the Facebook debacle raises public concerns about the use of personal data, a new international test of ten and 15-year-olds is to be introduced by the Organization for Economic Cooperation and Development (OECD) – a powerful influence on national education policies at a global scale.  Almost without being detected, it is also emerging as a key component of the current Ontario Student “Well-Being” Assessment, initially piloted from 2014 to 2016 by Ontario People for Education as the core objective of its Measuring What Matters project.

Most data collected about students since the 1990s has came from conventional international, national and provincial examinations of knowledge and cognitive skills. Preparing students for success in the 21st century workplace has been a major driver of most initiatives in testing and accountability.  International test results such as OECD’s Program of International Student Assessment (PISA) have also become surrogate measures of the future economic potential of nations, feeding a global education race among national education systems.

The advent of Big Data is gradually transforming the nature of student assessment. While the initial phase was focused on stimulating competitive instincts and striving for excellence, more recent initiatives are seeking to “broaden the focus of student assessment” to include what is termed “social and emotional learning (SEL).” Much of the motivation is to secure some economic advantage, but that is now being more broadly defined to help mould students committed to more than individual competitiveness.  With the capacity to collect more “intimate” data about social and emotional skills to measure personality, education policymakers are devising curriculum and assessment programmes to improve personality scores. Despite the Cambridge Analytica controversy, personality data is well on the way to being used in education to achieve a variety of competing political objectives.

The ‘Big Five’ of Personality Profiling

The science of the psychographic profiling employed by Cambridge Analytica is hotly contested. It is, however, based on psychological methods that have a long history for measuring and categorizing people by personality. At its core is a psychological model called the “five factor model” of personality – or the “Big Five.” These include “openness”, “conscientiousness”, “extroversion”, “agreeableness” and “neuroticism” (OCEAN). Personality theorists believe these categories are suitable for classifying the full range of human personalities. Psychologists have invented instruments such as the so-called ‘Big Five Inventory’  to capture OCEAN data for personality modelling.

Advent of Stealth Assessment

The upcoming 2018 OECD PISA test will include, for the first time, a battery of questions aimed at assessing “global competencies” with a distinct SEL orientation. In 2019, the OECD plans to launch its international Study of Social and Emotional Learning  Designed as a computer-based self-completion questionnaire, at its core the test is a modified version of the Big Five Inventory. The OECD version maps exactly onto the five factor personality categories with “emotional stability” substituted in place of “neuroticism.” When implemented, the social and emotional skills test will assess students against each of the Big Five categories.

The OECD Education Skills experts, working in collaboration with Pearson International, firmly believe that social and emotional skills are important predictors of educational progress and future workplace performance. Large-scale personality data is clearly seen by the OECD to be predictive of a country’s potential social and economic progress. Although both the OECD and the Ontario Student Well-Being advocates both claim that it is strictly a test of social and emotional skills, Williamson claims such projects employ the same family of methods used in the Cambridge Analytica personality quiz. Upon closer examination, the same psychological assumptions and personality assessment methods underpin most of the latest education ventures.

The OECD is already a powerful influence on the moulding of national education policies. Its PISA testing has reshaped school curricula, assessments and whole systems in the global education race.  It is increasingly likely that its emphasis on personality testing will, once again, reshape education policy and school practices. Just as PISA has influenced a global market in products to support the core skills of literacy, numeracy and science tested by the assessment, the same is now occurring around SEL and personality development.  Canada’s provincial and territorial ministers of education, working under the auspices of the Council of Ministers of Education, Canada (CMEC) have not only endorsed the OECD’s  proposed “global competencies,” but proposed a variation of their own to guide assessment policy.

The Ontario Student Assessment initiative, announced September 6, 2017, deserves closer scrutiny through the lens of datafication and personality profiling. It’s overarching goal bears repeating: “Update provincial assessment and reporting practices, including EQAO, to make sure they are culturally relevant, measure a wider range of learning, and better reflect student well-being and equity.”  Founder of People for Education Annie Kidder hailed the plan for “embedding” the “transferable skills” and positioning Ontario to take “a leading role in the global movement toward broader goals for education and broader measures of success in our schools.”

Critics of large-scale student assessments are quick to identify the underlying influence of “globalization” and the oft-stated goal  of preparing students for the highly competitive “21st century workplace.”  It can be harder to spot currents moving in the opposite direction and heavily influenced by what Kathryn Ecclestone and Denis Hayes aptly termed the “therapeutic education ethos.” Ten years ago, they flagged the rise of  a “therapeutic education” movement exemplified by classroom activities and programs, often branded as promoting ‘mindfulness,’ which pave the way for “coaching appropriate emotions” and transform education into a disguised form of “social engineering” aimed at producing “emotionally literate citizens” who are notably “happy” and experience “emotional well-being.”

Preparing students to be highly competitive human beings or to be creative and cooperative individuals is risking re-framing public education in terms of personality modification, driven by ideological motivations, rather than the pursuit of meaningful knowledge and understanding. It treats children as ‘guinea pigs’ engaged in either market competition preparation or social engineering, and may well stand in the way of classroom teachers pursuing their own evidence-based, knowledge-centred curriculum aims.

Appropriating and misusing personality data by Facebook and Cambridge Analytica led to a significant world-wide public backlash. In education, however, tests and technologies to measure student personality, according to Williamson, are passing unchallenged. It is equally controversial to capture and mine students’ personality data with the goal of shaping students to “fit into” the evolving global marketplace.  Stealth assessment has arrived and being forewarned is forearmed.

Why is education embracing data mining and personality profiling for schoolchildren? What are the connections between Facebook data mining and recent social-and-emotional learning assessment initiatives?  Should students and parents be advised, in advance, when student data is being minded and mapped against personality types?  Why have Canadian assessment projects like the Ontario Measuring What Matters- Student Well-Being initiative escaped close scrutiny?  Should we be more vigilant in tracking and monitoring the use and abuse of Big Data in education? 

Read Full Post »

“Canadians can be proud of our showing in the 2015 Programme for International Student Assessment (PISA) report,” declared Science consultant Bonnie Schmidt and former Council of Ministers of Education (CMEC) director Andrew Parkin in their first-off-the mark December 6, 2016 response to the results. “We are, ” they added, “one of only a handful of countries that places in the top tier of the Oganization for Economic Development and Cooperation (OECD) in each of the three subjects tested:science, reading and math.”

pisa2015cmeccover“Canada” and “Canadian students,” we were told, were once again riding high in the once-every-three-years international test sweepstakes. If that that effusively positive response had a familiar ring, it was because it followed the official line advanced by a markedly similar CMEC media release, issued a few hours before the commentary.

Since our students, all students in each of our ten provincial school systems, were “excelling,” then it was time for a little national back-slapping. There’s one problem with that blanket analysis: it serves to maintain the status quo, engender complacency, obscure the critical Mathematics scores, and disguise the lopsided nature of student performance from region to region.

Hold on, not so fast, CMEC — the devil is in the real details and more clearly portrayed in the OECD’s own “Country Profile” for Canada. Yes, 15-year-olds in three Canadian provinces (Alberta, British Columbia, and Quebec) achieved some excellent results, but overall Mathematics scores were down, and students in over half of our provinces trailed-off into mediocrity in terms of performance. Our real success was not in performance, but rather in reducing the achievement gap adversely affecting disadvantaged students.

Over half a million 15-year-olds in more than 72 jurisdictions all over the world completed PISA tests, and Schmidt and Parkin were not alone in making sweeping pronouncements about why Canada and other countries are up and others down in the global rankings.

Talking in aggregate terms about the PISA performance of 20,000 Canadian students in ten different provinces can be, and is, misleading, when the performance results in mathematics continue to lag, Ontario students continue to underperform, and students in two provinces, Manitoba and Saskatchewan, struggle in science, reading, and mathematics.  Explaining all that away is what breeds complacency in the school system.

My own PISA 2015 forecast was way off-base — and taught me a lesson.  After the recent TIMSS 2015 Mathematics results released in November 2016, an  East Asian sweep, led by Singapore and Korea, seemed like a safe bet. How Finland performs also attracts far less attention than it did in its halcyon days back in 2003 and 2006. The significant OECD pivot away from excellence to equity caught me napping and I completely missed the significance of moving (2012 to 2015) from pencil-and-paper to computer-based tests. 

Some solace can be found in the erroneous forcecasts of others. The  recent Alberta Teachers’ Association (ATA) “Brace Yourself” memo with its critique of standardized testing assessment, seemed to forecast a calamitous drop in Alberta student performance levels. It only happened in Mathematics.

Advocates of the ‘Well-Being’ curriculum and broader assessment measures, championed by Toronto’s People for Education, will likely be temporarily thrown off-stride by the OECD’s new-found commitment to assessing equity in education. It will be harder now to paint PISA as evil and to discredit PISA results based upon such a narrow range of skills in reading, math and science.

The OECD’s “Country Profile” of Canada is worth studying carefully because it aggregates data from 2003 to 2015, clarifies the trends, and shows how Canadian students continue to struggle in mathematics far more than in reading and science.

Canadian students may have finished 12th in Mathematics with a 516 aggregate score, but the trend line continues to be in decline, down from 532 in 2003. Digging deeper, we see that students in only two provinces, Quebec ( 544) and BC (522) actually exceeded the national mean score. Canada’s former leader in Mathematics performance, Alberta, continued its downward spiral from the lofty heights of 549 (2003) to 511 (2015).

Since Ontario students’ provincial mathematics scores are declining, experts will be pouring over the latest PISA results to see how bad it is in relation to the world’s top performing systems. No surprises here: Ontario students scored 509, finishing 4th in Canada, and down from 530 on PISA 2003. Excellence will require a significant change in direction.

The biggest discovery in post-2015 PISA analysis was the positive link between explicit instruction and higher achievement in the 2015 core assessment subject, science. The most important factor linked with high performance remains SES (soci0-economic status), but teacher-guided instruction was weighted close behind and students taught with minimal direction, in inquiry or project-based classes, simply performed less well on the global test.

The results of the 15-year-olds are largely determined over 10 years of schooling, and not necessarily the direct consequence of the latest curriculum fad such as “discovery math.’’

It’s better to look deeper into what this cohort of students were learning when they first entered the school system, in the mid-1990s. In the case of Canadian students, for example, student-centred learning was at its height, and the country was just awakening to the value of testing to determine what students were actually learning in class.

Where the student results are outstanding, such as Singapore and Estonia, it is not solely attributable to the excellence of teaching or the rigour of the math and science curriculum.

We know from the “tutoring explosion” in Canada’s major cities that the prevalence of private tuition classes after school is a contributing factor, and may explain the current advantage still enjoyed in mathematics by Pacific Rim students.

Children of Chinese heritage in Australia actually outperformed students in Shanghai on the 2012 PISA test, and we need to explore whether that may be true for their counterparts in Greater Vancouver. The so-called “Shanghai Effect” may be attributed as much to “tiger mothers” as it is to the quality of classroom instruction.

Whether Canada and Canadians continue to exhibit high PISA self-esteem or have simply plateaued does not matter as much as what we glean over the next few years from studying best international practice in teaching, learning, and assessment.

Surveying PISA student results, this much is clear: standing still is not an option in view of the profound changes that are taking place in life, work, and society.

 

Read Full Post »