researchED Auckland 2018

researchED2018

Isn’t it crazy that, in 2018, we’re still “working out what works” in Education?

In fact, some of us do already have a pretty good idea of what works, but getting the right people to listen is a different problem altogether.

And so, a group of like-minded individuals (and maybe a couple of sceptics) gave up their Saturday on Queen’s Birthday weekend to attend New Zealand’s very first researchED conference in Auckland. researchED is a growing movement based in the UK but spreading internationally, “a grass-roots, teacher-led project that aims to make teachers research-literate and pseudo-science proof” (and by golly does this country need proofing). Founder Tom Bennett quickly realised that his own teacher training was based more on edu-myths and dogma (e.g. learning styles) than any scientific, evidence-based research.  He’s not the only one.  Daisy Christodoulou’s book, Seven Myths About Education, is the coffee that any waking 21st century learning fanatic should smell.  Briar Lipson at the New Zealand Initiative hasn’t spent very long in this country, but has already sized up our education system very well and should be commended for bringing researchED to New Zealand.

Every talk raised serious questions about how we teach in New Zealand, and everyone was there in the belief that we can, and should, be doing better.  Not surprisingly, the academics are calling for the Ministry of Education to change their ways and look for evidence before adopting fads as policies, while the pragmatic principals and teachers cannot afford to wait and are simply getting on with things.

The common factor of the day was subject knowledge and the importance of committing knowledge to long-term memory.  The 21st century learning ethos suggests that we should leapfrog, or at least skim over, these foundational skills in a bid to produce generic critical thinkers and problem solvers, but surely common sense tells us we cannot reasonably expect students to think critically or solve problems unless they actually have some knowledge to work with.

I have no desire to repeat what has been said so well by others, so instead I will direct readers to a newly created blog by Derek Hopper, a music teacher at Tauraroa Area School who has read up on what works and is spreading the word.  He and his colleagues are seeing significant improvements in student behaviour and achievement. Happy students, happy teachers.  Having already spoken to a maths teacher at Tauraroa who is offering guidance to their primary teachers, I believe this school may well provide the model for other schools to follow.

Some other reflections of the day:

Tom Bennett, founder of researchED: Teachers might think that indulging in (catering for individual) learning styles is a harmless bit of fun, but there is no time to waste when teaching children from disadvantaged backgrounds.  Every minute counts.

Katharine Birbalsingh, keynote speaker and founder/Headmistress of the evidence-informed Michaela Community School in London: Her teachers do not play “Guess what’s in my head?”, i.e. they don’t question their students before the relevant knowledge has been taught, so that every student, regardless of their background, has an equal chance of answering the teachers’ questions correctly.  A subtle but powerful way to address social inequity and level the playing field.

Dr. Michael Johnston, Victoria University: When new skills are learned and practised sufficiently, they become automatic and free up the working memory to concentrate on higher-order thinking.  With particular reference to mathematics pedagogy, the current NCEA internal assessment system provides little incentive for students to practise skills and procedures to the point of automaticity, and if they haven’t reached that point, then they will struggle with the cognitive demands of solving the contextualised problems presented in assessment.

Prof. Elizabeth Rata, Auckland University: Already widely known for her views on the lack of academic knowledge in the curriculum.  When she used the definition of the apostrophe as an example of understanding the epistemic structure of academic knowledge, I genuinely thought she was going to ask the audience if they had spotted the misplaced apostrophe in the previous slide.  She didn’t.  I suddenly felt alone.

Dr. Graham McPhail, Auckland University: There is little evidence that deep learning occurs through subject integration.  Wineburg and Grossman (2000) warned that ‘often the choice to implement a new curriculum is based on symbolic factors, such as a desire to be seen as progressive and in the forefront of reform’.

Louise Zame, primary school teacher:  When listening to a teacher speak so eloquently about the professional challenges of implementing Inquiry Learning…to a bunch of 5-7 year olds…you realise just how much the Ministry of Education has lost the plot.  As part of her Master’s research, Louise asks the pertinent question: what content knowledge do young students (aged 5-7 years) gain through inquiry learning?

Dr. Shaun Hawthorne, Cognition Education Ltd: Prof. John Hattie has recently updated his list of influences on student achievement, and top of the list is now “collective teacher efficacy” with a whopping effect size of 1.57.  For those who don’t know about Hattie’s effect size measure, almost everything on the list has a positive effect, so teachers and schools should not be too complacent. They should be looking to maximise their impact, and punching above the average effect size of 0.40.

To finish:

  • I was probably the only person excited to spend a bit of time in the Vaughan Jones Room during the lunch break.
  • Great care must be exercised when evaluating “evidence-based research”.  There is a lot of rubbish out there.  For example, the Numeracy Development Projects “research” showed that if you teach children strategies then children will learn strategies.  Big deal.
  • The panel discussion at the end left me in no doubt of the monumental challenge we face trying to fix New Zealand’s education system. To quote John Morris, “Currently education policy is being determined by political imperatives. It should not be. All policy initiatives, and in education there are so many of them, should be evidence-based.”
  • Tom Haig from the NZPPTA was naturally highly sensitive to the political undertones of the day and felt the debate was too one-sided.  Perhaps that’s because there is little to debate when we rely on evidence.  If the focus on credible and reliable evidence can take the politics out of Education, then bring it on I say, for I can think of no group of stakeholders less politically-minded than our precious children.

Dr Audrey Tan, Mathmo Consulting
8 June 2018

NCEA – where less is more

Our “revolutionary” National Certificate(s) of Educational Achievement (NCEA) secondary school qualification, built on ideals of inclusion and equity, has failed to deliver on many fronts.

The New Zealand Initiative has highlighted some serious problems with the NCEA system. Briar Lipson’s well researched report, Spoiled by Choice: How NCEA Hampers Education and What It Needs To Succeed, exposed the harsh reality behind the dramatic growth in numbers of students achieving NCEA Level 2 each year. NCEA performance may be “improving”, but the international survey PISA shows that our 15-year-olds’ capabilities in maths, science and reading are declining.

Spoiled_by_choice_Fig1

Moreover, despite there being a minimum requirement for literacy and numeracy credits, a Tertiary Education Commission (TEC) study in 2014 found that reasonably large proportions of students with NCEA Level 1 (approximately half) or NCEA Level 2 (four out of 10) were not functionally literate, and similar proportions were not functionally numerate.

TEC_Literacy_Fig1
TEC_Numeracy_Fig2

So what can we actually ascertain about a student with an NCEA qualification?

Very little, according to the Initiative’s second report, Score! Transforming NCEA Data. It brought to attention the huge variation in grade distribution across subject sub-fields. It is comparatively easier to gain an Excellence grade in Languages and Performing Arts than it is in Mathematics and Statistics. And not surprisingly, they found that more students pass internal assessments than external assessments.

Score_Fig4
Score_Fig2

In an attempt to redress these imbalances, the economists at the Initiative came up with a Weighted Relative Performance Index (WRPI) that endeavours to make “sense” of a student’s NCEA credits and provide a fairer comparison of performance between students. 1

It was a laudable attempt. However, one blatant inequity in the NCEA system that was not addressed by the WRPI, and has yet to be discussed widely, is the absence of a sensible time restriction applied to individual external assessments.

Allow me to explain. In Mathematics, students at any of the three NCEA Levels may be entered for a three-hour external examination comprising up to three achievement standards. Each achievement standard is a self-contained assessment/paper, sealed up in its own plastic wrapping. (Biodegradable, I hope.)

It would seem reasonable that each of the three papers should be completed in approximately one hour. 2 Therefore, it would seem reasonable that if a school decides to enter a student for fewer than three achievement standards, then the duration of the exam should be reduced, i.e. one hour for one paper, two hours for two papers. But no, not with NCEA! Schools may enter students for one or two papers, and students still get three hours to complete them!

This obviously puts students entered for three papers at a disadvantage. Are these students supposed to console themselves that they are receiving a better education, even if their peers come out with higher grades because they had more time?

Schools can, and do, game the system by entering their students for fewer than three papers. But students can also game the system by electing to not attempt all of the papers for which they are entered. If they leave the plastic wrapping intact, they will receive a Standard Not Attempted/Assessed (SNA), and apparently this is better than a Not Achieved (N).

Really?? According to this memo, it’s better for schools because School Result Summaries will include N’s but not SNA’s. Also, SNA allows for the possibility that “a student ran out of time so an N would not be a fair result”. In other words, it’s better to fail to try than to try and fail.

For those chasing an Endorsement, it is actually in their best interests to attempt fewer papers and go for higher grades. For example, for a student chasing an Excellence endorsement, two Excellence grades would be preferable to three Merit grades.

Our national secondary school examination system actually rewards students for doing less work.

The acknowledgement of effort is lacking even within an individual paper. In a traditional marking scheme, every correct answer would contribute to the final mark, but NCEA is a standards-based assessment system with “top down” marking. 3 Therefore, confident students can take a gamble and jump straight to the harder parts of a question. It’s a risky strategy, but if it pays off, they may achieve Excellence having answered roughly a third of the paper. This flies in the face of the instruction printed on the cover page: “You should attempt ALL the questions in this booklet.” If you do well, then some of what you do could turn out to be a waste of time.

NCEA is a system that does little to incentivise students to put in maximum effort or to persevere if the results are likely to be sub-optimal. Until these problems are addressed, mediocrity will prevail. We welcome the Ministry of Education’s review of NCEA this year, and hope to be part of that discussion.

Dr Audrey Tan, Mathmo Consulting
21 March 2018

1   Ironically, NZQA in their utopian socialist bubble didn’t want us to compare students, but it’s happening anyway. Students compare themselves, universities have already come up with their own weighted metrics, and employers are learning that “E” grades on a Record of Achievement are actually higher than “A” grades.

2   Prior to 2013, each paper used to start with the recommendation “You are advised to spend 60 minutes answering the questions in this booklet.”, but not any more.

3   Anecdotally, not all markers appear to follow this methodology. Perhaps they too feel that any positive efforts should be acknowledged, even if ultimately ignored in the final score.