From personal experience and talking to colleagues, I am aware of the problems facing graded lesson observations and I knew that the evidence is stacked against the process although I had never looked into it myself.
I have decided to take a look.
So what does the evidence say:
Reliability and Validity
Robert Coe (2014) explains that graded lesson observations are neither reliable or valid. On reliability he cites the Classroom Observations and the MET Project (2010):
- “If a lesson is judged ‘Outstanding’ by one observer, the probability that a second observer would give a different judgement is between 51% and 78%.”
- “If your lesson is judged ‘Inadequate’ there is a 90% chance that a second observer would give a different rating.”
On validity, he refers to Do We Know a Successful Teacher When We See One? Experiments in the Identification of Effective Teachers (2011) where head teachers and experienced teachers were asked to grade videos of teaching:
- “fewer than 1% of those judged to be ‘Inadequate’ are genuinely inadequate”.
- “of those rated ‘Outstanding’, only 4% actually produce outstanding learning gains”
- “overall, 63% of judgements will be wrong”
On a set of slides entitled Leadership for teacher learning uploaded in 2014 by Dylan Wiliam, citing Hill, Charalambous and Kraft (2012), it states that “achieving a reliability of 0.9 in judging teacher quality through lesson observation is likely to require observing a teacher teaching 6 different classes, and for each lesson to be judged by 5 independent observers.”
Dylan in an interview with Ollie Lovell (2018) went on to explain that the same reliability rating can be achieved by a teacher being observed in 30 different classes by the same observer.
Counterproductive and Damaging
Joe Kirby (2014) explains that grading lessons is counterproductive and damaging.
In his blog post Who’s afraid of lesson observations? (2013) he points to the number of Guardian Secret Teacher articles where teachers share frustrations with the system and also education bloggers writing on the topic. He quotes the pressures felt by almost 50 teachers with the graded lesson observation process.
It is well known that graded lesson observations cause the teacher stress as pointed out in this Guardian article, Graded lesson observations are a major cause of stress for FE teachers (2014). In the article we are told that the single most significant cause of disputes is lesson observations. Nerves are said to let teachers down, while others are performing for the observations, and the process has been used to bully and intimidate members of staff.
In my own personal experience those that get a less than good judgement feel like a failure, that they are not good enough and the first reaction from many is to think about leaving the profession. This is surely not a message that managers want to send to teachers.
There is also the other end, Good or Outstanding. At good most teachers are satisfied, they have ‘passed’ the process, they can move on. At outstanding most will tell you that they could not perform like that every lesson, there was too much work and preparation.
Some teachers have an ‘observation lesson’ ready for when they get the notification. Some have been known to use the same lesson for different observations, because they know that ‘it works’. I do not blame these teachers, they are playing the game. They did not choose to be a part of the game, but they know the rules and they are looking out for their own best interests.
In everything that I have read, ‘development of teachers’ is hardly mentioned. It seems to be assumed that this process is not developmental. In my experience it is not – at all. I can safely say that I have learned nothing from any graded lesson observation I have had. At the end I get a number and I know whether or not I have to do it again.
Given the evidence against reliability and validity and the widely accepted view that they are counterproductive and damaging, why do they persist?
Perhaps school leaders see it as a way to quantify teaching and learning, they know the proportion of teachers that are rated good or above. After Tom Sherrington (2014) saw a school leader do this he wrote “”how pitifully delusional … how toxic”.
Alex Quigley (2015) calls the process the “zombie of supposed school-improvement”.
David Didau (2014) compared ” a belief in the validity of lesson grading” to “a belief in witchcraft”.
Ross McGill (2018) told us that “40-45% of headteachers STILL IGNORE the evidence” by continuing with graded lesson observations which he finds “abhorrent”.
After reading the evidence it is hard to disagree with any of the above. One thing that has struck me is that almost everything I have cited is from 2015 or earlier. Here we are at the end of 2018 with countless teachers still facing the process and all they want to do is develop professionally and help their students learn more. When will the madness end?
Monk Maths 
Go find another school!
LikeLike
I understand that your response comes from your passion and strength of belief and values.
There are many positives about my workplace this is just one of the big negatives.
LikeLike
In order to understand why the practice persists, in the face of pretty compelling evidence, you perhaps need to get to grips with terms such as – managerialism, marketisation & austerity.
LikeLike
There are pressures that management face and they have to make difficult decisions to adapt to their circumstances.
But as David Didau says “Of all the imperfect methods of we could use to evaluate teacher effectiveness, [graded lesson observations] is the least perfect.” different ways can be found.
Bournemouth and Poole, City of Oxford College (Activate Learning), Calderdale College, New College Stamford, and College of North West London (now merged to United Colleges Group) in 2014 were all reported to ditch graded lesson observations, all of which have since had Good judgements from Ofsted.
LikeLike