The problem with grades | A conversation with Amarbeer Singh Gill & Jennifer Curran

Grades give us comfort and numbers, they feel concrete. But what if our obsession with quantifying learning is actually getting in the way of understanding it? Today, we're rethinking assessment. Hey everyone, I'm Shane Leaning.

Welcome to Education Leaders, the child-topping leadership podcast for school leaders just like you. I'm an organizational coach and in this show, I get to know the teachers, leaders and innovators making a difference in education across the world. Now, before we jump in today, I'm super thankful that this episode is supported by the International Curriculum Association and the University of Warwick. Stay tuned to learn more.

Now, I've got a treat for you today. I've got two guests. I've got Amabia Singh Gill and Jennifer Curran. Singh, as he's more commonly known, works in teacher education and currently works for Ambition Institute.

And you may have heard of him. He's written lots of great stuff for the Education Endowment Foundation, Chartered College of Teaching. He's also presented at many conferences. Jen is a research scientist.

She's a tutor at Ambition Institute as well. She mentors ITT students at Northumbria University and is super passionate about evidence-informed PD that is really digestible for teachers. Now, together, they've been doing some really interesting exploration of assessment. And that's what we're discussing today.

So let's just jump right into the conversation. I think first it's important to separate two things. So there's attainment and there's progress and the way we deal with them is different. So when we use grades, we are communicating like a shared meaning.

So if a student gets a grade eight in maths at GCSE, what we're saying is that this student is highly competent at GCSE level maths. When we then take that same system though and use it in a different context. So if we do an end of year test in say year eight and a student gets a grade like five, for example, in year eight, it's not clear what the meaning is at that point. So it might be that the meaning is when the student sits the actual GCSE, we expect them to get a grade five.

Or it could be that if they were to sit the GCSE today, they would get a grade five or like something else entirely, right? That clarity is lost. So grades are really helpful when there is a shared understanding because that's what they've been created for. In terms of scores, the challenge of scores is they give us a full sense of security, right?

So a student could get 65% on a test, but if they were to sit the test like slightly earlier in the day or slightly later in the day or a day beforehand or a day afterwards, they would get some variation, right? So athletics is a really good example of this. So for example, Usain Bolt doesn't run his world record time every time he sprints. He's within a margin of error, right?

So give or take a few, in his case, milliseconds, we can say actually at 90, 95% of the time, he'll be within this sort of range of values. And really, when we think about scores, we should be applying a similar sort of thinking. So a student might have got 65% on that given day, but that point represents a value within a range that they're likely to get on any given day. So it's not to say that like, you know, don't use scores or don't use grades, but it's one of the things that when I was using them in my practice, I hadn't really considered these things.

Like I had been lowered into a full sense of security when using scores. Like, you know, this student is definitely better if they've got 65%, their students just got 60%. Well, actually, like they're probably within each other's margin of error. So that's when it comes to attainment.

When it comes to progress, like this is a whole different ballpark. So you could get two students get the same grades. Two students have got a grade eight, but they could be completely opposite ends of that boundary. And the bottom student could make loads of progress, but still remain within the same grade boundary.

And so on the surface of it, it doesn't look like they made progress. Whereas you could have a student who's like on the bottom end of grade eight, and then another student is on the top end of grade seven, and they can make really minimal progress, but it will look like they've made massive gains because they've jumped above that grade boundary. And so it gets really tricky because grades just aren't granular enough for us to figure out progress in a meaningful way. And then when we come back to scores, again, because we're comparing ranges of values, it's not something you can really do.

Like, when we get that point score, it's a point within a range, and then to compare and arrange with another range, like it's just really tricky, if not impossible. Like Becky Allen, also I'm post Professor Becky Allen to this because she's written a fantastic blog. A few years ago, if anyone interested in finding out more, please yeah, Google Becky Allen Progress, and it will come up. Cause she does it in a lot more detail than I can do in a few minute explanation.

I absolutely love what you're saying. These conversations always blow my mind because I get used to my context. So I'm in China as well. So scores, they love scores.

Like lots of schools love scores here. And it's usually out of a hundred. There's no like usually ABCD. It's usually just scores out of a hundred.

Or in the Gaokao, the big exam, you know, maybe out of 650, but it's points, right? It's scores. And yeah, trying to kind of work. Oh, okay.

So last time I got 83, but you know, this time I got an 85, so I'm doing better. But of course you're talking about moving within a range, which makes much more sense. I'm just interested to follow up. Like, does that mean you have to assess way more frequently?

Does this mean you actually have to give a lot more grades so you can work out these averages? Cause at first I thought you were anti-grades. Now I'm changing my mind. So I'm definitely not anti-grades.

I'm pro-grades being used in the right context. Is it like, get out of clause there. It's not to say like, so we do need scores and we do use grades. There's not a problem with using them with grades as long as there's a shared like meaning.

So if you're communicating with stakeholders, students, parents, teachers, does everyone involved know when you communicate that grade, what it means and do they have that shared understanding? And that will likely vary from school to school outside of, you know, national standardised exams. You'll likely vary from school to school because they are sitting different exams. They've been working from different curriculums, like those sorts of things.

And so that shared meanings are important. In terms of scores, again, like do students understand? So if, yeah, even your example, if a student got 83 one day, or 85 another day, like do they understand that it's difficult to say that they've like got better, probably haven't gotten worse, but it's really difficult to say they have got better. And that also assumes that the content is very similar from each test.

There's lots of factors there. So the challenge that Jen and I faced with the article is, there aren't any easy answers. Like we actually wrote the article thinking, oh yeah, we'll be able to give colleagues something really easy digestible terms take away and using the practise. And we probably left ourselves with more questions by the end of it, then we definitely anticipate starting off with and probably giving colleagues more conundrums to solve themselves, to be perfectly honest.

These are really important questions for us to be asking though, aren't they? And really almost good conundrums to present ourselves with because that's the reality of the complex work we do, isn't it? And you can shy away from it and simplify it into, you know, a little scale and just try to kind of la, la, la, la, la. Let's not talk about this.

And, you know, let's just look at that data on a whole set, or you can start to look into this. I wonder, Jen, what your thoughts are on what we just talked about, or, you know, is there any other kind of alternative ideas that are buzzing around your head? I think there's a couple of things that are really important to build on what things should. I think when we're thinking about shared meaning, there's something really important just about the stakes that we're assigning to these things.

So you asked, like, do we just need more of this assessment? And really good teachers are doing some sort of formative assessment all of the time. They're not giving out necessarily a number all of the time, but what they are doing is just constantly assessing, okay, my pupils are confident with that. They know how to do that.

They don't know how to do this yet, or this is where they've got a misconception. And those little bits of constant formative assessment and the reaction to them as well, I think is what's really key here. So when we look at sort of things that have been in vogue in schools over the years, like the idea of formative assessment people are pretty confident with. We should be doing some of this.

We should be asking questions. Some schools are particularly into the format of the question. So, you know, whether it should be multiple choice, whether it should be an online survey, where that data goes. And actually all of those things are kind of secondary to, are you doing it?

And what's the teacher doing with that information? So if you're just asking the formative assessment questions and then carrying on with your lesson plan, doesn't have a huge amount of meaning. But if you're using this formative assessment questions and then doing something different and then responding to the information that you get back, it is assessment, but it's not high stakes. It's not something that's gonna have to be kind of submitted on a form or shared more widely necessarily.

It's what's informing the teacher constantly. And I think the best teachers have that kind of practice so embedded that they probably don't realise how much they are actually doing it. But there are hundreds of those little moments with an lesson, like whole class, small groups, individuals just starting to do that kind of, okay, do they know this? And if they know this, where do I go next?

And if they don't know that, what is it that's confusing them? And how do I unpack that? Yeah, I think that's the really important part of getting some more meaning from this information so that they're not just kind of moving forward like you say, the number out of a hundred, but pupils are moving forward and they're really straightforward. This is what I need to look at next.

This is my kind of next little step. Yeah, it's that classic thing we do when we start talking about assessment is quickly jumping to the idea of grades and scores and tests and we can't help ourselves because it's also easy to conceptualise, but you're so right, it's that ongoing formative assessment. I wonder then like schools that are trying to grapple around that area then about these targeted, formative assessments, like what should school leaders be doing to kind of help their teachers to get to that point of feeling more confident with that approach? It's a really good question.

I think generally schools are getting better at sort of grounding their practise in what we know about cognitive science, for example. So understanding memory and understanding retrieval and how that works in order to set up kind of meaningful retrieval practise. But then I think there's a balance here of you need really effective professional development to give teachers some strategies for formative assessment and show them what you want them to do with formative assessment. What you don't wanna do is make that higher stakes and make it something that you're gonna go in and observe or take off or look at lesson plans for when people are doing a certain thing because it's not the nature of how doing that really effectively works.

And so I think there's a really tricky balance here between like, we want to give our teachers effective strategies that we know work, but we also want to empower them to apply them in the way that is most effective for the students in front of them. That's a really tricky balance, isn't it? Is something that I find myself just bothered with a lot of the time is that balance between, yeah, what we know should work between the context of the teachers. I wonder, Singh, have you had any experience of kind of almost kind of navigating that and helping teachers through this kind of thinking?

Yeah, and I think actually times where I've had really fruitful conversations ties in something that Jen mentioned in the article, right? Which was around leaders being really clear themselves on the rationale for why they are doing the things they're doing around data on assessment and also sharing that. So often leaders can have a really clear purpose, but like for various reasons and all of them understandable that doesn't always get filtered down to colleagues. So being really clear on, we are doing this summative assessment and these are the reasons for it.

Or we are gonna focus on formative assessments and these are the reasons for it. So that's the first thing is like around rationale. And the second thing that I think has really changed my thinking on this and facilitate some of the conversations is framing it in terms of opportunity costs. So we have a fine amount of time and resources and in fact, schools are very time and resource poor.

So the time we then spend administering, marking, grading, analysing summative assessments can be quite extensive. And that's also time that we can't spend doing something else, whether that might be planning formative assessments, planning lessons, constructing really good retrieval practice, thinking about really good examples, all of those other things, which are likely to facilitate learning. So the more time we spend trying to like measure learning, there's a trade off between less time being able to be spent on facilitating it. So there's two things, like the idea of having a rationale and communicating it clearly and also the idea of opportunity cost as well.

Have we got that balance right in terms of the trade off between the time we're putting into it and the value getting out of it? This is a really helpful framing for me. And I guess what you're saying is maybe there is no exact scientific balance between how much you measure and how much you do, but actually it's a conversation that you need to have. Like when you're thinking about implementing something talking with your teams or talking with your teachers about what that balance should be and what the opportunity costs are involved with taking on or taking away.

For sure. And one of the values of talking with a team is, so my background is in secondary maths and math teachers will always say, things are different in maths, sometimes rightly, sometimes wrongly, but they're definitely different. It would be silly for me to assess in the exact same way that a PE colleague or a DT colleague or an art colleague or an English colleague might do, it just wouldn't make sense. There might be principles that we can share between the two subjects, but there are fundamental differences.

The impact of a grader marking a maths assessment, it's not gonna change between one person marking another person. Whereas if you are a more subjective discipline like English or art, there's much more variability between markers. So how are we accounting for that? Are the stakes high enough within our school that we need to account for that?

Jen's a primary colleague and things will look different in her setting as well. So how are we accounting for the subject disciplines, the phase specificity? All of those things start to come into play and you're right that like the key thing that unlocks that is having those conversations before we start to roll things out, even perhaps piloting them based on conversations and then reflecting and refining them before we start to scale up afterwards. I think there's something here as well about the inferences that teachers and leaders are making from the data that they're currently collecting.

So if there are existing summative assessment practices, which that's what I said there are in pretty much every school, right? Everybody's doing something, just taking the time to think about, right? When we do our data drop, this is what we collect. Like we could all see the numbers on the paper, but actually what inferences are the senior leaders making from those numbers?

What are we saying that that means? And how have we decided that? And is that the right set of inferences to be making? I think it's very easy to feel like quantitative data is very concrete.

This is what it tells us. So it tells us they got 83. Okay, but actually what does that mean? And what does that mean for us as a school in terms of where we should go next?

I think taking away a little bit of the certainty from that summative assessment and starting to think about like actually, what inference does that mean as a school? What does it tell us about what we're doing well? What could improve on what our next focus might be rather than it just being a sort of, this is our data drop, this is our practice, this is what we do. Like the conversation that happens after that data drop, I think is really important.

I want to take a moment to tell you about this event from the ICA. It's called the Learning Effect, Play Paradox Passion. And I would encourage you to join the International Curriculum Association for its two day event, which is gonna bring school leaders and educators together for engaging keynote presentation and a diverse range of teacher led workshops. In this day, you'll get to explore and examine how thoughtful integration of play, paradox and passion can create transformative learning experiences all under the guidance of International Curriculum Association professionals.

This is taking place in Amsterdam in the Netherlands on the 20th and 21st of March, 2025. And you can find the tickets and more information using the link in the show notes. You know, recruiting and developing great teachers is one of the biggest challenges we face in international schools. That's why I'm excited about the University of Warwick Center for Teacher Education.

Their QTS and PGCEI with QTS programs are specifically designed for international schools combining online learning with hands-on classroom experience. Check out the link in the show notes to learn more how they develop teachers in your school. In my context, I mainly work with international schools. So not only have you got the different subjects happening, but you've also got very different teaching approaches.

For example, you know, I'm in China, you know, so imagine the difference of a teacher who's been trained in China to a teacher who's been trained in the US, for example, vastly different approaches. And of course that means different approaches to assessment. And the idea of a leader kind of coldly looking at the grades or like looking and then making, as you say, inferences, quick inferences about what happens. I like your call to action to kind of maybe sit with uncertainty a little bit more.

But there's a balance, right? You're not totally giving up control. Would those grades and those data start a conversation? You know, is that how you think would be a useful framing?

Or does it come in later? Does the teacher start a conversation with their observations? Yeah, I think there are definitely layers to it. One of the things that I found when I went into a school as an assessment lead and adopted existing assessment practices in the school, people would submit data to me and I'd say like, what is that data based on?

Because it wasn't a standardized test. I know even in a primary context, there are lots of standardized tests that one could choose to use. And in this particular example, it wasn't. It was like teacher assessment data.

And then starting that conversation with, okay, like, where's this come from? How have the teachers made these decisions? Teachers were quite surprised and almost like a little bit defensive of, you know, do you not trust my judgment? And we had to really kind of open that conversation.

It's like, it's not about suggesting there's anything wrong with the data you submitted, but it's about understanding where it's come from. And, you know, if there isn't moderation practices in existence, for example, if everybody's just looking at their own books in their own classroom, then how do we know if this is equivalent from reception tier six? How do we know if this is actually one data set or if it's just a lot of quite separate smaller data sets? So the conversation there had to start from what is this data based on?

And then once you've got it all together, then yeah, it becomes like, okay, so what patterns do we notice? What are we interested in from this data? What are we going to do next? Rather than just, we've got X percentage of pupils at this point.

What you've just described, Jen, you know, the idea of approaching a teacher and talking about their data and that defense mode kicking in. I think it's something that so many people resonate with, but I like your framing. It's almost inviting them to notice, like what do you notice about that? And what if we were to compare it in some way?

Like what would you notice then? And then the question, what do we do about that? Rather than, I've seen this, this is what that means. It's too easy to jump to a conclusion, isn't it?

Very, very quickly to drop at a conclusion and then drop in an interventionist style to go, okay, what are you going to do to close that gap rather than what does the data tell you first? Because maybe the data doesn't tell you exactly what it first seems. And when teachers are doing formative assessment really effectively, if you do start a conversation about, you know, let's think about this group of pupils within the data, what do we know about them? Then rather than a teacher feeling defensive of what they've put in, they're actually able to share their expertise and to say, this people's really confident in this area, we're working on this, this is what's next for them.

And a lot of that information they'll already have because they are the experts in the pupils in their classroom, they know what they know, but that's not necessarily reflected just in that raw data set. And so having the space to have that conversation sometimes reveals that a lot of those things are already happening. Like you say, rather than going in with a, what are we gonna do about this group? And I think that notion that like Jen just shared there, like ties into, again, that's something else that she mentioned in the article, which was like teachers having a really solid understanding of like the way this subject progresses.

So that's when you are doing those formative assessments, be that before introducing some new content, be that after introducing some new content, like you have this really clear picture of your head of that learning journey for those students and you can figure out, oh, they're like, they're roughly here, this is where I need to take them next, or then they're not here, this is where those gaps are and how I can fill them in. I think the challenge here is, so as a maths teacher, I will see my students, I'll have a handful of classes, they'll see them multiple times a week. So I'd be able to build that picture really quickly. And for me as their teacher, it's much more meaningful than necessarily the previous year's end of year assessment data or the previous term's end of year assessment data, because it doesn't tell me about like what they know and what they're able to do.

So having that picture allows me to answer those questions, like what can they do now? What do they know? But that's obviously a challenge if you are teaching a subject, you know, like philosophy, for example, where you really don't see your students all that often. And so taking that time to build that picture becomes much more difficult, but it hinges really on supporting colleagues to like know their subject, like inside out, so they can picture that journey those students are on and then be able to pinpoint, right, this is where they are now, this is where I need to go back and fill in the gaps, this is where I'm taking them to and supporting them on that process.

That almost comes full circle to what you were talking about at the beginning about how do they use grades in their subject and allowing them to know how they work, because yeah, an expert teacher will know that journey, will have seen that journey play out many times and will have developed a good intuition through their formative assessment practices to understand where they are on that journey. But that requires a real confidence in leadership to be able to know where the line falls between you holding the grades and the data or the scores and them as a professional. In our sector across the world at the minute, I know in the UK and certainly across internationally, there is this tension that is playing out in education at the minute between what we can quantify really in a really kind of hard way and where the teacher sits as the holder of the social science that they're delivering as an educator, right? Probably the phrase of this podcast, right?

There aren't any easy answers. So the EEF in their implementation guidance talk about data, they dedicate a couple of pages specifically to this. And one of the points they make is we as like humans, we like concrete things and so numbers feel concrete and so we tend to put a larger emphasis on them and perhaps they deserve. We want to really be clear here.

It's not because we're doing anything wrong, but it's just how as humans, like our nature is to look for concrete things. And in an environment like schools are in, where it's time and resource pool, we will often reach for those things because they're concrete, they're easier to quantify, like they're easier to make sense of. The challenge then comes as you say, Shane, like trying to accept and live with that uncertainty, live with that ambiguity, which can be uncomfortable and perhaps unfamiliar, but also then understanding that is like one part of data that sits within this broader sphere that we should be using, like the EEFs were triangulate. So it might be that actually we do use some of that assessment data, but we might compliment that by going in and speaking to students or speaking to staff or doing some lesson observations and learning walks.

When we start to do all of these things and or others, we start to build again, this broader picture. Like in a similar way, we just spoke about how as teachers, we have this idea of student learning in our head and we're on that journey. We can start to build this idea of the school journey in our head and where those teachers are in their professional development journey and then use all these bits of data to find out what professional development is going to be most supportive to my teachers at this current point in time. That's really helpful.

And I must say also a massive admission from a maths teacher to say numbers aren't everything. So, you know, well done on that. Yeah, definitely true. Well, this has been great.

And I wonder then thinking about there's a leader who's listening. They've had a good exploration. I think this has been a fascinating conversation. But let's say a school leader was just coming and saying, look, this is really blowing my mind and maybe we're doing things wrong here or we're leaning too heavily on a certain approach.

Like, where would you advise a leader to start when thinking about their approach to grades, to scores, to progress in the school? I think the initial question and it sounds like this example person is perhaps already doing that but it's evaluating your current practices and it's not just thinking like, what do we do? But it's thinking what meaning do teachers currently give to the practices that we have at the moment and what meaning do pupils currently give to the practices that we have at the moment? So are those, for example, half-termly data drops used in conversation with staff?

Are they communicated to pupils and parents? Is there a shared meaning? What do each of those layers look like at the moment? And then that gives you space to think about, is that how we want them to look?

And I think the other place is really to focus in on curriculum. So it feels like taking a slight step away from assessment, but it's not because of the link that Zing's made really clearly. Like, if teachers are very confident in their own curriculum journey, that allows them to make that really meaningful assessment. So I think it's also just taking the space to look at your curriculum and how stable that is, right?

Because one of the challenges that we're all working with is that the curriculum changes a lot. And so you have really experienced teachers who might have taught a subject for a very long time, but they probably haven't taught the same curriculum for that whole period. And so we're also just having some space to understand the curriculum journey and whether it is clear for staff where pupils are in that process and what's next for them and what's come before so that they can make those really meaningful assessments. I think they're the two places that I'd start.

That was incredibly helpful, Zing. Do you have anything to add in there? Look, it's echo what Jen said firstly. And secondly, like, we're really mindful how some of the thoughts we shared might land and like the internal process of, you know, a question like, am I doing things wrong?

Do I need to do things differently? Like, these are good questions to be asking yourself, like, in schools we owe it to our students to think, you know, are we doing the best possible thing for them? But if anyone is listening and thinking, you know, I haven't quite understood this idea. We can't do these things justice in a 30-minute podcast.

Jen and I are really lucky that we get to spend days, weeks grappling with some of these ideas. So if anyone is out there and they're sort of thinking, you know, I'm really not sure where I sit with this right now, like, please give yourself some grace. It took us ages and we've definitely come a long way now. Thinking like this definitely had some like moments of self-reckoning as well.

But like, do take that time and give yourself that time to go away, to think and to read. I mentioned Professor Becky Allen's blog earlier. It's a great starting point. There's a book by Daisy Christodoulou's book called Making Good Progress, another great starting point for thinking about some of the key points of assessment theory.

But yeah, like, we always do the best for our students. So yeah, do give yourself that grace and that time. This chat with Singh and Jen really did get me thinking on how we become too dependent on numerical data when we're assessing progress. I loved that emphasis on understanding the limitations of grades and scores.

You know, it's not that they're meaningless, but they need real proper context to be useful. A few key takeaways for me was that distinction between attainment and progress and that we need different approaches for each. Also, that formative assessment is happening constantly in good classrooms. It's not just about formal tests.

And of course, that importance of creating space for meaningful conversation around assessment data rather than jumping into conclusions. I wonder, did it get you thinking? How could you start evaluating your current assessment practice? What meaning do teachers, students, parents, give to your current approaches?

Is there a shared understanding? And how confident are your teachers in your curriculum journey? I'd love to hear your thoughts. I know Singh and Jen want to.

You can find them online. I've included some of their social links in the show notes and also a link to their article where they discuss this in a lot of detail. Education Leaders is hosted by me, Shane Leaning. Thanks to the show editor, Pete McGill from the original Music by Guillermo Silva.

I'm really happy that you're tuned in today. Thank you so much. And as ever, if we don't speak before, I'll see you here next week. If you're interested in the work of my show's partners, the University of Warwick and the International Curriculum Association, head to the show notes to get links to learn more.