Is it ok to have four issues not five in an LSA issues section ? (teacher doing Delta)
‘Look, I wrote lots this week’ (iGCSE student)
Can I see ITI’s Delta results anywhere ? (prospective Delta applicant)
and my response to all of the above was … discomfort. They are the wrong questions (and yes, I know one of them isn’t a question, but they were all looking for a response). They are all reasonable questions, but when they get asked too often and other things don’t get asked, then they are the wrong questions.
And I’ve been noticing how often I get asked them since I read a blog post by Seth Godin a week or two ago – about it being easy to measure the number of words in a book, but that not telling us very much of use about the book and the temptation to do (or notice, or rely on) the thing we can measure because we can measure it. I think it was the title that had the most impact – measuring nothing with great accuracy. The further I got through writing this though, the more I think there is a difference between measurability and countability and he is talking about the latter, but the point still works – in fact it becomes more interesting.
Lots of people ask how many / how much questions about Delta assignments and we encourage them in this with guideline documents that make suggestions about word counts for sections and numbers of issues or activities or sub section headings, so why worry about being asked if an issues section can have five issues not four ? The numbers (and suggested word counts) should help a person get a feel for overall structure that adds up to a concept of what an assignment must contain, but the most important things in the concept are not countable. They must be measurable or we could not assess the assignment, but they are often quite hard to measure and they have to be measured against criteria, not counted (mostly, as the word count criterion does lead to counting, but none of the others do). To explain how an assignment can be assessed as having ‘analysed the specific area with accuracy, identifying key points’ (3a) takes an outline of what you would expect to be included and examples which would differ for any given focus and a comment on the balance of what is and isn’t there (some of which is partly countable in the sense that you at least have to have enough of it, which is perhaps part of what muddies this further), whereas things you can count clearly and easily are simpler to identify. Perhaps the word count is the only purely countable thing. Other things like ‘it needs reference to reading’ might be partly countable. And we lead people to thinking about numbers by making numerical suggestions. One in text citation is never going to be enough to carry an LSA, so we suggest a minimum number but neither might ten be enough if all ten are references to blog posts like this rather than something with the academic weight needed, or if they are from reading on areas that don’t connect to language or teaching, or don’t connect to the assignment focus. Numbers are a good idea in that less than six suggests there will be a problem, which gives a newly starting out on the Delta teacher a concrete initial goal, but there are many caveats to go with that. More is usually better, but not to the point of losing your own voice in it. How they are used can be as important as how many there are. So being asked if having 4 issues instead of 5 is ok comes across as slightly worrying in that it isn’t a matter of whether there are 4 or 5 (or even 3 or 10) issues that is going to contribute to the assignment passing or failing, but whether they are convincing, have a real feel of a teacher’s experience, tie back into things said in the analysis, are illustrated and demonstrate some awareness (be that directly or through reading) of different contexts and levels. But numbers do help. If someone hasn’t written one of these before and has only seen one example (as all centres are quite parsimonious with examples for good reason), then it gets across the message about sub headings (if you have written 700 generalised words about learners that cannot be easily divided up with sub headings there is a high chance it doesn’t have many or even any of the features I’ve just listed that ensure you meet criteria).
And the student who said ‘look, I wrote lots’ knew as well as I did that the ‘lots’ might still not match up against the higher band he wants to achieve, but at least one of the more basic requirements of the task was met. So again, as above, within the context of the whole, the counting works as a useful adjunct (as long as it is an adjunct).
Sometimes though it is not even useful as an adjunct. The Delta results we just got back (from December 2013 submission) were great, especially in that we got a lot of higher grades for Module Three this time. Though I should rephrase that, our teachers got the grades, with some draft marking help from the tutors. I’m chuffed to bits for the teachers and very proud of the team who draft mark on Module Three as that balance of showing someone what they might do without telling them or starting to do it for them, of showing them how to look at something in a different way without making them feel they have failed in a first attempt and thereby demotivating them is tricky at the the best of times, but within Delta perhaps trickiest on the extended assignment. But I also remember this time last year when the percentage of merits was a lot lower and there wasn’t one distinction (let alone several) and it was the same team, with the same ethos and largely the same structure and resources in place. I add to courses and systems all the time and I think we get better and better at all elements of supporting people through Delta, but there hasn’t been a sudden and dramatic change in our approach to Module Three, just gentle on-going adjustments and a year more experience. Lots of factors affect any batch of results. The biggest one being the focus of the teachers concerned. Some courses are a lot less successful grade percentage wise if you count things, often due to no fault on the part of the teachers – people get married, promoted, made redundant, have lots of other good and bad things happen in their lives. Most of the people we take on the courses prove able to succeed given enough time and determination, but one or other of those can often take an unexpected body blow for all sort of reasons. If we publish our results I think we should publish them all the time, but then we would be inviting people to judge us on them and that would mean we think they show what we can do. I think they show what individual teachers could do at a particular moment in time, but what they don’t show (and what is most important) is how much difference we made to any individual and what they were able to achieve. So they don’t show what we can do.
A sentence from The Talent Code stuck in my head, from a football coach ‘My greatest challenge is not teaching Tom Brady but some guy who can’t do it at all, and getting them to a point where they can. Now that is coaching’. We help some people who are largely ‘Delta ready’ with a good idea of what they are taking on, plenty of developmental support in their background and other Cambridge teaching qualifications under their belt. They often only need a gentle touch with draft guidance (though deserve the same amount of attention and support to maximise potential). Others have much more difficulty working out how to cope with the writing style, showing they have applied theory to their decisions, making the decisions (often at some point on any course someone howls in frustration on the forums that they have never had to design a course before and they can’t work out where on earth to start), prioritising (now I have got all this data, what am I supposed to do with it ?). What matters is that we make a difference for people (getting them the highest grade they are capable of). But the results don’t show that. They don’t show how many people went from finding the whole thing quite confusing to achieving an effective academic voice, finding priorities in the data they collected and making a coherent whole out of the sections overall and getting a pass. They don’t show who would have got a pass if simply in possession of a syllabus document and the Cambridge advice, but suggestions from draft marks helped them achieve a higher grade. They don’t show what people learnt or how long it took them to do so. We can’t count this. Perhaps if we had enough before and after tests and assessments in place we could measure it (though it would be more qualitative than quantitative, with the system as it is at present, the only thing that comes to mind that exists within the course / system as it stands is the forum questions, how many, about what and how far they are from where they actually need to be, but that wouldn’t take account of those who lurk).
You can count results.
But you can’t see how much people learnt.
And that comes up again and again in blogs I read of teachers whose writing I enjoy, very often as the reason why they are leaving the profession.
I shall count my blessings (yes, that was intentional) at least the blessing of not – yet – being in a sector where you are judged primarily on things you can count.
The morning after I posted this the mailing from Stephen Downes (the 5th subject in there) said ‘counting is fraught with difficulties’, though I excuse myself from having spotted that as it seems to require more familiarity with maths than I have and now, (some 24 hours later) in McGonigal’s book Reality is Broken I’ve just read the quote from Kelvin ‘If you cannot measure it, you cannot improve it.’. I guess I have to come back to measuring and counting soon and try and reconcile some of what I said in this post with at least the second of those (think I may just let the first lie ..).
sallyhirst 