Outcomes, Evidence and Assessment

Let me start with an apology and a clarification. First, I’ve blogged about learning outcomes quite recently, and, although I think I’ve taken a slightly different tack here, there may be some repetition. Sorry. The clarification is for those colleagues who I promised that I wouldn’t say anything about what they said about this: I haven’t, and I promise that any resemblance is purely coincidental. No, this came out of a discussion with a colleague who was struggling with phrasing learning outcomes, and in particular trying to get past the idea of writing an outcome which was not just a description of the task, but rather a statement of evidence of a transferable skill: i.e. what had the students learned as a result of the task. I am starting, as well, with certain assumptions about learning outcomes. First, that they should be SMART (Specific, Measurable, Achievable, Relevant and Timed). Secondly, when successfully completed, a learning outcome should demonstrate to the teacher that the learners have learned something. 
Take these, for example:
1. Learners will be able to use present perfect.
2. Learners will be able to use present perfect to describe past experiences.
3. Learners will be able to use present perfect in five sentences describing their past experiences.
Outcome 1, essentially, is the thing you are trying to achieve. This is what you want students to be able to do. However, outcome 2 is better because present perfect has a number of different usages in English, and this is clearer.  God help me, but it is more Specific. 
Outcome number three, however, ticks all five of the sacred SMART requirements. It is specific, in that evokes a precise use of a particular grammar structure, it is measurable by simple inclusion of a number, it is certainly Achievable, assuming that the lesson is for Entry level 3 ESOL, it is Relevant, in that being able to describe what one has done in one’s working life is a helpful thing to be able to do, and if this were an ESOL for employment class, this would be easily something which would suit. I don’t need to explain the time frame, do I? 
So let’s say that, for the sake of argument, this is the outcome of a lesson. So far so good, right? I mean, it’s SMART, there is clear evidence of having achieved said outcome, it’s pretty good, right? They have demonstrated that they can do those things. 
By writing those sentences, however, does this mean that the learners will have demonstrated that they know how to use the grammatical structure? Can we assume with any certainty that subsequently the learner will be able to take that point and reapply it in another context? Essentially, is the grammar a known thing? 
This is where it gets challenging. In literal terms, the writer of the outcome is only claiming that they the learners can make those 5 sentences: not that the learners will know the grammar. I don’t think, as well, that anyone would really assume that they did know the grammar at this point, only that they are further along the path of being able to do it. If that’s the case, however, then what is the point of the super specific learning outcome? How does it help anyone? If we as teachers are acknowledging that this is a bit fake in terms of what the students have done, then where is the value of learners reflecting on this as a means of marking their achievement? Perhaps they, and we, might that they managed to do it but would like to work on the grammar a bit more, but then really we are thinking not about the evidence of the learning outcome, but rather the vaguer, more woolly aim, like outcome 2 above. 
But what about skills development? In principle a learning outcome should be a statement of the transferable skill learned. And taking SMART as our touchstone, a learning outcome can be easily formed for a skill like listen for gist, or read for detail. “Read a text and extract 5 details”. Except it’s not that simple. All you can ever say for sure that the learners will have evidenced here is that they can read one text and extract 5 details from that text alone. They have certainly had practice in that skill and be developing it, but you couldn’t say for sure that they will now be able to read any text appropriate for the level and extract 5 details. Because of the need to measure the achievement, a language skills outcome can only ever be a description of the aim of the task in one lesson, not a statement of a transferable skill. When you ask the question “has learning happened?” it all gets a bit tricky to define. 
What about pure communicative outcomes like “ask for information at the bus station” or be able to tell someone 5 things about jobs you have had” ESOL, and indeed ELT generally is blessed/cursed with the challenges of marrying up functions, skills, lexis and grammar in course design, and success at a communicative function can be achieved without the “right” grammar for the job. One of the challenges of getting students past the Entry 3 (B1 on CEFR) threshold in a target language setting is dealing with the fact that they can often get by pretty well. Thus success at speaking is hard to measure without taking into account the grammar bring used. An outcome like “tell someone 5 things about jobs you have had” could be achieved by pretty much any learner in any class at Entry 3 or above but may not use present perfect, or indeed any tense structure in English which refers to the past. “I work for 5 year at Batley Beds. I work now for Batley Beds.” gets the idea across, albeit inelegantly. 

So how do we write “good” learning outcomes for an ESOL class? Speaking personally, and very definitely outside my usual role, I don’t think we can. The nature of an “evidenceable” learning outcome necessarily restricts us to what is achieved in the classroom, and simply does not allow us to make general comments as to the wider learning.  A very wise man once suggested to me that rather than focussing on specific measurability, and to avoid producing fake evidence like “write 5 sentences using present perfect”a more honest and realistic phrasing would be “be better able to use present perfect to talk about past experiences”. This is perhaps an objective rather than an outcome, but it places the onus on the teacher to monitor carefully the students’ language production. This is, of course, what good teachers do: assessment of language production in a classroom is not simply through the achievement of learning outcomes, but a continual steady process. Language teachers, or at least the decent ones, don’t issue a task and then sit back and wait for the results to roll in before providing assessment feedback, but rather tend to patrol a room suggesting and monitoring students. Formative assessment is not solely through quizzes or (oh God, this makes my heart sink on an ESOL lesson plan) “Q&A”. It is a process which starts at the beginning of the lesson and which continues throughout. Using a selection of learning outcomes which purport to demonstrate learning is essentially false because in a language class you are always measuring learning in a hundred different ways. It would be impossible to realistically record all of this when planning, and very probably pointless. 
Measurable learning outcomes seem based in a model of teaching where learning occurs as a result of direct input, and an for me an ESOL class doesn’t work on these terms. Because the thing being taught is usually the same as the thing being used to teach, evidence, as suggested by Hattie, for the effectiveness of direct instruction supported by questioning is largely irrelevant: there is no point in direct instruction if the people you are instructing can’t understand what you are saying. No, rather, there is still a place in an ESOL class for collaborative work and scaffolding: development of language ability in adults is through using that ability in combination with finding out and only occasionally being told about it. Development of that ability is not always directly evidenceable, and even where it is, the reliability of that evidence is highly questionable.
Perhaps we need to turn our back on the input/output behaviourism of the learning outcome. Forget SMART and be a little more laid back. Unfortunately, this doesn’t fit in with the prevailing educational wind in post 16 learning in the UK. But then, one of the challenges of teaching ESOL in an FE context that we are a bit of a misfit, lauded and celebrated when colleges want to brag about their diversity, but in terms of funding, time tabling and classroom practice, we are a bit of a pain. But then I wouldn’t have that any other way. 
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s