Thinking about Assessment - Part 2

THINKING ABOUT ASSESSMENT (part 2) – A follow-up on Thinking About Assessment… Again)

Having decided that we were going to pilot the alternative assessment program, we had to inform students of our plans, and listen to what they had to say about it. We were ready to “abort the mission” in case of rejection. They accepted it with no reservations. Still, it was to my surprise that, at the end of the first lesson, one of them came to me and said (in L1, of course, as this is the beginner group) “See you next class… but you will only see me because you told us we won’t have to take that final test.” It took me a couple of seconds to grasp the meaning of what she was telling me. She went on: “I’m too old to suffer with tests. In my life, I’ve taken all the tests I needed to take… Now, I’m interested in learning!”

And that was what we needed to know that we were on the right path. The focus had naturally shifted from teaching and testing to learning. The learners had assumed their rightful place at center stage, taking control of the process. “Now, I’m interested in learning!”

Lately I’ve been following Adrian Underhill and Jim Scrivener’s blog on ‘Demand-HighTeaching’, and two of their questions really hit a nerve: How can we stop “covering material” and start focusing on the potential for deep learning? How can I shift my attention from “successful task “to “optimal learning”? Well, this was exactly what we wanted to explore in our “assessment quest”.

Anyway, back to my tale to tell… After the first evaluation of their oral performance, we decided to give them a weekly “assessment opportunity”. On week 4(of 10), the focus was “Listening”.

In the past we had been cautious of venturing into evaluating the listening skills with the adult groups. Adults are afraid of listening, terrified by its unexpectedness,   petrified by the possibility of failure.  Adults are interesting language learners; they bring a whole lot of baggage with them:

• Their beliefs, more often than not tainted by their previous language learning experience – usually their formal learning of the mother tongue (which they had already acquired in their childhood), with the grammar exercises, linguistic analysis, etc.
• Their personal history. Your student is most likely a self-respecting human being, a skillful professional, someone who undoubtedly has a lot to teach you, who can tell a number of success stories, and learning English is not one. At least not yet. 
• Their needs and expectations:  They ‘ve come to us because they want to be part of the world who can speak English. That is the question, isn’t it? “Do you speak English?” or “Can you speak English?” 

These learners, more than any other language learner, need to be able to speak, to communicate effectively! Well, communication implies a message that is sent and, consequently, received: Listening! How can we ever assess language learning without analyzing listening?  If they don’t understand what is said to them, how can they respond?

 Anyway, assessing their Listening skills, after no more than 12 (twelve) classroom hours, for most of them twelve contact hours. How do you do it? Preferably without any extraordinary acrobatic feat, just keeping it simple and structured, with the appropriate scaffolding, and making sure that the lesson is designed focused on enabling optimal learning, while providing you – teacher – an opportunity to assess whether the goals have  been achieved, and how far they have been developed.    

Here is the step by step:

1. We had previously explored the following exponents:

• What’s his/her name?  His/Her name is…
• Where are you from? I’m from…
• Where is he/she from?  He’s/She’s from…
• Vocabulary: countries

2. On the second lesson, I showed them a PPT with international celebrities… At first I showed a photograph and asked the questions ‘What’s his name?’ and ‘Where is he from?’ (before revealing the name and the flag) Here are some samples:

3. After two or three samples, I invited the students to ask the questions: ‘X, ask Y.’

4. Then, they worked on their books, which brought an information gap activity. Both students had pictures of six people. One student had information on three of the people (names and countries of origin), while the other had to look at a different page, where they had information on the other three. The structure and vocabulary was very much the same as my PPT had prompted: What’s his/her name? Where is he/she from?

5. Next, they were asked to look at an incomplete dialogue – again from the book, and work in pairs to predict what was missing.

6. After a couple of minutes, I asked them to listen to the dialogue and check if they had made the correct choices.

 7. Just before giving them the listening task, I replayed a recording from the previous lesson, and they repeated the names of the countries.

8. Next, I gave them the worksheet with the following task:

They heard the following dialogues:

 Dialogue I

A: Hello! I’m Luis, from Mexico.

B: Hello, Luis. I’m Akemi, from Japan.

Dialogue II

C: Hello. My name’s Charles. What’s your name?

D: Hi, Charles. I’m Mike. I’m from the United States. Where are you from?

C: I’m from London, in England.

D: Oh, yeah? I’m from Chicago.

Dialogue III

E: Hi, I’m Loretta. I’m from Sydney, Australia.

F: Hi, Loretta. I’m Jason. I’m from Australia, too.

E: Oh, wow! Are you from Sydney?

F: No. I’m from Melbourne.

They were graded both on the correct country, and the correct spelling of the country’s name.

As you may have noticed, nothing fancy. The PPT could have been easily substituted with good old flashcards. I used written and audio material from the book. My main worry was to make sure they were “comfortable” when they got to the listening task. The listening element was introduced with the dialogue (steps 5/6), but they had the chance to predict what they were going to hear before they heard it. It was safer that way.

They also had plenty of meaningful and varied practice on the target piece of language. The dialogues they heard were, in a way, familiar to them.  In this lesson, before getting to step 8, they were given at least three different opportunities to produce and listen to the names of the countries, as well as the language structures surrounding them.

Now, the important thing is that this lesson was, as the first one I described here, not designed to test. It was designed to teach, it had learning at its core. The assessment opportunity was created, but it only took as long as those three short dialogues – which, by the way, they heard only once.

So, once again, I invite your input. How do you see this project? Can you help us by suggesting activities and procedures we can use with these pilot groups? We are counting on your thoughts, your suggestions, your criticism… We are waiting for you!

Lueli Ceruti

Aligning learning outcomes, instructional strategies and assessment – an example using mLearning and Digital Images by Vinícius Lemos

In the October 2012 special issue of the ELT Journal – The Janus Papers – Stephen Stoynoff looks back at the changes in language assessment and analyzes the transitions under way. With the emerging dominance of a sociocultural paradigm in which learning is seen as a developmental, socially-constructed, interactive, and reflective process, classroom-based assessment will (pp. 527-528):

- integrate the teacher fully into the assessment process including planning assessment, evaluating performance, and making decisions based on the results of assessment

 - be conducted by and under the direction of the learners' teacher (as opposed to an  external   assessor); 

- yield multiple samples of learner performance that are collected over time and by means of multiple assessment procedures and activities; 

- be applied and adapted to meet the teaching and learning objectives of different classes and students;

-  integrate learners into the assessment process and utilize self- and peer-assessment in addition to teacher-assessment of learning; 

- foster opportunities for learners to engage in self-initiated enquiry; 

- offer learners immediate and constructive feedback; 

- monitor, evaluate, and modify procedures to optimize teaching and learning.

Likewise, the National Capital Learning Resource Center (2004) enumerates the following distinguishing features of alternative assessment:

1) Are built around topics or issues of interest to the students;

2) replicate real-world communication contexts and situations;

3) involve multi-stage tasks and real problems that require creative use of language rather than simple repetition;

4) require learners to produce a quality product or performance;

5) include evaluation criteria and standards which are known to the student;

6) involve interaction between assessor (instructor, peers, self) and person assessed;

7) allow for self-evaluation and self-correction as they proceed.

Hence, there’s been a growing interest in integrating classroom teaching, learning, and assessment. According to the Eberly Center for Teaching Excellence, Carnegie Mellon University, assessments, learning objectives, and instructional strategies need to be aligned so that they reinforce one another, as the image below shows.

Jon Mueller has a frequently updated webiste entitled Authentic Assessment Toolbox  that not only provides solid theoretical background on authentic assessment, but also offers a variety of tools in which the assessments are perfectly aligned with the learning objectives and the instructional activities. Cecília Lemos has also written inspiring posts on alternative asssessment in her popular blog Box of Chocolates.


Burger (2008) proposes the use of Outcomes-Based Education (OBE), in which the first step in planning teaching is identifying the learning outcomes; these outcomes then determine the teaching and assessment that follow so that the learning can be easily assessed via performance. Aligning learning objectives and instructional activities is not hard at all. The difficult part of the triangle is the assessment part, especially when it comes to oral performance.

How can the teacher possibly assess every student’s performance on an oral task designed to assess the attainment of a learning outcome that was developed by way of perfectly aligned instructional activities? 

How can learners be integrated into the assessment process?


 I’m going to propose an example based on an earlier post on this blog by my colleague Vinicius Lemos – mLearning and Digital images. What he describes in his post is an instructional strategy resulting from previous strategies in which students were taught the clothing vocabulary and the present continuous to talk about what one is wearing. I will attempt here to close the triangle above by spelling out the learning objectives that are implicit in the task and suggest a way of assessing students’ resulting performance.


  Learning outcome 1: Given a specific event, students will select and photograph the appropriate pieces of clothing to wear and describe their picture to their classmates using the present continuous and the correct indefinite article before each piece of clothing.

  Learning outcome 2: Given a picture with pieces of clothing that suggest a specific event, students will be able to ask questions using “Are you going to…” and vocabulary to talk about specific events.


 I suggest having students work in pairs rather than in groups to perform the activity, according to the outcomes above: Student A shows and describes his picture using the required language; student B asks questions to guess the event. Then they switch roles.

 Students can practice this exchange with two or three different pairs, as the teacher walks around and monitors their performance. The third or fourth time around, they are asked to record their exchanges, using their smartphones or, if available, the computer lab or a set of iPads. After they finish, they listen to their performance and engage in self-assessment of their part of the recording, according to a can-do checklist that can contain items such as:

 - I can name all the pieces of clothing. 
 - I can use the correct article for pieces of clothing in the singular starting with a vowel or consonant sound and no article for plural. 
- I can describe what I’m wearing using the present continuous. 
- I can name events such as school, work, picnic, wedding, etc. 
- I can ask questions about where a person is going based on their outfit. 
 - I can produce the language described above in a natural way, without too much hesitation or many long pauses to think. 


They judge their performance and if they think it needs improvement, they can record the conversation again, making the necessary adjustments. Then they send the recording to the teacher, who will use rubrics to assess students’ attainment of the two outcomes above. The teacher’s rubrics need to be similar to the students’, but should contain at least three levels of performance with appropriate descriptions.

Suppose each unit in the language program’s assessment cycle consists of five learning outcomes. Then each outcome can be worth 20 points. If the teacher conducts these types of assessments right after the instructional strategy, in such a way that the strategy is the assessment and vice-versa, at the end the student will have a grade on a 0-100 scale for oral performance.


Who needs a midterm or end-of-term oral test after that?


 The proposed assessment system here is in keeping with Stoyoff's (2012) list of characteristics of contemporary classroom-based assessment: it integrates the teacher fully into the process; it is conducted by the teacher; it can be one of a variety of samples of learnt performance collected over time, using multiple procedures; it meets the learning objectives, it integrates learners into the assessment process; it offers immediate and constructive feedback; and it allows the teacher to monitor, evaluate, and modify procedures to optimize teaching and learning.


 References:

Burger, M. (2008). The alignment of teaching, learning and assessment in English home language grade 10 in District 9, Johannesburg (Dissertation). University of South Africa, Pretoria, South Africa.

 
 
National Capital Language Resource Center (NCLRC). (2004) Assessing learning: Alternative assessment. In The essentials of language teaching. Retrieved from http://www.nclrc.org/essentials/assessing/alternative.htm

Stoynoff, S. (2012). Looking backward and forward at classroom-based language assessment. In ELT Journal, V. 66/4 – Special Issue: The Janurs Papers, pp. 523-532.

This post is cross-posted in my blog http://isabelavillasboas.wordpress.com/
If you want to read more about assessment and other TEFL issues, pay me a visit there.