{"id":3237,"date":"2020-07-09T10:00:20","date_gmt":"2020-07-09T10:00:20","guid":{"rendered":"http:\/\/sites.reading.ac.uk\/t-and-l-exchange\/?p=3237"},"modified":"2020-07-09T10:11:23","modified_gmt":"2020-07-09T10:11:23","slug":"introducing-group-assessment-to-improve-constructive-alignment-impact-on-teacher-and-student","status":"publish","type":"post","link":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/2020\/07\/09\/introducing-group-assessment-to-improve-constructive-alignment-impact-on-teacher-and-student\/","title":{"rendered":"Introducing group assessment to improve constructive alignment: impact on teacher and student"},"content":{"rendered":"

\"\"<\/a><\/h2>\n

Daniela Standen<\/strong><\/a>, School Director of Teaching and Learning, ISLI\u00a0\u00a0Alison Nicholson<\/strong>, Honorary Fellow, UoR<\/p>\n

Overview<\/strong><\/h3>\n

In summer 2018-19 Italian and French in Institution-wide Language Programme, piloted paired Oral exams. The impact of the change is explored below. Although discussed in the context of language assessment, the drivers for change, challenges and outcomes are relevant to any discipline intending to introduce more authentic and collaborative tasks in their assessment mix. Group assessments constitute around 4% of the University Assessment types (EMA data, academic year 2019-20).<\/p>\n

Objectives<\/strong><\/h3>\n
    \n
  • improve constructive alignment between the learning outcomes, the teaching methodology and the assessment process<\/li>\n
  • for students to be more relaxed and produce more authentic and spontaneous language<\/li>\n
  • make the assessment process more efficient, with the aim to reduce teacher\u00a0workload<\/li>\n<\/ul>\n

    Context<\/strong><\/h3>\n

    IWLP provides credit-bearing language learning opportunities for students across the University. Around 1,000 students learn a language with IWLP at Reading.<\/p>\n

    The learning outcomes of the modules talk about the ability to communicate in the language.\u00a0 The teaching methodology employed favours student\u2013student interaction and collaboration.\u00a0 In class, students work mostly in pairs or small groups. The exam format, on the other hand, was structured so that a student would interact with the teacher.<\/p>\n

    The exam was often the first time students would have spoken one-to-one with the teacher. The change in interaction pattern could be intimidating and tended to produce stilted Q&A sessions or interrogations, not communication.<\/p>\n

    Implementation<\/strong><\/h3>\n

    Who was affected by the change?<\/strong><\/p>\n

    221 Students<\/p>\n

    8 Teachers<\/p>\n

    7 Modules<\/p>\n

    4 Proficiency Levels<\/p>\n

    2 Languages<\/p>\n

    What changed?<\/strong><\/p>\n

      \n
    • The interlocution pattern changed from teacher-student to student-student, reflecting the normal pattern of in-class interaction<\/li>\n
    • The marking criteria changed, so that quality of interaction was better defined and carried higher weight<\/li>\n
    • The marking process changed, teachers as well as students were paired. Instead of the examiner re-listening to all the oral exams in order to award a mark, the exams were double staffed. One teacher concentrated on running the exam and marking using holistic marking criteria and the second teacher listened and marked using analytic rating scales<\/li>\n<\/ul>\n

      Expected outcomes<\/strong><\/p>\n

        \n
      • Students to be more relaxed and produce more authentic and spontaneous language<\/li>\n
      • Students to student interaction creates a more relaxed atmosphere<\/li>\n
      • Students take longer speaking turns<\/li>\n
      • Students use more features of interaction<\/li>\n<\/ul>\n

        (Hardi Prasetyo, 2018)<\/p>\n

          \n
        • For there to be perceived issues of validity and fairness around \u2018interlocutor effects\u2019 i.e. how does the competence of the person I am speaking to affect my outcomes. (Galaczi & French, 2011)<\/li>\n<\/ul>\n

          \u00a0<\/strong>Mitigation<\/strong><\/p>\n

            \n
          • Homogeneous pairings, through class observation<\/li>\n
          • Include monologic and dialogic assessment tasks<\/li>\n
          • Planned teacher intervention<\/li>\n
          • Inclusion of communicative and linguistic marking criteria<\/li>\n
          • Pairing teachers as well as students, for more robust moderation<\/li>\n<\/ul>\n

            Impact<\/strong><\/h3>\n

            Methods of evaluation<\/strong><\/p>\n

            Questionnaires were sent to 32 students who had experienced the previous exam format to enable comparison.\u00a0 Response rate was 30%, 70% from students of Italian. Responses were consistent across the two languages.<\/p>\n

            8 Teachers provided verbal or written feedback.<\/p>\n

            \u00a0Students\u2019 Questionnaire Results<\/strong><\/p>\n

            Overall students\u2019 feedback was positive.\u00a0 Students recognised closer alignment between teaching and assessment, and that talking to another student was more natural. They also reported increased opportunities to practise and felt well prepared.<\/p>\n

            However, they did not feel that the new format improved their opportunity to demonstrate their learning or speaking to a student more relaxing.\u00a0 The qualitative feedback tells us that this is due to anxieties around pairings.<\/p>\n

            Teachers\u2019 Feedback<\/strong><\/p>\n

              \n
            • Language production was more spontaneous and authentic. One teacher commented \u2018it was a much more authentic situation and students really helped each other to communicate\u2019<\/em><\/li>\n
            • Marking changed from a focus on listening for errors towards rewarding successful communication<\/li>\n
            • Workload decreased by up to 30%, for the average student cohort and peaks and troughs of work were better distributed<\/li>\n<\/ul>\n

              Reflections<\/strong><\/h3>\n

              Overall, the impact on both teachers and students was positive. Student reported that they were well briefed and had greater opportunities to practise before the exam. Teachers reported a positive impact on workloads and on the students\u2019 ability to demonstrate they were able to communicate in the language.<\/p>\n

              However, this was not reflected in the students\u2019 feedback. There is a clear discrepancy in the teachers and students\u2019 perception of how the new format allows students to showcase learning.<\/p>\n

              Despite mitigating action being taken, students also reported anxiety around \u2018interlocutor effect\u2019.\u00a0 Race (2014) tells us that even when universities have put all possible measures in place to make assessment fair they often fail to communicate this appropriately to students. The next steps should therefore focus on engaging students to bridge this perception gap.<\/p>\n

              Follow-up<\/strong><\/h3>\n

              Follow up was planned for the 2019-20 academic cycle but could not take place due to the COVID-19 pandemic.<\/p>\n

              References<\/h3>\n

              Galaczi & French, in Taylor, L. (ed.),<\/strong> (2011). Examining Speaking: Research and practice in assessing second language speaking. Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, S\u00e3<\/em>o Paulo, Dehli, Tokyo, Mexico City: CUP.<\/p>\n

              Fulcher, G.<\/strong> (2003). Testing Second Language Speaking. <\/em>Ediburgh: Pearson.<\/p>\n

              Hardi Prasetyo, A.<\/strong> (2018). Paired Oral Tests: A literature review. LLT Journal: A Journal on Language and Language Teaching<\/em>, 21(Suppl.), 105-110.<\/p>\n

              Race, P.<\/strong> (2014) Making Learning happen (3rd<\/sup> ed.), Los Angeles; London: Sage<\/p>\n

              Race, P.<\/strong> (2015) The lecturer’s toolkit : a practical guide to assessment, learning and teaching (4th<\/sup> ed.), London ; New York, NY : Routledge, Taylor & Francis Group<\/p>\n

               <\/p>\n","protected":false},"excerpt":{"rendered":"

              Find out more about the impacts following ISLI’s pilot of paired Oral exams.<\/p>\n","protected":false},"author":124,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0},"categories":[10,2,12],"tags":[601,602,598,71],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/posts\/3237"}],"collection":[{"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/users\/124"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/comments?post=3237"}],"version-history":[{"count":0,"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/posts\/3237\/revisions"}],"wp:attachment":[{"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/media?parent=3237"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/categories?post=3237"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.reading.ac.uk\/t-and-l-exchange\/wp-json\/wp\/v2\/tags?post=3237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}