Introducing group assessment to improve constructive alignment: impact on teacher and student

Daniela Standen, School Director of Teaching and Learning, ISLI  Alison Nicholson, Honorary Fellow, UoR

Overview

In summer 2018-19 Italian and French in Institution-wide Language Programme, piloted paired Oral exams. The impact of the change is explored below. Although discussed in the context of language assessment, the drivers for change, challenges and outcomes are relevant to any discipline intending to introduce more authentic and collaborative tasks in their assessment mix. Group assessments constitute around 4% of the University Assessment types (EMA data, academic year 2019-20).

Objectives

  • improve constructive alignment between the learning outcomes, the teaching methodology and the assessment process
  • for students to be more relaxed and produce more authentic and spontaneous language
  • make the assessment process more efficient, with the aim to reduce teacher workload

Context

IWLP provides credit-bearing language learning opportunities for students across the University. Around 1,000 students learn a language with IWLP at Reading.

The learning outcomes of the modules talk about the ability to communicate in the language.  The teaching methodology employed favours student–student interaction and collaboration.  In class, students work mostly in pairs or small groups. The exam format, on the other hand, was structured so that a student would interact with the teacher.

The exam was often the first time students would have spoken one-to-one with the teacher. The change in interaction pattern could be intimidating and tended to produce stilted Q&A sessions or interrogations, not communication.

Implementation

Who was affected by the change?

221 Students

8 Teachers

7 Modules

4 Proficiency Levels

2 Languages

What changed?

  • The interlocution pattern changed from teacher-student to student-student, reflecting the normal pattern of in-class interaction
  • The marking criteria changed, so that quality of interaction was better defined and carried higher weight
  • The marking process changed, teachers as well as students were paired. Instead of the examiner re-listening to all the oral exams in order to award a mark, the exams were double staffed. One teacher concentrated on running the exam and marking using holistic marking criteria and the second teacher listened and marked using analytic rating scales

Expected outcomes

  • Students to be more relaxed and produce more authentic and spontaneous language
  • Students to student interaction creates a more relaxed atmosphere
  • Students take longer speaking turns
  • Students use more features of interaction

(Hardi Prasetyo, 2018)

  • For there to be perceived issues of validity and fairness around ‘interlocutor effects’ i.e. how does the competence of the person I am speaking to affect my outcomes. (Galaczi & French, 2011)

 Mitigation

  • Homogeneous pairings, through class observation
  • Include monologic and dialogic assessment tasks
  • Planned teacher intervention
  • Inclusion of communicative and linguistic marking criteria
  • Pairing teachers as well as students, for more robust moderation

Impact

Methods of evaluation

Questionnaires were sent to 32 students who had experienced the previous exam format to enable comparison.  Response rate was 30%, 70% from students of Italian. Responses were consistent across the two languages.

8 Teachers provided verbal or written feedback.

 Students’ Questionnaire Results

Overall students’ feedback was positive.  Students recognised closer alignment between teaching and assessment, and that talking to another student was more natural. They also reported increased opportunities to practise and felt well prepared.

However, they did not feel that the new format improved their opportunity to demonstrate their learning or speaking to a student more relaxing.  The qualitative feedback tells us that this is due to anxieties around pairings.

Teachers’ Feedback

  • Language production was more spontaneous and authentic. One teacher commented ‘it was a much more authentic situation and students really helped each other to communicate’
  • Marking changed from a focus on listening for errors towards rewarding successful communication
  • Workload decreased by up to 30%, for the average student cohort and peaks and troughs of work were better distributed

Reflections

Overall, the impact on both teachers and students was positive. Student reported that they were well briefed and had greater opportunities to practise before the exam. Teachers reported a positive impact on workloads and on the students’ ability to demonstrate they were able to communicate in the language.

However, this was not reflected in the students’ feedback. There is a clear discrepancy in the teachers and students’ perception of how the new format allows students to showcase learning.

Despite mitigating action being taken, students also reported anxiety around ‘interlocutor effect’.  Race (2014) tells us that even when universities have put all possible measures in place to make assessment fair they often fail to communicate this appropriately to students. The next steps should therefore focus on engaging students to bridge this perception gap.

Follow-up

Follow up was planned for the 2019-20 academic cycle but could not take place due to the COVID-19 pandemic.

References

Galaczi & French, in Taylor, L. (ed.), (2011). Examining Speaking: Research and practice in assessing second language speaking. Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo, Dehli, Tokyo, Mexico City: CUP.

Fulcher, G. (2003). Testing Second Language Speaking. Ediburgh: Pearson.

Hardi Prasetyo, A. (2018). Paired Oral Tests: A literature review. LLT Journal: A Journal on Language and Language Teaching, 21(Suppl.), 105-110.

Race, P. (2014) Making Learning happen (3rd ed.), Los Angeles; London: Sage

Race, P. (2015) The lecturer’s toolkit : a practical guide to assessment, learning and teaching (4th ed.), London ; New York, NY : Routledge, Taylor & Francis Group

 

Developing psychoeducational materials for individuals with learning disabilities

Dr Allán Laville, a.laville@reading.ac.uk, (Dean for D&I and Lecturer in Clinical Psychology) and Charlotte Field (Research Assistant and student on MSci Applied Psychology)

Overview

To improve access to psychoeducational materials by addressing the diverse needs of those accessing Improving Access to Psychological Therapy (IAPT) services. We worked on creating materials that could be used to describe psychological disorders such as Depression and Generalized Anxiety Disorder (GAD) to those who have learning disabilities. Here we reflect upon the benefits of completing this project via a student- staff partnership as well as the potential benefits of using within IAPT.

Objectives

  • This project was funded by SPCLS Teaching & Learning Enhancement Fund and was to create psychoeducational materials suitable for those with learning disabilities that depict Depression, GAD and Panic Disorder.
  • To effectively utilise student and staff feedback in the creation of these materials.

Context

The above project was undertaken as within IAPT, Psychological Wellbeing Practitioners (PWPs) typically use materials that are text heavy when explaining psychological disorders. This can create access barriers to those with learning disabilities, arguably within service and at a university teaching level.

The aim of the project was to create visual representations of how the person may be feeling depending on the psychological disorder.

Implementation

Allán Laville (Dean for Diversity and Inclusion) designed the psychoeducational materials for learning disabilities concept and then approached Charlotte Field to see whether she wanted to take part in the development of these materials. It was important to include Charlotte here as she is training as a PWP and has also studied Art.

Charlotte Field’s experience

The preliminary stage in the project involved doing rough sketches of how Depression, GAD and Panic would be represented. These were discussed and evaluated within an initial focus group with other students on the MSci Applied Psychology Cohort 5. The subsequent reflection and review of the feedback received enabled me to produce drawings that were more interactive as well as providing a more literal and figurative version of each disorder to help make things clearer. In doing so, making the drawings more accessible and appropriate for those with learning disabilities. I had the opportunity to review feedback on the completed drawings for a second time before the drawings were submitted.

Impact

Charlotte shares her view of the impact of completing this activity:

The materials here have been developed to add to the resources which could improve access for those with learning disabilities within Improving Access to Psychological Therapies (IAPT). As the rest of the MSci cohort and I are training as PWPs this was especially relevant to develop our clinical skills. These materials will be used in the training of future MSci cohorts – both within in-class role-plays and summative role-play assessments.

Reflections

Allán Laville reflections:

The student-staff partnership was key to the success of the project as we needed to ensure that the student voice was at the forefront. This was achieved in the work Charlotte completed herself as well as within the focus group and subsequent feedback on the psychoeducational materials over email. Based on this positive experience, we are keen to continue this approach to innovative T&L practices.

Charlotte Field’s reflections:

The student-staff partnership is of great importance as it builds collaboration and crucial links between students and staff. This is particularly important with projects such as this as it combines the knowledge and expertise from experienced staff members with the student’s current experience working within these services.

Follow up

In future, we will aim to develop similar psychoeducational materials for treatment interventions within Low Intensity Cognitive Behavioural Therapy. For example, materials for Behavioural Activation, which aims to increase individual’s routine, necessary and pleasurable activities to improve one’s mood.  This intervention would lend itself well to pictorial representations.

Developing and embedding electronic assessment overviews

Dr Allán Laville, a.laville@reading.ac.uk , Chloe Chessell and Tamara Wiehe

Overview

To develop our assessment practices, we created electronic assessment overviews for all assessments in Part 3 MSci Applied Psychology (Clinical) programme. Here we reflect on the benefits of completing this project via a student-staff partnership as well as the realised benefits for students.

Objectives

  • To create electronic assessment overviews for all 8 assessments in Part 3 MSci Applied Psychology (Clinical).
  • To create the overviews via a student-staff partnership with Chloe Chessell. Chloe is a current PhD student and previous MSci student.

Context

The activity was undertaken due to the complexity of the Part 3 assessments. In particular, the clinical competency assessments have many components and so, only providing an in-class overview has some limitations. The aim was for students to be able to review assessment overviews at any time via Blackboard.

Implementation

Allán Laville (Dean for Diversity and Inclusion) and Tamara Wiehe (MSci Clinical Educator) designed the electronic assessment overview concept and then approached Chloe Chessell to see whether she wanted to take part in the development of these overviews. It was important to include Chloe here as she has lived experience of completing the programme and therefore, can offer unique insight.

Chloe Chessell’s experience

The first stage in assisting with the development of electronic assessment resources for MSci Applied Psychology (Clinical) students involved reflecting upon the information my cohort was provided with during our Psychological Wellbeing Practitioner (PWP) training year. Specifically, this involved reflecting upon information about the assessments that I found particularly helpful; identifying any further information which would have benefitted my understanding of the assessments; and suggesting ways to best utilise screencasts to supplement written information about the assessments. After providing this information, I had the opportunity to review and provide feedback on the screencasts which had been developed by the Clinical Educators.

Impact

Chloe shares her view of the impact of completing this activity:

The screencasts that have been developed added to the information that I had as a student, as this format allows students to review assessment information in their own time, and at their own pace. Screencasts can also be revisited, which may help students to ensure they have met the marking criteria for a specific assessment. Furthermore, embedded videos/links to information to support the development of key writing skills (e.g. critical analysis skills) within these screencasts expand upon the information my cohort received, and will help students to develop these skills at the onset of their PWP training year.

Reflections

Staff reflections: The student-staff partnership was key to the success of the project as we needed to ensure that the student voice was at the forefront. The electronic assessment overviews have been well received by students and we are pleased with the results. Based on this positive experience, we now have a further 4 student-staff projects that are currently being completed and we hope to publish on the T&L Exchange in due course.

Chloe Chessell’s reflections:

I believe that utilising student-staff partnerships to aid course development is crucial, as it enables staff to learn from student’s experiences of receiving course information and their views for course development, whilst ensuring overall course requirements are met. Such partnerships also enable students to engage in their course at a higher level, allowing them to have a role in shaping the course around their needs and experiences.

Follow up

In future, we will aim to include interactive tasks within the screencasts, so students can engage in deep level learning (Marton, 1975). An example could be for students to complete a mind map based on the material that they have reviewed in the electronic assessment overview.

Developing Diversity and Inclusion teaching: The importance of D&I and Ethical Practice

Dr Allán Laville, Psychology and Clinical Language Sciences, a.laville@reading.ac.uk

Overview

In the training of Psychological Wellbeing Practitioners (PWPs), teaching must include a focus on Diversity and Inclusion (D&I) as well how this relates to ethical practice. Therefore, I created a 15-minute screencast that tied key D&I principles to clinical practice, with a particular focus on ethical practice within this area.

Objectives

  1. To support students in being aware of key D&I and ethical principles and how these principles relate to their clinical practice.
  2. To support students in writing a 500-word reflective piece on the importance of considering D&I in their ethically-sound, clinical practice.

Context

PWP programmes include D&I training within the final module of the clinical programme, but to meet the British Psychological Society (BPS) programme standards, D&I training needs to be incorporated throughout. Furthermore, this training should be tied to the BPS programme standard on Ethical Practice teaching (Module PY3EAA1/PYMEAA).

Implementation

The first step was to identify the key sources to include within the screencast. These were wide ranging from legislation (Equality Act, 2010), positive practice guides (Improving Access to Psychological Therapies) and ethical practice guidelines (British Psychological Society) and reference to the University’s Fitness to Practise policy.

The second step was to think about how students could engage with the screencast in a meaningful way. Based on an earlier T&L Exchange project report of mine (https://sites.reading.ac.uk/t-and-l-exchange/2019/07/23/developing-innovative-teaching-the-importance-of-reflective-practice/), I wanted to include an element of reflective practice. Students were asked to write a 500-word reflective piece on their own take-home points from the screencast and preferably, following the Rolfe, Freshwater, and Jasper (2001) reflective model of: a) what is being considered, b)  so what, which I say to my students is the ‘why care?’ part! And c) now what i.e. from reviewing what and so what, detailing your SMART action plan for future clinical practice.

Example by Will Warley, Part 3 MSci Applied Psychology (Clinical) student.

Impact

The student feedback about the screencast and completing the reflective piece has been very positive. This has been across both the MSci in Applied Psychology (Clinical) as well as the Charlie Waller Institute (CWI), PG (Cert) in Evidence-Based Psychological Treatments (IAPT Pathway). The training materials have also been shared with members of the SPCLS Board of Studies for CWI training programmes.

In regard to national level impact, I have presented this innovative approach to D&I teaching at the BPS Programme Liaison Day, which included the BPS PWP Training Committee and Programme Directors from across the UK. The presentation was received very well including requests to disseminate the materials that we use in the teaching at UoR. Therefore, these materials have now been circulated to all PWP training providers in the UK to inform their D&I provision.

Reflections

One core reason for the success of this activity was the commitment and creativity of our students! Some students used software to create excellent mind maps, interactive presentations or a YouTube video! There was even an Instagram account used to illustrate the main take-home points from the screencast, which I thought was particularly innovative. Overall, I was absolutely delighted to see such high levels of student engagement with topics that are so important – both personally and professionally.

In regard to better implementation, it is possible that slightly more guidance could have been provided regarding how to approach the reflective task, but the brief of ‘be as creative as possible!’ worked very well indeed!

Follow up

I will be following up with the BPS PWP Training Committee in 2020 to see how this activity has developed within other PWP training providers! We will then create a summary of all innovative approaches to including D&I in PWP programmes and how these meet the programme standards.

Links

https://my.cumbria.ac.uk/media/MyCumbria/Documents/ReflectiveModelRolfe.pdf

Student YouTube video as submission on reflective task: https://youtu.be/hMU6F_dknP4

Universally Speaking: crossing cultural & generational boundaries – a seminar series

Dan Jones, School of Psychology & Clinical Language Sciences, d.jones6@reading.ac.uk

 

Overview

The ‘Universally Speaking’ series provides a platform for students, staff and community members to exchange ideas on culture, heritage, customs, values and traditions, via a seminar presentation. Each seminar is followed by an informal drinks reception to facilitate further discussion and interactions between the different communities.

Objectives

  • To offer an outstanding holistic student learning experience by promoting extra-curricular activities in the School.
  • To celebrate and promote the diverse School: lends on the diverse experiences of our staff, students and local communities to help students become global citizens and directly experience the benefits of a diverse and multinational learning environment.
  • To equip students with the aspirations, confidence and skills: opportunity to present and talk to a range of different people.

Context

The School of Psychology & Clinical Language Sciences is a wonderfully diverse School – this series was launched to promote and celebrate this diversity. The series provides an opportunity for members of the School to reflect upon different experiences and perspectives of the world, and to take a moment to discuss these with others. Ultimately, it is a tool to promote and explore difference, leading to greater tolerance and acceptance of it.

Implementation

Once funding was gained, along with the student partner, we formed a student committee to help support the different aspects of the series: promotion, advertisement, organisation, and invitations to community members. The committee was made up of five students, and two members of staff (myself included). The seminars were run on a monthly basis, starting in February 2019 and running until June 2019. Talks were delivered by a range of volunteers: UG students, PGR students, PCLS staff and other University members (including the University Chaplain, who is hopefully going to repeat their highly interesting session).

Overall, the series was a success, with positive feedback received and a consistent attendance, including up to eight members of the public attending the final session of the academic year. Due to the positive reception, we are hoping to make this a permanent fixture on the PCLS calendar.

Impact

The feedback on the series has been overwhelmingly positive. Quotes from attendees nicely summarise the benefits that have been gained from the series so far:

“The ability to increase my knowledge on other countries education and research style/system. Learn about peoples’ experience – first-hand experience. Love it!”

“Hearing about the differences from personal perspectives. Helping people embracing the differences.”

“Really interesting to hear about cultures and customs in other countries and how one should consider them when assessing actions and situations.”

Many of the quotes reflect on learning about and understanding difference; skills that lead to more tolerance and acceptance of difference – ultimately, this is what the series contributes to the PCLS community.

Reflections

The only negative of the series was attendance: considering the size of PCLS, we only averaged around 40 attendees across the series. There were several reasons why this may have been the case, including the timing, exam periods and advertising. We are aiming to address these issues if the series is to continue. One step that we have taken is to utilise the skills of the School marketing officer to help with promotion and advertising.

Follow up

The launch of the seminar series was made possible with PLANT funding – this funding ended in July 2019. To maintain the series over the course of the next academic year, and to enable collaboration with other groups across the University, additional funding has been sought from the School of PCLS. We already have the next seminar planned for January 2020, in collaboration with the UoR Islamic Society.

 

 

 

 

 

 

 

 

 

 

 

 

Rev Dr Mark Laynesmith, Anglican Chaplain at the University, reports on a project set up with the University’s Institute of Education to explore increasing knowledge diversity among school children.

Stories of Our Studies

Simon Floodgate, Institute of Education, s.floodgate@reading.ac.uk

Overview

A form of inter-active, reflective practice for students in which Playback Theatre (an improvisatory form) is used to ‘play back’ individual stories of students’ experiences regarding all aspects of their studies.  This process can support emotional literacy and well-being and promote professionalism in students at all levels of study.

Objectives

  • To develop students’ ability to both express and assert themselves in the world and to support them to be more successful within their studies. (TLDF Priority 2.2)
  • To support students to feel valued, gain greater awareness of their skills and articulate these to better address the challenges they face in the field of education and the workplace (TLDF Priority 2.3)

Context

To address concerns regarding student well-being and emotional literacy as highlighted both nationally, within the University and the IOE where workload and pressures have specifically impacted upon initial teacher training (ITT) students who are transitioning into teaching professionals.

Implementation

The pilot year, within the IOE, was focussed upon the training of a student performance group with a couple of performance-workshops undertaken with Secondary ITT students and IOE staff.  Both sessions were evaluated and the students involved as the performance team, were also asked to evaluate the benefits to them of engagement in the project.  The project enters a second year (2019-20), with further funding, to adapt the contact sessions.  This will lead to two different versions of Stories of Our Studies.  A full length, two-hour version will incorporate a full Playback Theatre performance of 1-1/2 hours duration in a more public setting.  A second shorter version will align the performance elements with discursive and written aspects focussed upon critical incident analysis (Lister and Crisp, 2007).  This will blend the elements for more captive audiences within module teaching sessions.

Impact

As a pilot project, Stories of Our Studies achieved its objectives.  A student team was trained to deliver the contact sessions alongside the project leader.  The project was presented to both PGCE Secondary ITT students and IOE staff, enabling feedback from different perspectives.  Staff were able to appreciate the potential impact upon student well-being.  The PGCE students were able to effectively reflect upon their learning, in particular focussing upon their school teaching placements. They were able to subjectively reflect upon how these experiences felt to them but also objectively appreciate what occurred, how their experiences were similar or different to others and to be able to consider themselves as professional teachers soon to embark upon their chosen profession. The TLDF priorities 2.2 and 2.3 were both met.

Reflections

The enthusiasm and willingness of the UG students who trained in the form was exceptional and their empathy and artistry were commented upon following both performance-workshops.  This was a major factor in the pilot’s success.  The structure of the session with the main performance aspect following some Morenian sociometry facilitated a relaxed and intimate atmosphere thus enabling audience members to openly share.  The use of the form – Playback Theatre – was vital to the success of the pilot.

Although participants gained a lot from their engagement in the session, there is a further need to develop the sustainability of the reflective process.  To this end the project will be developed into longer and shorter iterations (as mentioned above).  There remains some difficulty in encouraging students to attend extra-curricular sessions and, for many, to attend events in which drama/theatre are mentioned.  This is a difficulty in attracting both student-performers and audience members.  Word of mouth will help and, like a stone gathering moss, momentum will attract more interest and students to engage with it.

Follow up

See above. The project has entered a second year with further TL enhancement (mini) funding.  It is evolving with the incorporation of critical incident analysis and a further blending of the performance and written reflection elements.

We already have more performance-workshops booked in the diary for 2019-20 than for last year, including presentation at the University’s T&L conference in January 2020.

Contact has been made with the RUSU society, Open Minds, to investigate the potential of some performances to a larger student audience outside of timetabled teaching.

The performance-workshop, photographed last year, will be filmed to create a marketing online clip to promote the project.  Recruitment of new student-performer members has already begun.

 

 

 

 

 

 

 

 

Photo of Playback Theatre in action

Reframing Identity 360

Kate Allen, Department of Art, k.allen@reading.ac.uk

Overview

An investigative artwork that explores identity using 360 cameras developed through practical, alumni led workshops and socially engaged art with current art students, school groups and the general public. Part of ArtLab Movement’ at Tate Exchange (TEx) 2019 at the Tate Modern on March and be archived on the ArtLab website.

Objectives

- Contribute to live art event/out-reach work experience led by Alumni at Tate Exchange 1-3 March 2019

- Explore identity capture with 360 cameras

- 360 cameras experimentation including designing, capturing, printing and editing.

- Create portraits with purpleSTARS, people with learning disabilities and children from Widening Participation schools in Reading.

Context

Reframing Identity explored self-portraits in shot in 360, developed as a response to Tania Bruguera’s Turbine Hall Commission concerning institutional power, borders and migration. Can 360 self-portraits raise awareness of how interconnected we are, when no person is ever behind the 360 camera, everyone is included.

Implementation

Alumni and Virtual Reality artist Kassie Headon researched ideas in response to Tania Bruguera installation at Tate Modern inspired by Bruguera’s ideas on inclusion, connecting to Kate Allen’s research with purpleSTARS a group of people with and with learning disabilities who aim to make museums more inclusive. Kassie demonstrated to students and purpleSTARS how to use the GoPro Fusion Camera and the app to edit 360 content. Activities to share the 360 self portrait concept with visitors were developed including drawing cylindrical self-portraits which they could then wear on their heads for a 360 selfie. Students facilitated the Reframing Identity 360 workshop as part of ArtLab Movement at TEx. Using 360 cameras was a new experience and concept for our students and most people visiting the TEx. The 360 self-portraits were exhibited via live video stream from the 360 cameras on an iPad displayed at the Tate and let participants explore the views, which they could manipulate and distort to create the desired effect. Participants 360 self-portraits were also printed or sent to the visitors phone.

Impact

The impact of Reframing Identity 360 created access and inclusion with new technologies for students and the public. Experiencing the live video stream frequently gave visitors an ‘Oh Wow’ moment. TEx gave an opportunity for research led teaching with Dr Allen purpleSTARS, Alumni Kassie Headon and current BA students to explore the concept of 360 self-portraits gain professional practice experience facilitating the workshops and technical skills working, with the 360 camera. The 360 cameras are now part of the digital equipment available to students with a core team of ArtLab students now familiar with their potential and how to use them.

Reflections

Working with new technologies in collaboration with Alumni, ArtLab students and purpleSTARS led to new perspectives on ideas of inclusion and self -portraiture. The experimental research occurred in response to work at the Tate and in collaboration with visitors to TEx. The project built capacity and awareness of new technology being introduced into the Art Dept learning through research and practical experiences the potential to create artworks and inclusive engagements.

Follow up

Kassie Headen continued to work with the 360 camera collaborating with widening participation schools during the ArtLab summer workshops 2019 exploring spaces and manipulating 2d versions of 3d space.

We are developing further research collaborations and research led teaching opportunities for ideas exploring inclusion in museums and immersive virtual reality artworks/experiences using Oculus Rift technology.

Links and References

We created a 360 recording of our Reframing Identity event at the Tate https://www.thinglink.com/mediacard/1158753748827242499?autoplay=0&autorotate=0&displaytitle=1&rel=1

ArtLab documents the workshop

https://readingartlab.com/2019/04/25/artlab-tate-exchange-visual-diary-2nd-and-3rd-march-2019/

purpleSTARS web documentation

https://purplestars.org.uk/2017/11/12/purplestars-at-tate-gallery-2018/

Tate Exchange webpage

https://www.tate.org.uk/whats-on/tate-modern/tate-exchange/workshop/reading-assembly-movement

Creating screencast videos to support and engage post-graduate students

Sue Blackett – Henley Business School, 2018-19

Image of Sue Blackett

Overview

I participated in the university’s Personal Capture pilot as a Champion for my school to trial the Mediasite tool to create screen cast videos for use in teaching and learning. My aim was to help PGT students get to grips with key elements of the module. The videos facilitated students in repeatedly viewing content with the aim of increasing engagement with the module. Some videos were watched multiple times at different points throughout the term indicating that information needed to be refreshed. 

Objectives

  1. To connect with the cohort and establish module expectations. 
  2. Reduce class time taken up with module administration. 
  3. Provide coursework feedback in an alternative form and reinforce its feedforward use for the final exam. 
  4. To provide exam revision advice and highlight areas of focus. 
  5. Support students with weaker English language skills. 
  6. Provide module materials in a reusable, accessible and alternative form. 

Context

The target audience was students on ACM003 Management Accounting Theory & Practice, a postgraduate course where 91% of students were native Mandarin speakers. English language skills were an issue for some students, so capture video provided opportunities for students to re-watch and get to grips with the content at their leisure. In addition, I wanted to free up class contact time so I could focus on content in areas that had been more challenging on the previous run of the module. Also, by using different colours and font sizes on the PowerPoint slides, the visual emphasis of key points reinforced the accompanying audio. 

Implementation

The first video recorded was a welcome to the module video (slides and audio only) that covered the module administration i.e. an overview of module, outline of assessment, key dates, module text book etc. The content for the video was relatively straightforward as it was taken out of the first lecture’s slides. By isolating module admin information, more information could be added e.g. mapping assessable learning outcomes to assessments and explaining the purpose of each type of assessment. In first recording the video, I did not follow a script as I was trying to make my delivery sound more natural. Instead, I made short notes on slides that needed extra information and printed off the presentation as slides with notes. As this is the same strategy that I use to deliver lectures, I was less concerned about being “audio ready” i.e. not making errors in my voice recording. 

 In the second and third videos (coursework feedback and exam revision advice), I included video of myself delivering the presentations. As the recordings were made in my home office, additional visual matters had to be considered. These included: what I was wearing, the background behind me, looking into the camera, turning pages, etc. The second attempts of each recording were much more fluent and therefore uploaded to Blackboard. 

 The last two recordings were quite different in nature. The coursework feedback used visuals of bar charts and tables to communicate statistics accompanied by audio that focused on qualitative feedback. The exam revision video used lots narrative bullet points. 

Examples of my videos:

Welcome to module: https://uor.mediasite.com/Mediasite/Play/7a7f676595c84507aa31aafe994f2f071d

Assessed coursework feedback: https://uor.mediasite.com/Mediasite/Play/077e974725f44cc8b0debd6361aaaba71d

Exam revision advice: https://uor.mediasite.com/Mediasite/Play/94e4156753c848dbafc3b5e75a9c3d441d

Resit Exam Advice: https://uor.mediasite.com/Mediasite/Play/e8b88b44a7724c5aa4ef8def412c22fd1d

Impact

The welcome video did have impact as it was the only source of information about the administration for the course. When students arrived at the first class with the text book, this indicated that they had been able to access the information they needed to prepare for the course. Student response to the personal capture pilot project questionnaire was low (18%), however, the general feedback was that the videos were useful in supporting them during the course. 

 Analysis of analytics via MediaSite and Blackboard provided some very interesting insights: 

  1. Most students did not watch the videos as soon as they were released. 
  2. Some of the videos were watched multiple times throughout the term by weaker and stronger students. 
  3. Some students were not recorded as having accessed the videos. 
  4. Students were focused for the first 20 – 60 seconds of each video and then skipped through the videos. 
  5. Few students watched the videos from start to finish i.e. the average time watched for the 4 min 49 secs welcome video was 2 min 10 secs. The coursework feedback video was 9 mins 21 secs, however, average viewing time was 3 mins 11 secs. The revision video followed the same trend being 8 mins 41 secs long with an average watching time of 2 mins 55 secs.
     

Review of video along with watching trends showed that students skipped through the videos to the points where slides changed. This suggested that the majority were reading the slides rather than listening to the accompanying commentary which contained supplementary information. 

 As no student failed to meet the admin expectations of the course, those that had not watched the video must have been informed by those who had. 

Reflections

The analytics were most illuminating. Me appearing in videos was supposed to establish bonds with the cohort and increase engagement, however, my appearance seemed to be irrelevant as the students were focused on reading rather than listening. This could have been due to weaker listening skills but also highlights that students might think that all important information is written down rather than spoken.  

 Videos with graphics were more watched than those without so my challenge will be to think about what content I include in slides i.e. more graphics with fewer words and/or narrative slides with no audio. 

 I will continue with capture videos, however, I will do more to test their effectiveness, for example I will design in-class quizzes using KahootMentimeter, etc. to test whether the content of the videos has been internalised. 

Follow-up

I’ve become much quicker at designing the PowerPoint content and less worried about stumbling or searching for the right words to use. I have been able to edit videos more quickly e.g. cutting out excessive time, cropping the end of the video. Embedding videos in Blackboard has also become easier the more I’ve done  it. The support information was good, however, I faced  a multitude of problems that IT Support had to help me with, which, if I’m honest, was putting me off using the tool  (I’m a Mac user mostly using this tool off campus).