Rethinking assessment design, to improve the student/staff experience when dealing with video submissions

Rachel Warner, School of Arts and Communication Design

Rachel.Warner@pgr.reading.ac.uk

Jacqueline Fairbairn, Centre for Quality Support and Development

j.fairbairn@reading.ac.uk

Overview

Rachel in Typography and Graphic Communication (T&GC) worked with the Technology Enhanced Learning (TEL) team to rethink an assignment workflow, to improve the student/staff experience when dealing with video submissions. Changes were made to address student assessment literacies, develop articulation skills, support integration between practice and reflection, and make use of OneDrive to streamline the archiving and sharing of video submissions via Blackboard.

This work resulted in students developing professional ‘work skills’ through the assessment process and the production of a toolkit to support future video assessments.

Objectives

  • Improve staff and student experiences when dealing with video assignment submissions. Specifically, streamlining workflows by improving student assessment literacy and making use of university OneDrive accounts.
  • Support students to develop professional skills for the future, through assessment design (developing digital literacies and communication skills).
  • Provide an authentic assessment experience, in which students self-select technologies (choosing software and a task to demonstrate) to answer a brief.

Context

The activity was undertaken for Part 1 students learning skills in design software (e.g. Adobe Creative apps). The assignment required students to submit a ‘screencast’ video recording that demonstrated a small task using design software.

Rachel wanted to review the process for submitting video work for e-assessment, and find ways to streamline the time intensive marking process, particularly in accessing and reviewing video files, without compromising good assessment practice. This is also acknowledged by Jeanne-Louise Moys, T&GC’s assessment and feedback champion: “Video submissions help our students directly demonstrate the application of knowledge and creative thinking to their design and technical decisions. They can be time-consuming to mark so finding ways to streamline this process is a priority given our need to maintain quality practices while adapting to larger cohorts.’”

The TEL team was initially consulted to explore processes for handling video submissions in Blackboard, and to discuss implications on staff time (in terms of supporting students, archiving material and accessing videos for marking). Designing formative support and improving the assessment literacy of students was also a key driver to reduce the number of queries and technical issues when working with video technologies.

Implementation

Rachel consulted TEL, to discuss:

  • balancing the pedagogic implications of altering the assignment
  • technical implications, such as submission to Blackboard and storage of video

To address the issue of storing video work, students were asked make use of OneDrive areas to store and submit work (via ‘share’ links). Use of OneDrive encouraged professional behaviours such as adopting a systematic approach to file naming, and it meant the videos were securely stored on university systems using a well-recognised industry standard platform.

To further encourage professional working, students were required to create a social media account to share their video. YouTube was recommended; it is used prolifically by designers to showcase work and portfolios, and across wider professional settings.

Students were provided with a digital coversheet to submit URLs for both the OneDrive and YouTube videos.

The most effective intervention was the introduction of a formative support session (1.5hr). Students practiced using their OneDrive area, set up YouTube accounts and reviewed examples of screencasts. This workshop supported students to understand the professional skills that could be developed through this medium. The session introduced the assessment requirements, toolkit, digital coversheet and allowed students to explore the technologies in a supported manner (improving students’ assessment literacy!)

The assignment instructions were strategically revised, to include information (‘hints and tips’) to support the students’ development of higher production values and other associated digital literacies for the workplace (such as file naming conventions, digital workflows, and sourcing online services).

Students were provided with the option to self-select recording/editing software to undertake the screencast video. Recommended tools were suggested, that are free to use and which students could explore. ‘Screencast-o-matic’ and ‘WeVideo’ provide basic to intermediate options.

Impact

Marking the submissions was made easier by the ability to access videos through a consistent format, using a clearly structured submission process (digital coversheet). The ability to play URL links directly through OneDrive meant Rachel was able to store copies of the videos into a central area for future reference. Students also provided a written summary of their video, highlighting key video timings that demonstrate marking criteria (so the marker does not have to watch whole video).

Rachel rationalised her approach to marking by developing a spreadsheet, which allowed her to effectively cross reference feedback against the assessment criteria (in the form of a rubric) and between assignments. This greatly speeded up the marking workflow and allowed Rachel to identify patterns in students work, where common feedback statements could be applied, as appropriate.

The assessment highlighted gaps in students existing digital literacies. The majority of students had not made a video recording before and many were apprehensive about speaking into a microphone. After the completion of the screencasts, previously unconfident students noted in their module reflections that the screencast task had developed their confidence to communicate and explore a new technology.

Reflections

The modifications to the assessment:

  • Reflected professional digital competencies required of the discipline;
  • Allowed students to explore a new technology and way of working in a supported context; and,
  • Built confidence, facilitated assessment literacy, and encouraged reflection.

Future modifications to the screencast submission:

  • Peer review could be implemented, asking students to upload videos to a shared space for formative feedback (such as Facebook or a Blackboard discussion board).
  • The digital coversheet had to be downloaded to access URL links. In future, students could paste into the submission comment field, for easier access when marking.
  • Rachel is developing a self-assessment checklist to help students reflect on the production values of their work. The summative assessment rubric is focused on video content, not production values, however, it would be useful for students to get feedback on professional work skills. For example, communication skills and use of narrative devices which translate across other graphic mediums.

Toolkit basics:

a thumbnail image of a toolkit document, full access available via links in webpage

  • Outline task expectations and software options, give recommendations
  • Source examples of screencasts from your industry, discuss with students.
  • Provide hints and tips for creating effective screencasts.
  • Provide submission text. Consider asking students to use the ‘submission comment’ field to paste links to their work, for quick marker access to URLs.
  • Plan a formative workshop session, to practice using the software and go through the submission process (time invested here is key!).
  • Create a self-assessment checklist, to enhance the production quality of videos and highlight transferrable skills that can be developed by focusing on the quality of the production.
  • Consider creating a shared online space for formative peer-feedback (e.g. Blackboard discussion forum).
  • Consider using a marking spreadsheet to cross-reference feedback and highlight good examples of screencasts that can be utilised in other teaching.

Links

Screencast example: (YouTube link) This screencast was altered and improved after submission and marking, taking onboard feedback from the assessment and module. The student noted ‘After submission, I reflected on my screencast, and I changed the original image because it was too complex to fit into the short time that I had available in the screencast. I wanted to use the screencast to show a skill that I had learned and the flower was simple enough to showcase this’. Part of the module was to be reflective and learn from ‘doing’, this screencast is an example of a student reflecting on their work and improving their skills after the module had finished.

Screencast example: (YouTube link) This screencast was a clear and comprehensive demonstration of a technique in PhotoShop that requires multiple elements to achieve results. It has a conclusion that demonstrates the student’s awareness that the technique is useful in other scenarios, other than the one demonstrated, giving the listener encouragement to continue learning. The student has used an intro slide and background music, demonstrating exploration with the screencast software alongside compiling their demonstration.

Screencast example: (YouTube link) This demonstrates a student who is competent in a tool, able to use their own work (work from another module on the course) to demonstrate a task, and additionally includes their research into how the tool can be used for other tasks.

Other screencast activity from the Typography & Graphic Communication department from the GRASS project:  (Blog post) Previous project for Part 1s that included use of screencasts to demonstrate students’ achievements of learning outcomes.

Blackboard Collaborate cross-campus tutorials as a useful tool to enhance the Part One Pharmacy student experience at the University of Reading Malaysia

Dr Darius Widera, School of Chemistry, Food and Pharmacy
d.widera@reading.ac.uk

Overview

After a successful application to act as one of the early adopters of Blackboard Collaborate at the University of Reading, this technology platform was used for a series of cross-campus tutorials within the Fundamentals of Physiology (PM1AM) module between the University of Reading’s Whiteknights and Malaysia campuses. The format was well-received, and contributed to an enhanced student experience.

Context

The official inauguration of the University of Reading Malaysia (UoRM) campus in EduCity, Johor Bahru, in early 2017 and the start of the MPharm (Malaysia) programme in the academic year 2016/17 offer excellent opportunities for further internationalisation of the University of Reading and specifically within Pharmacy education.

The University of Reading Malaysia offers a double accredited (UK and Malaysia) 2+2 MPharm (Hons) degree where the students study for two years at the Malaysia campus followed by two consecutive years in Reading.

The PM1A module and its UoRM counterpart, PM1AM, cover the basics of biology and human physiology including genetics, biochemistry and cell biology. According to student feedback, these topics tend to be challenging for the students, especially in light of the fact that significant numbers of Pharmacy students do not have A-level biology to provide background knowledge.

In response to this feedback, several tutorials have been introduced to provide students with interactive opportunities to revise the content of lectures and practical sessions and to close any potential knowledge gaps.

Thus, there was a need for the development of a cross-campus solution to ensure that both MPharm cohorts (UoR and UoRM) are provided with a similar form of tutorials.

Objectives

  • To explore if Blackboard Collaborate can be used for cross-campus delivery of tutorials covering the content of the genetics lecture series within the PM1A/PM1AM module.
  • To investigate if cross-campus virtual classroom/teleconference represents an appropriate pedagogical tool for delivery of tutorials in Pharmacy and how this deliver method affects student engagement and interactivity.
  • To assess if these sessions could help 2+2 MPharm students to prepare for their two years of study in Reading.

Implementation

The Blackboard Collaborate platform was used to develop a series of tutorials in genetics. The online sessions were led by Dr Widera (live video capture via a webcam) at the University of Reading’s Whiteknights campus and streamed to students at the UoRM. The student group was composed of 11 Malaysian Part One MPharm students. The content of the tutorials was covered in the respective lecturees. It was expected that students would have factual knowledge of the topic, although at heterogeneous levels.

All students were equipped with PCs with headsets and webcams. Blackboard Collaborate functions including ‘raise hand’, virtual whiteboard, chat, and direct interaction with all or individual students (either via audio or video) were used. In addition, external tools (e.g. Microsoft PowerPoint presentations and the Poll Everywhere app) were used via the ‘share screen’ function of Blackboard Collaborate. For the tutorial, an introductory PowerPoint presentation was designed and a screencast deposited on YouTube as a contingency plan. Multiple choice questionnaires (MCQs) were set up on the Poll Everywhere platform, and short answer questions (SAQs) were included in an additional PowerPoint presentation. After each MCQ/SAQ, students were given time to decide on an answer (individually via Poll Everywhere), followed by an interactive discussion.

The overall length of each tutorial session was 50 minutes. Individual anonymous post-hoc feedback was collected to evaluate student opinions on the usefulness, overall style, and delivery. In addition, a technical report and an experience log was collated and submitted to the Technology Enhanced Learning team. Finally, the content, deliver and potential changes were discussed with students and peers during a visit to the UoRM.

Impact

During the tutorials no serious technical issues were encountered, although students at UoRM did experience slight lagging in their connections (with video and audio becoming slightly out of sync). Students showed high levels of interaction and successfully used most of the Blackboard Collaborate features. Importantly, other than in UoR in-class tutorials, students engaged and interacted early on. This is reflected in the feedback collected after the first session (“I like how it is interactive and fun”). The tutorial format also seemed to help students to revise the content of the lectures (“Useful to enhance my biology knowledge”, “It helps me to revise”, “It helps me to find out my difficulties with previous lectures”). Moreover, students appreciated that the session was different compared to conventional lectures (“It was different from just sitting in the classroom and listening to lecturers”, “it was another way of learning outside the classroom”). Last but not least, it was appreciated that the tutorials were run by Reading-based staff that the 2+2 students would meet during their two years in Reading (“can meet Dr Widera and learn from him”). No negative feedback was received.

Follow up

Following the feedback received, further tutorials involving other lecturers teaching on the PM1A module will be developed and implemented.