Daniel Grant, Associate Professor in Clinical Pharmacy & Pharmacy Education, Pharmacy Director of Teaching & Learning & Dr Taniya Sharmeen Research Fellow
Click here to read the full report.
This slide summarises the project:
Daniel Grant, Associate Professor in Clinical Pharmacy & Pharmacy Education, Pharmacy Director of Teaching & Learning & Dr Taniya Sharmeen Research Fellow
Click here to read the full report.
This slide summarises the project:
Madeleine Davies, Cindy Becker and Michael Lyons- SLL
A feedback audit and consultation with the Student Impact Network revealed a set of practices DEL needs to amend. The research produced new student-facing physical and online posters, designed by a ‘Real Jobs’ student, to instruct students on finding their feedback online, and generated ‘marking checklists’ for staff to indicate what needs to be included in feedback and what needs to be avoided.
The ‘DEL Feedback Action Project’ addresses the persistent issue of depressed NSS responses to Department of English Literature assessment and feedback practices. The responses to questions in ‘teaching quality’ sections are favourable but the 2018 NSS revealed that, for English Studies, Reading is in the third quartile for the ’Assessment and Feedback’ section and the bottom quartile for question 8 (scoring 64% vs the 74% median score) and question 9 (scoring 70% vs the 77% median score).
In October 2018, DEL adopted eSFG. An EMA student survey undertaken in January 2019 polled 100 DEL students and found that, though students overwhelmingly supported the move to eSFG, complaints about the quality of DEL feedback persisted.
Michael Lyons began the project with an audit of DEL feedback and identified a number of areas where the tone or content of feedback may need improving. This material was taken to the Student Impact Network which was shown anonymised samples of feedback. Students commented on it. This produced a set of indicators which became the basis of the ‘marking checklist’ for DEL staff. Simultaneously, DEL staff were asked to discuss feedback practice in ‘professional conversations’ for the annual Peer Review exercise. This ensured that the combined minds of the whole department were reflecting on this issue
Student consultation also revealed that many students struggle to find their feedback online. With this in mind, we collaborated with TEL to produce ‘maps to finding feedback’ for students. A ‘Real Jobs’ student designer converted this information into clear, readable posters which can be displayed online or anywhere in the University (the information is not DEL-specific). The posters will be of particular use for incoming students but our research also suggested that Part 3 students are often unaware of how to access feedback.
The results of the initial audit and consultation with students indicated where our feedback had been falling short. We wrote a summary of these finding for DEL HoD and DDTL.
Research into marking rubrics revealed that DEL marking would not be suited to using this feedback practice. This is because they can be inflexible and because DEL students resist ‘generic’ feedback.
The student-facing posters and staff-facing ‘marking checklist’ speak to two of the main issues with DEL feedback that were indicated by students. The latter will deter overly-brief, curt feedback and will prompt more feedforward and comment about specific areas of the essay (for example, the Introductory passage, the essay structure, referencing, grammar, use of secondary resources, etc).
With DEL staff now focused on the feedback issue, and with students equipped to access their feedback successfully, we are hoping to see a marked improvement in NSS scores in this area in 2020-21.
For ‘surprises’, see ‘Reflections’.
The pressure on academic staff to mark significant amounts of work within tight deadlines can lead to potential unevenness in feedback. We are hoping that our research prompts DEL to streamline its assessment practice to enhance the quality and consistency of feedback and feedforward.
Students’ responses in the Student Impact Network also suggested that additional work is required on teaching students how to receive feedback. Over-sensitivity in some areas can produce negative scores. With this in mind, the project will terminate with an equivalent to the ‘marking checklist’ designed for students. This will remind students that feedback is anonymous, objective, and intended to pave the way to success.
Monitoring NSS DEL feedback scores in the 2020-21 round, and polling students in the next session to ensure that they are now able to access their feedback.
Continuing to reflect on colleagues’ marking workload and the link between this and unconstructive feedback.
Daniela Standen, ISLI
ISLI teaches almost exclusively international students. Many of our programmes run all year round, so ISLI had to move to teach exclusively online in the Summer Term. This case study outlines the approach taken and some of the lessons learnt along the way.
In April 2020 as the country was into lockdown and most of the University had finished teaching, ISLI was about to start a ‘normal’ teaching term. The Pre-sessional English Programme was about to welcome 100 (mostly new) students to the University. The January entry of the International Foundation Programme was less than half-way through their studies and the Academic English Programme was still providing language and academic literacy support to international students.
Moving to online teaching was greatly facilitated by having in house TEL expertise as well as colleagues with experience of online teaching, who supported the upskilling of ISLI academic staff and were able to advise on programme, module and lesson frameworks.
We thought that collaboration would be key, so we put in place numerous channels for cross-School working to share best practice and tackle challenges. ISLI TEL colleagues offered weekly all School Q&A sessions as well as specific TEL training. We set up a Programme Directors’ Community of Practice that meets weekly; and made full use of TEAMS as a space where resources and expertise could be shared. Some programmes also created a ‘buddy system for teachers’.
Primarily the School adopted an asynchronous approach to teaching, synchronous delivery was made particularly difficult by having students scattered across the globe. We used a variety of tools from videos, screencasts, narrated PowerPoints and Task & Answer documents to full Xerte lessons. Generally using a variety of the above to build a lesson. Interactive elements were provided initially mostly asynchronously, using discussion boards, Padlet and Flipgrid. However, as the term progressed feedback from students highlighted a need for some synchronous delivery, which was carried out using Blackboard collaborate and TEAMS.
It has not been easy, but there have been many positive outcomes from having had to change our working practices. Despite the incredibly short timescales and the almost non-existent preparation timel, our PSE 3 students started and successfully finished their programme completely online, the IFP January entry students are ready to start their revision weeks before sitting their exams in July and international students writing dissertations and post graduate research were supported throughout the term.
As a School we have learnt new skills and to work in ways that we may not have thought possible had we not been forced into them. These new ways of working have fostered cross-School collaboration and sharing of expertise and knowledge.
We have learnt a lot in the past three months. On average it takes a day’s work to transform one hour of face to face teaching into a task-based online lesson.
Not all TEL tools are equally effective and efficient, below are some of our favourites:
If time were not a consideration Xerte would also be on the list.
Copyright issues can have a real impact on what you can do when delivering completely online. Careful consideration also needs to be given when linking to videos, particularly if you have students that are based in China.
ISLI is now preparing for Summer PSE, which starts at the end of June. Many of the lessons learnt this term have fed into preparation for summer and autumn teaching. In particular, we have listened to our students, who told us clearly that face-to-face interaction even if ‘virtual’ is really important and have included more webinars and Blackboard Collaborate sessions in our programmes.
Zainab Abdulsattar (student – Research Assistant), Tamara Wiehe (staff – PWP Clinical Educator) and Dr Allán Laville, a.laville@reading.ac.uk, (Dean for D&I and Lecturer in Clinical Psychology). School of Psychology and CLS.
To help Part 3 MSci Applied Psychology students address the emotional aspect of engaging with and interpreting assessment feedback, we have created a Blackboard feedback tool, which draws on self-help strategies used in NHS Mental Health services. This was a TLDF funded project by CQSD and we reflect upon the usefulness of the tool in terms of helping students manage their assessment feedback in a more positive and productive way for both now and the future.
Assessment and feedback are continually rated as the lowest item on student surveys despite efforts from staff to address this. Whilst staff can certainly continue to improve on their practices surrounding providing feedback, our efforts turned to how we could improve student engagement in this area. Upon investigation of existing feedback-focused tools, it has become apparent that many do not acknowledge the emotional aspect of addressing assessment feedback. For example, the ‘Development Engagement with Feedback Toolkit (DEFT)’ has useful components like a glossary helping students with academic jargon, but it does not provide resources to help with feedback related stress. The aim was to address the emotional aspect of interpreting feedback in the form of a self-help tool.
Zainab Abdulsattar’s experience:
Firstly, we carried out a literature review on feedback in higher education and the use of self-help resources like cognitive restructuring within the NHS used to treat anxiety and depression. These ideas were taken to the student focus group: to gather students’ thoughts and opinions on what type of resource they would like to help them understand and use their feedback.
Considering ideas from the literature review and the focus group, we established the various components of the tool: purpose of feedback video, problem solving and cognitive restructuring techniques, reflective log and where to go for further support page. Then, we started the creation of our prototype Blackboard tool. At tool creation stage, we worked collaboratively with the TEL team (Maria, Matt and Jacqueline) to help format and launch the tool. Upon launch, students were given access to the tool via Blackboard and a survey to complete once they had explored and used the tool.
Our prototype Blackboard tool met the main objective of the project, to address the emotional aspect of the interpreting assessment feedback. The cognitive restructuring resource aimed to identify, challenge and re-balance students negative or stressful thoughts related to receiving feedback. Some students reported in the tool survey that they found this technique useful.
As well as this, the examples seemed to help students link their past experiences of not getting a good grade. Students also appreciated the interactive features like the video of the lecturer [addressing the fact that feedback is not a personal attack] and were looking forward to the tool being fully implemented during their next academic year. Overall, the student survey was positive with the addition of some suggestions such as making the tool smart phone friendly and altering the structure of the main page for ease of use.
Zainab Abdulsattar’s reflections:
The success of the tool lied in the focus group and literature review contributions because the students’ focus group tool ideas helped to further contribute to the evidence-based self-help ideas gathered from the latter. Importantly, the hope is that the tool can act as an academic aid promoting and improving students’ independence in self-managing feedback in a more positive and productive way. Hopefully this will alleviate feedback-related stress for both now and the future in academic and work settings.
In the future, we hope to expand the prototype tool into a more established feedback-focused tool. To make the tool even more use-friendly, we could consider improving the initial main contents page. For example, presenting the options like ‘I want to work on improving x’ then lead on to the appropriate self-help resource instead of simply starting with the resource options [e.g. problem solving, reflective log].
Will Hughes – School of Built Environment (Construction Management & Engineering)
The personal capture pilot project helped me to develop and test ideas to advance what I had been previously trying using YouTube. One important lesson for me was that shorter duration videos better engage students. I also learned how to record videos featuring more than simply a talking head. Using this technology for augmenting the usual pedagogic techniques was very useful. I would like to replace some of my lecturing using screen-cast videos, but I have learned that there is more to this than simply recording pre-prepared lectures for my students.
My aim was to produce detailed explanations of points too elementary or too complex to address in lectures and to replace some one-to-one meetings. I aspired to produce a series of 5-10 minute videos that responded to specific student questions generated from lectures and emails. One specific idea was to support reflective portfolio writing.
My motivation to join the personal capture project was to acquire screen-casting skills and to better understand the technology.
There were two key groups I chose to produce recordings for:
I tried using the webcam and laptop provided in the pilot. With these, I made some videos using the Mediasite tool, but the video and audio quality were not as high as I would have liked and the editing offered by Mediasite was very primitive, with no opportunity to fix issues like colour grading, for example. I preferred using my own professional-grade camera, microphone and lighting. I realised that I needed much better software than Mediasite and bought a license for Camtasia, which opened up a lot of interesting possibilities and made it possible to achieve what I had in mind.
Dialogue with students was around presenting them with a video and asking them to let me know what they thought, whether it helped and what kind of things they would like me to cover in future.
The most well-received videos were those that summarised assignment guidance in 10-11 minutes. My video on research conceptualization proved popular. The assignment summaries in CEM102 Business of Construction, for a Reflective Portfolio and a Case Study, were very impactful and prompted a lot of student approval.
One unanticipated experience was in using the technology for replacing a lecture cancelled due to bad weather; 66% of the students accessed this 55-minute lecture but for an average view time of only 18 minutes which I found to be a depressing statistic.
Things improved as I progressed. Planned use of personal capture was much better than using it to overcome lecture cancellations. The pedagogical challenge is to figure out how to produce short videos that are useful to students. It was useful to work out how to provide simple overviews of things that would be helpful in the students’ learning and produce short videos based on this. I found filming at home better than filming in the office. I have learned the importance of issuing reminders about Blackboard-posted videos as students can miss the initial announcement and then never see the video produced for them.
I found the Mediasite tool itself clunky and challenging in terms of its permissions, lack of utility and quality.
I still believe personal capture is useful but I am thinking about changing my strategies for how to use it. The changes are not technical put pedagogical. As I move to part-time working and ahve less contact time with students, personal capture may become indispensable for me.
Kat Hall, School of Chemistry, Food and Pharmacy, k.a.hall@reading.ac.uk
The Centre for Inter-Professional Postgraduate Education and Training (CIPPET) provide PGT training for healthcare professionals through a flexible Masters programme built around blended learning modules alongside workplace-based learning and assessment. This project aimed to evolve the department’s approach to delivering one of our clinical skills workshops which sits within a larger 60 credit module. The impact was shown via positive student and staff feedback, as well as interest to develop a standalone module for continuing further learning in advanced clinical skills.
The aim of this project was to use controlled condition assessment approaches to develop behavioural competence at the higher levels of Miller’s pyramid of clinical competence 1.
Miller’s Pyramid of Clinical Competence
The objectives included:
Health Education England are promoting a national strategy to increase the clinical skills training provided to pharmacists, therefore this project aimed to evolve the department’s approach to delivering this workshop. The current module design contained a workshop on clinical skills, but it was loosely designed as a large group exercise which was delivered slightly differently for each cohort. This prevented students from fully embedding their learning through opportunities to practise skills in alongside controlled formative assessment.
Equipment purchase: As part of this project matched funding was received from the School to support the purchase of simulation equipment which meant a range a clinical skills teaching tools could be utilised in the workshops. This step was undertaking collaboratively with the physician associate programme to share learning and support meeting objective 2 across the School.
Workshop design: the workshops were redesigned by the module convenor, Sue Slade, to focus on specific aspects of clinical skills that small groups could focus on with a facilitator. The facilitators were supported to embed the clinical skills equipment within the activities therefore promoting students in active learning activities. The equipment allowed students the opportunity to simulate the skills test to identify if they could demonstrate competence at the Knows How and Shows How level of Miller’s Pyramid of Clinical Competence. Where possible the workshop stations were facilitated by practising clinical practitioners. This step was focused on meeting objectives 1, 2, 3 and 4.
Workbook design: a workbook was produced that students could use to identify core clinical skills they required in their scope of practice and thus needed to practise in the workshop and further in their workplace-based learning. This scaffolding supported their transition to the Does level of Miller’s Pyramid of Clinical Competence. This step was focused on meeting objectives 1 and 3.
All four objectives were met and have since been mapped to the principles of Curriculum Framework to provide evidence of their impact.
Mastery of the discipline / discipline based / contextual: this project has supported the academic team to redesign the workshop around the evolving baseline core knowledge and skills required of students. Doing this collaboratively between programme teams ensures it is fit for purpose.
Personal effectiveness and self-awareness / diverse and inclusive: the positive staff and student feedback received reflects that the workshop provides a better environment for student learning, enabling them to reflect on their experiences and take their learning back to their workplace more easily.
Learning cycle: the student feedback has shown that they want more of this type of training and so the team have designed a new stand-alone module to facilitate extending the impact of increasingly advanced clinical skills training to a wider student cohort.
What went well? The purchase of the equipment and redesigning the workshop was a relatively simple task for an engaged team, and low effort for the potential return in improved experience. By having one lead for the workshop, whilst another wrote the workbook and purchased the equipment, this ensured that staff across the team could contribute as change champions. Recruitment for an advanced nurse practitioner to support the team more broadly was completed quickly and provided support and guidance across the year.
What did not go as well? Whilst the purchase of the equipment and workshop redesign was relatively simple, encouraging clinical practitioners to engage with the workshop proved much harder. We were unable to recruit consistent clinical support which made it harder to fully embed the project aims in a routine approach to teaching the workshop. We considered using the expertise of the physician associate programme team but, as anticipated, timetabling made it impossible to coordinate the staffing needs.
Reflections: The success of the project lay in having the School engaged in supporting the objectives and the programme team invested in improving the workshop. Focusing this project on a small part of the module meant it remained achievable to complete one cycle of change to deliver initial positive outcomes whilst planning for the following cycles of change needed to fully embed the objectives into routine practice.
In planning the next series of workshops, we plan to draw more widely on the University alumni from the physician associate programme to continue the collaborative approach and attract clinical practitioners more willing to support us who are less constrained by timetables and clinical activities.
Based on student and staff feedback there is clearly a desire for more teaching and learning of this approach and being able to launch a new standalone module in 2020 is a successful output of this project.
Miller, G.E. (1990). The assessment of clinical skills/competence/performance. Acad Med, 65(9):S63-7.
Simon Floodgate, Institute of Education, s.floodgate@reading.ac.uk
A form of inter-active, reflective practice for students in which Playback Theatre (an improvisatory form) is used to ‘play back’ individual stories of students’ experiences regarding all aspects of their studies. This process can support emotional literacy and well-being and promote professionalism in students at all levels of study.
To address concerns regarding student well-being and emotional literacy as highlighted both nationally, within the University and the IOE where workload and pressures have specifically impacted upon initial teacher training (ITT) students who are transitioning into teaching professionals.
The pilot year, within the IOE, was focussed upon the training of a student performance group with a couple of performance-workshops undertaken with Secondary ITT students and IOE staff. Both sessions were evaluated and the students involved as the performance team, were also asked to evaluate the benefits to them of engagement in the project. The project enters a second year (2019-20), with further funding, to adapt the contact sessions. This will lead to two different versions of Stories of Our Studies. A full length, two-hour version will incorporate a full Playback Theatre performance of 1-1/2 hours duration in a more public setting. A second shorter version will align the performance elements with discursive and written aspects focussed upon critical incident analysis (Lister and Crisp, 2007). This will blend the elements for more captive audiences within module teaching sessions.
As a pilot project, Stories of Our Studies achieved its objectives. A student team was trained to deliver the contact sessions alongside the project leader. The project was presented to both PGCE Secondary ITT students and IOE staff, enabling feedback from different perspectives. Staff were able to appreciate the potential impact upon student well-being. The PGCE students were able to effectively reflect upon their learning, in particular focussing upon their school teaching placements. They were able to subjectively reflect upon how these experiences felt to them but also objectively appreciate what occurred, how their experiences were similar or different to others and to be able to consider themselves as professional teachers soon to embark upon their chosen profession. The TLDF priorities 2.2 and 2.3 were both met.
The enthusiasm and willingness of the UG students who trained in the form was exceptional and their empathy and artistry were commented upon following both performance-workshops. This was a major factor in the pilot’s success. The structure of the session with the main performance aspect following some Morenian sociometry facilitated a relaxed and intimate atmosphere thus enabling audience members to openly share. The use of the form – Playback Theatre – was vital to the success of the pilot.
Although participants gained a lot from their engagement in the session, there is a further need to develop the sustainability of the reflective process. To this end the project will be developed into longer and shorter iterations (as mentioned above). There remains some difficulty in encouraging students to attend extra-curricular sessions and, for many, to attend events in which drama/theatre are mentioned. This is a difficulty in attracting both student-performers and audience members. Word of mouth will help and, like a stone gathering moss, momentum will attract more interest and students to engage with it.
See above. The project has entered a second year with further TL enhancement (mini) funding. It is evolving with the incorporation of critical incident analysis and a further blending of the performance and written reflection elements.
We already have more performance-workshops booked in the diary for 2019-20 than for last year, including presentation at the University’s T&L conference in January 2020.
Contact has been made with the RUSU society, Open Minds, to investigate the potential of some performances to a larger student audience outside of timetabled teaching.
The performance-workshop, photographed last year, will be filmed to create a marketing online clip to promote the project. Recruitment of new student-performer members has already begun.
Photo of Playback Theatre in action