How should MSc Placements be assessed? Gathering the views of students to inform assessment

19 February 202419 February 2024 By michaelkilmister Comment Closed

By: Paul Jenkins, School of Psychology & Clinical Language Sciences, p.jenkins@reading.ac.uk

two women sitting on stairs having a conversation — Image credit: Buro Millennial on Pexels.

Overview

The School of Psychology and Clinical Language Sciences (PCLS) offers several postgraduate degree programmes, nearly all of which include a placement element. Getting the assessment right is an important challenge to fairly evaluating students on placement. As part of an ongoing review of programmes with placement components, a piece of work was commissioned to look at how placements should be assessed within PCLS.

Objectives

The primary aims of this project were to explore:

What elements of placements MSc students felt were important to assess; and
How MSc students felt these elements should be assessed.

Context

As a ‘taught’ component of the course, any assessment needs to be carefully planned and contribute “directly to learning and skill development” (UoR, 2023). Student feedback indicated that the current method of assessing placements, which comprises a written report of what was done and learned on placement, was unsatisfactory. For instance, students felt that it did not reflect the amount of work put in over the course of the placement and that the final grade was too reliant on one piece of written work.

It was felt that gaining insight into current students’ views would be helpful to inform future changes to the way(s) in which MSc placements might be assessed, making this process proportionate and more useful for students.

Implementation

In February 2023, a grant from the UoR T&L Initiatives Fund was awarded to address the question of how MSc placements should be assessed.

A focus group discussion was conducted in June 2023, with participants recruited from PCLS MSc students. The focus group lasted around 45 minutes. In addition, a 1:1 interview was held in July 2023 with another individual who wanted to share their views on the subject, and this is included to add detail to the data obtained from the focus group.

To frame the focus group and interview, open-ended questions were developed to explore participants’ experiences, opinions, and thoughts regarding placement and its assessment. The facilitator (a member of staff within PCLS) was present to encourage a relaxed atmosphere and supplement prepared questions with prompts to gather participants’ views and pursue themes relevant to the research questions. The following is a sample of the questions asked during the interview:

What are the important elements of an MSc placement to be assessed?
How do you think MSc placements should be assessed?

The focus group was audio-recorded and the facilitator also kept notes to help keep track of themes and provide a more holistic picture of the discussion (Kornbluh, 2023). The students were also given a document on different types of assessment and an exemplar of how a placement might be assessed to act as ‘stimulus material’ to prompt detailed discussion of their views on assessment.

Impact

The findings of the discussions provided insight into how students think placements should be assessed. In terms what students considered important to be assessed, several different themes emerged:

Assessing what was learned

Students talked about the importance of assessing what was learned, as opposed to a more cursory assessment of the time or activities spent in placement; for instance if: “technically, you put in the work but you didn’t actually apply it to anything”. They reflected on the different environments and services within which placements took place, such as some being online and others being conducted in-person, and the importance of asking students “to prove” that they have engaged with placement. The importance of certain skills (e.g., teamworking, presentation skills) learned on placement was highlighted, and also how such skills relate to students’ futures.

Reflecting on one’s own development

Several students commented on how they have developed over the course of placement, and how this could be included in the assessment. For instance, one student suggested that assessments could cover “what skills are we learning and how much are we able to apply it… and how we’re changing”. Another noted discussions they have with their supervisors, whereby they “don’t just talk about what I do… [but] also some sort of reflection,” and that this brings in “reflection of how you see yourself”.

Capturing diversity of experiences

The discussion also covered the reality that students will have different experiences of placement and how it can be “a very subjective experience,” including different types and levels of supervision. For instance, one student commented that “the difference between person to person doesn’t always end in […] what they’re doing but also where they started from, because we also came into the programme with very different experiences”. Students also highlighted differences in effort put in by those on placement, sharing the perception that there were some students “who are doing everything they possibly can” and others who “slowly move to the back… waiting for things to be handed to them”.

As part of the project, students also discussed how these skills and elements of placement should be assessed and, again, several themes arose:

Continuous assessment

Students discussed having the opportunity to reflect ‘as they go’ and potential problems with a unitary, retrospective assessment. Whilst they felt that having a reflective piece is “a nice idea,” one student commented how a lot of experiences gained on placement are difficult to recall at the time of submission. They were also wary of having too much overlap between pieces of assessment, such as a reflective report and report of activities, and one student suggested being “forced to keep track of what you have been doing… in a detailed manner”.

Having been offered a list of potential assessment types to review in the focus group, one student felt that Reflective Diaries could be a better approach, perhaps used alongside an hours log. Another suggested that Learning Logs with “certain points to learn about” could be helpful, perhaps covering “small reports on small things”. Another suggested a “spaced out diary… or some form of input from our supervisor” could be of use, although also stated that they were unsure “how feasible that would be”. It was also suggested that a website (or blog) could be used to help students log experiences and remain accountable. Of note, some students chose to do this independently, with one saying: “I keep a log for myself”.

Oral presentations

Many students mentioned advantages of an oral presentation over written work, including being “better able to express what I’m doing when I speak”. Another commented that “when you write, you downplay” what was done on placement and that an oral form of assessment can be less constrained by “academic rules”. Another student agreed, saying that a presentation would “let someone express [their experience] much better” and another concluded: “I think just talking would be better [than a written assignment]”.

Students suggested that oral presentations offer a chance to “talk through your experience” and also to field questions (e.g., “What do you think you specifically learned?”), which “makes you reflect a lot more”. They also commented on the advantages of having other individuals present. A student noted that presenting in a group means that you “get to see what other people have been doing [and] how they’ve developed their skills” which could even “change your perspective”. It was commented that this approach can be “helpful to your peers as well, not just you”.

In a similar vein, one student suggested a viva voce (a one-to-one oral examination) whereby students “talk to our supervisors… and have that discussion” about their experiences.

Assessing the thoroughness of the experience

One student suggested that having written assignments can limit introspection, and get one “writing it for the sake of having a reflective piece to submit” rather than discussing “how much have I grown”. By contrast, they suggested that, in oral presentations, “flow is better – easier – and it really gives you cause to think about how you have developed”. Further reflecting on oral presentations, one student commented that “it’s up to you how you present it and how you convey how much you’ve learned, what you’ve learned, how much you’ve grown” and “how you justify what you’ve done in your placement hours”.

Reflections

The insight gained from this work has proved invaluable when formulating assessment for the coming academic year. Students’ views on the possibility of interpersonal assessment has informed the structure of oral presentations where students are given the opportunity to discuss an aspect of placement in front of their peers. The marking criteria have been developed to incorporate some of this feedback, such as inclusion of autonomy, personal development, and showing relevant skills.

Whilst it only represents a small study, some practical suggestions could be proposed. For instance, when evidencing and discussing their placement experiences, students were clear that oral presentation offers several advantages over written methods (a more common approach to work-based assessment; Ferns & Moore, 2012). The importance of assessing skills development over time was highlighted, which could be considered when setting and providing structure for both formal and informal assessment (e.g., Bates et al., 2013). Finally, it is perhaps also important for educators to keep in mind that students begin placement with different experiences, variation which has the potential to impact both their learning and achievement.

Follow up

The summer of 2024 will be the first-time oral presentations have run for several ‘placement’ modules. We shall continue to refine the assessment itself (and marking criteria) based on further feedback and look into whether concerns about the written reflective piece remain; if so, an assessment that relies more on continuous engagement could be considered.

References

Bates, J. et al. (2013). Student perceptions of assessment and feedback in longitudinal integrated clerkships. Medical Education, 47, 362–374. https://doi.org/10.1111/medu.12087
Ferns, S., & Moore, K. (2012). Assessing student outcomes in fieldwork placements: An overview of current practice. Asia-Pacific Journal of Cooperative Education, 13(4), 207–224.
Kornbluh, M. (2023). Facilitation strategies for conducting focus groups attending to issues of power. Qualitative Research in Psychology, 20, 1–20. https://doi.org/10.1080/14780887.2022.2066036
University of Reading. (2023, December). Assessment and the Curriculum Framework. https://sites.reading.ac.uk/curriculum-framework/assessment/

Using constructivism to achieve a decolonised accounting curriculum

8 December 202318 February 2024 By michaelkilmister Comment Closed

By: Ekililu Salifu, Henley Business School, e.salifu@henley.ac.uk

Men sitting on chairs with back to camera — © Photo by Sam Balye on Unsplash

Overview

This article explores using constructivism as a pedagogical approach to achieving the objective of a decolonised accounting curriculum. It discusses how constructivism can be used to effectively outline the constraints of the received perception of accounting as a pseudo-technical subject while making room for alternative representations.

Objectives

The primary aim of this activity was to explore the use of constructivism learning theory to achieve a decolonised accounting curriculum. The next objective was to design and effectively deliver a more inclusive globalised curriculum for post-graduate financial reporting in particular, and accounting in general.

Context

Accounting curriculum is perceived to be pseudo-technical, relying on the application of technical rules and principles that are universally accepted. This is the received view of our postgraduate accounting students, over 90% of whom are from an international background. Student feedback suggested that while they wanted to learn and apply conventional financial reporting and accounting, they also wanted to see a representation of themselves in some of the discussions. During the delivery and redesign of the curriculum for ACM002 Financial Reporting and Regulation (now ACM006 International Financial Reporting and Regulation), I explored the use of constructivism as a pragmatic pedagogical approach to explore the constraints of this notion while making room for the generation of alternative explanations. ACM002 (now ACM006) is a compulsory financial reporting module for MSc International Accounting and Finance students, and currently has 26 students registered on it.

Implementation

Decolonisation can take on different meanings but is used in this context to mean the recognition of the constraints placed by monocultural and largely westernised perspectives or hierarchies in accounting and the making room for alternative representations. Decolonising the accounting curriculum faces unique disciplinary constraints, as the largely Western knowledge systems we pass on are considered to be ‘universal’, especially in the wake of the near-universal acceptance of international financial reporting standards. A decolonised accounting curriculum needs to emphasise its ability to meet local needs and cultivate globally transferable skills.

I started with an informal focus group with some students from the 2021/2022 cohort, to collate feedback on what a decolonised financial reporting curriculum would mean to them, among other objectives. Students argued against the development and delivery of an overtly decolonised curriculum that continuously recognised the constraints of the existing curriculum, and especially discussed alternative representation. This was primarily borne from their belief while the present curriculum had hegemonic dispositions, it was still necessary as its completion would enhance their global competitiveness. Furthermore, for a decolonised curriculum to be meaningful to the student, they needed to see their own experiences represented in the discussions.

Constructivism offered a solution to designing and delivering a more decolonised curriculum. Constructivism, with its focus on student-centred learning, suggests that humans construct knowledge and meaning from their experience. As a learning theory, it suggests that students learn by relating new information to what they already know. In its ability to fosters active and collaborative learning, constructivism allows students to self-identify with co-produced knowledge.

The delivery of ACM002 was primarily lecturer-led, with relatively limited opportunities for students to reflect on what was being taught. However, scholars generally recognise that knowledge is co-produced, and a lecture-only mode of delivery is not ideal for the optimal co-production of knowledge. The amount of time that could be dedicated to student engagement in debates during lectures (as opposed to workshops) was limited, especially considering the content that still needed to be delivered.

The starting point of decolonising the curriculum was thus to rename the module from Financial Reporting and Regulation to International Financial Reporting and Regulation, to highlight the inclusivity within the module. Next was to expand the reading lists to include more critical debates on some of the module content. Workshop sessions, with the object of fostering debates among the students were introduced. In these sessions, students engaged in more critical discussions when they were able to call on their own experiences and relate those to the discussions at hand. This was in sharp contrast to when critical discussions were relayed to them by the lecturer.

Impact

Adopting constructivism significantly allowed for the curriculum to be relatively decolonised and overcome some of the student resistance. As the composition of students changes year-by-year, utilising static module structure and composition may not achieve the objectives of a decolonised curriculum for each cohort. Relying on only module renaming and diversifying the module content and material risks alienating some students who may not recognise a representation of their own experiences in what is being taught.

Leaning on the canon of the coproduction of knowledge between instructors and students (see de Carvalho et al., 2016; Padilla, 2019; Shahjahan et al., 2022)., encouraging and offering students the opportunity to call upon and debate key issues within financial reporting was vital to the paradox of balancing professional (often western-centric) knowledge, socialisation, and subjugated community based and socially knowledge.

Reflections

Decolonising curriculum requires the construction of an inclusive curriculum beyond dominant knowledge systems, as well as the cultivation of an environment that fosters relational teaching and learning. This means that it is a continuous process that requires constant iterations based on student-teacher interactions, recognising the differences in the lived experiences of individuals and the impact that might have on the learning process.

Follow up

I intend to run a short survey for this cohort at the end of the term, to evaluate the extent to which they are able to self-identify their individual and country context within the discussions we have covered in the module. I also intend to make a presentation on decolonisation and constructivism at the departmental level also, to assess the receptiveness of this approach, and to close the loop on the side of the academics.

References

Bada, S. O. & Olusegun, S. (2015). Constructivism learning theory: A paradigm for teaching and learning. Journal of Research & Method in Education, 5(6), 66-70.
Charles, E. (2019). Decolonizing the curriculum. Insights, 32, 24.
de Carvalho, J. J., Cohen, L. B., Correa, A. F., Chada, S., & Nakayama, P. (2016). The meeting of knowledges as a contribution to ethnomusicology and music education. World of Music, 5(1), 111–133.
de Carvalho, J. J., & Florez-Florez, J. (2014). The meeting of knowledges: A project for the decolonization of universities in Latin America. Postcolonial Studies, 17(2),122–139. https://doi.org/10.1080/13688790.2014.966411
Padilla, N. L. (2019). Decolonizing indigenous education: An Indigenous pluriversity within a university in Cauca, Colombia. Social & Cultural Geography, 22(4), 523–544. https://doi.org/10.1080/14649365.2019.1601244
Shahjahan, R. A., Estera, A. L., Surla, K. L. & Edwards, K. T. (2022). “Decolonizing” curriculum and pedagogy: A comparative review across disciplines and global higher education contexts. Review of Educational Research, 92(1), 73-113.
Subedi, B. (2013). Decolonizing the curriculum for global perspectives. Educational Theory, 63(6), 621-638.

Whose getting the development here? Utlising SoTL frameworks to reflect on personal development modules for senior level apprentices

31 August 20238 September 2023 By michaelkilmister Comment Closed

By: Dr Elizabeth Houldsworth, Associate Professor of Leadership, Organisations and Behaviour at Henley Business School, liz.houldsworth@henley.ac.uk

Group of people sitting around a table looking at paper and a laptop — Photograph from a staff development workshop at Whiteknights Campus, July 2023 © Liz Houldsworth

Overview

This entry offers an overview of a collaborative study by Dr Elizabeth Houldsworth (Henley Business School) and Emma Watton (Lancaster University Management School)

The researchers met at both their institutions to review personal development (PD) materials and to hear reports from learners on the impact of PD modules on their development as reflective practitioners. An exploratory empirical study was designed using Henley Business School students to consider the relative degree of reflective thinking displayed by learners in their PD assignments.

The approach was informed by scholarship of teaching and learning (SoTL) frameworks designed by Boyer (1990) and Kern et al. (2015) and, in particular, the need to ‘go public’ about the study. The collaborators reflected that the study had an unexpected positive impact on their own personal development as reflective practitioners.

Selfie photograph of Liz Houldsworth and Emma Watton. — Liz Houldsworth (left) and Emma Watton (right) © Liz Houldsworth

The project has had the following outputs, which are available for download by selecting the links:

1. A peer reviewed presentation to the Research in Management Learning and Education Conference in Banff June 2022 (abstract)
2. A peer reviewed conference presentation to Advance HE on the SoTL aspects of the collaboration (abstract)
3. Staff development workshops at Henley Business School and Lancaster University Management School. Materials from the final one, held at Whiteknights Campus in July 2023, is available for download here (PDF of PowerPoint slides)

References

Boyer, E.L. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton University Press.
Kern, B. Mettetal G, Dixson M, et al. (2015). The role of SoTL in the academy: Upon the 25th anniversary of Boyer’s Scholarship Reconsidered. Journal of the Scholarship of Teaching and Learning, 15(3): 1-14.

This entry is a first for the T&L Exchange. Liberated from the constraints of the usual case study, contributors are encouraged to present their writing in a style and format of their choosing. This format offers colleagues an opportunity to quickly share their research, practice and ideas in a way that is authentic to their research or project. These entries might be intended to inspire interest, stimulate debate, foster collaboration, propose new ideas or even entertain. If you would like to submit an entry like this, please email CQSDTandL@reading.ac.uk.

Reframing success in a partnership project

17 July 20237 June 2024 By michaelkilmister Comment Closed

Associate Professor Amanda Millmore, School of Law, a.millmore@reading.ac.uk

Update June 2024

This submission is now a published journal article co-authored with student partners. The link is here and the complete reference can be found below: ‘Reframing Success in A Pivoting Partnership – Student Mentors Trying to Engage: A Tale of Trial and Error’.

Objectives

Curriculum development – reviewing & designing materials and the Blackboard framework for a new elective first year module.
Peer mentoring – student partners in Part 2 offering support to students on the module, embedded within the module by linking student partners directly with each seminar group and including them in online drop-ins and in-person teaching.

Context

During the Covid-19 pandemic, our students had struggled with their sense of belonging, not feeling part of the School of Law community due to lockdowns, online teaching and restrictions on gathering socially. We were creating a new elective, Part 1 law module called “Law and Society”, and we wanted to work with students to develop the module. We were also conscious that we needed to improve support for our new first-year students to ease their transition into university and their studies by enhancing their sense of belonging. We came up with the idea of supporting the new students by building bridges with the cohort in the year above.

Implementation

Curriculum Design – the student partners worked together with staff to review the materials we had prepared and giving their thoughts on what would be helpful and work for the new Part 1 students.

Peer Mentoring – we embedded student partners as mentors with individual seminar groups. We introduced them online with a dedicated “Mentor” section on Blackboard, hosted a “Q&A” Padlet board for students to interact anonymously if they wished. The module was designed with the mentors embedded into it. Student partners were each paired with one of the teaching academics on the module to provide support. Mentors were timetabled to join online optional drop-in sessions (and the session was headed “Meet the Mentors”) and compulsory seminars to offer support with groupwork and formative activities. Academic staff highlighted the benefits of peer support and promoted the mentors and how they could help, while mentors encouraged formal and informal contact with the students in their designated classes.

When student mentees did not attend the optional drop-in (we had more student partners attending than we did students enrolled on the module) we pivoted to the student partners sharing their advice for new students, which we recorded in a document that we shared on Blackboard.

Impact

Curriculum Design – this aspect of the project was very successful, with student partners feeding into the design of the Blackboard module, reviewing the module materials to ensure that they were engaging and pitched at the appropriate level and on student recommendation we ensured the provision of clickable Talis reading lists.

Peer Mentoring – this aspect fell flat, as the Part 1 students did not want to be mentored. They did not attend sessions where the mentors were offering support, declined offers of help (even when they volunteered to join a WhatsApp group) and the student partners felt that we were flogging a dead horse trying to mentor first-year students who did not want to be mentored. Student partners then pivoted to carry out some research to find out what the barriers to engagement with the project were; beset with difficulties in seeking feedback from the Part 1 students who did not respond to questionnaires, offers of coffee and cake or focus groups, the few who did participate explained that they just did not feel the need for that kind of peer support.

Reflection

Whilst the mentoring aspect of the project did not land successfully with the Part 1 students, it was not due to problems with the partnership or even the design of the project, it was just that the Part 1 cohort did not want the support that we were offering. This may be peculiar to this particular cohort, who had been significantly affected by Covid at school, but it was not for want of trying.

Whilst not one of our explicit aims, the notable success of our partnership is the value to the student partners who worked as module designers, mentors and researchers, these students have had the opportunity to disseminate their experiences at conferences and in writing and can see real benefits to their partnership experiences, and they have developed tangible employability attributes, not least a high degree of resilience.

Amanda Millmore and student partners before presenting at the Change Agents’ Network conference 2022

Follow-up

Student partners co-presented this project at the CAN (Change Agents’ Network) conference at UCL in summer 2022 and we have now co-authored a journal article sharing our experiences.

We have continued with the good curriculum developments in the module, which continues to grow from strength to strength. The mentoring aspect of the project has not continued, but instead we ensure to signpost our students to their STaR mentors and PAL leaders for peer support.

Partnership working in the School of Law continues to be business as usual, and the hiccups on this project have not deterred us from trying new things with our student partners, ensuring that we see the benefits of partnership as part of the process and the positives for the partners.

References and links

We contributed to a blog after the CAN conference: CAN Case Study: A Pivoting Partnership – Student Mentors Trying to Engage: a Tale of Trial & Error | CAN 2022 (ucl.ac.uk)

Millmore, A., Collyer, B., Delbridge, E., Khan, A., Patil, I. and Williams, M. (2024) Reframing success in a pivoting partnership – student mentors trying to engage: a tale of trial and error. The Journal of Educational Innovation, Partnership and Change, 9(1). https://journals.studentengagement.org.uk/index.php/studentchangeagents/article/view/1208

If you’d like to know more about staff-student partnership in the School of Law, you can reach me at a.millmore@reading.ac.uk

Decentring Ableism: Creative Applications of Film Accessibility in Film/TV Practical Teaching

12 June 202320 July 2023 By michaelkilmister Comment Closed

Shweta Gosh, Department of Film, Theatre & Television, shweta.ghosh@reading.ac.uk

Overview

In this blogpost, Lecturer in Screen Practices and Industries Shweta Ghosh discusses her recent exploration of a new approach to teach film sound design using captions. Based on Shweta’s research on filmmaking and accessibility, this exploration serves as the foundation for a toolkit of film practice teaching methods that she intends to develop through 2023/24, which draw on principles of universal design and decentring ableism in the creative industries.

Objectives

The primary aim of the activity was to explore possible pedagogical applications of research on film accessibility in practical Film/TV teaching at the Department of Film, Theatre & Television. Key objectives were:

To develop student awareness of disability and Deaf culture, and the need for accessibility
To develop student capacities for confident exploration of diversities in audio-visual experience and development of creative ideas based on accessible filmmaking principles
To build accessibility into creative work with a foundational approach rather than incorporating it as an afterthought

Context

My doctoral research on filmmaking and accessibility revealed that filmmaking continues to centre nondisabled perspectives and practices, both on and off screen. Accessibility measures such as captions and audio description are often inserted in film/TV/video content as afterthoughts and accessible filmmaking research as well as practice demonstrates that the same measures considered at early stages (ideation, pre-production and planning) can make film and TV outcomes more accessible by default.

Additionally, accessibility measures can offer exciting possibilities to develop creative aspects of one’s work. The University of Reading Curriculum Framework outlines the need for teaching and learning practices to be accessible to all, and a key programme learning outcome of the new BA in Film & Television at the Department of Film, Theatre & Television programme is to “Create creative practice that is informed by an understanding of accessibility, sustainability and/or social engagement”. In alignment with these visions and outcomes, my exploratory activity was aimed at understanding student and staff response to the use of accessible filmmaking methods in film/TV practice teaching and learning.

Implementation

The activity involved working with two tutorial groups in the Part 1 Film/TV practice in Autumn term 2022 called ‘Introduction to Filmmaking’ (FT1ITF).

The idea was to explore the creative potential and inclusive outcomes of using creative captioning in Film/TV outputs. Group A and B tutors (Dr. James Kenward and I respectively) used a video by Artist Christine Sun Kim on rewriting closed captions from a Deaf perspective as a prompt for seminar discussion (released in advance on Blackboard), and facilitated student reflection on how captions can communicate diverse sound perspectives and the filmmaker’s creative intentions.

Initial discussion explored how the use of captions is widespread and how it makes audio-visual content accessible for Deaf viewers. This helped gauge student awareness and understanding of disability rights and accessibility more generally. Further discussion explored creative dimensions of captions in relation to ‘aural worlds’ (i.e., how each ‘world’ within an audio-visual work is built with different sound components and perspectives).

Students were then encouraged to identify the different components of the aural world in the video as well as the classroom, and map these on to a sound design template. This template, used by Part 1 students as a formative development blog submission, facilitates thinking and planning for practical project sound design, where each column represents a component of the aural world (ambient sound, voice/dialogue, etc.) and which can subsequently be mapped on to sound design and mixing software.

A discussion connecting these various elements enabled students to apply insights to develop creative ideas for the sound design of their own practical projects. Questions used by tutors to facilitate discussion were based on the following themes:

How do the captions in this video describe the creator’s intention? For example, what is the intended mood and tone with respect to the violin music in the captions before and after Christine Sun Kim changes them?
How do the detailed captions help us imagine / create an aural world that is more complex + inclusive?
If you had to caption your 10-shot sequence, how would you caption it with your sound design intentions? Have a go based on your current rough cut (in class / before your next edit session with the rough cut copy / during the edit with the captioning tool).
How can your ‘captioned’ intentions be mapped on to a sound design plan (esp. Mood section)?

The activity was successful in achieving its intended objectives. Practical subgroups in A tutorial group used captions during the workshop to develop creative intentions for sound design. One of the practical subgroups in tutorial group B explored the use of creative captions in their final practical output. While their use of captions was not assessed summatively, formative feedback was provided at an editing supervision meeting, and their attempt to understand and engage with captioning was positively recognised.

Reflection

Positive feedback from the group A tutor summarises the strengths of this activity and reaffirms that this can be an effective and interesting way to teach students film practice and accessibility.

“This was a very useful exercise and encouraged students to think about their creative practice in new and inventive ways. Students were not ‘taught’ accessibility, but utilised standard accessible filmmaking practice as a foundation to explore sound design choices in their films. Accessible practice was thus a given, ingrained into the work itself, rather than something to be viewed as separate or additional.

As the exercise confronts practitioners’ inherent biases as well as their expectations for the viewer, it works effectively to encourage students to critically analyse and evaluate their sound design choices in a targeted fashion. Given improving the quality of students’ sound design is a specific area of focus for the department, this exercise would be beneficial for students across practical modules.”

This exploratory project has also confirmed that there is an appetite amongst students to understand and engage with audio-visual perspectives that are different to their own, whether on and off screen. This is crucial to develop future film/TV makers whose practices are built on the principles of empathy and inclusion.

Follow up

The verbal feedback from students and interesting themes emerging from the trail this year (such as creative intentions, creative control, accessibility tools as enhancers or limiters of creativity), will be used to develop a detailed yet flexible version of this exercise, which can be used in next year’s Introduction to Filmmaking module as well as adapted for relevant Part 2, 3 and MA Film/TV practice modules. A seminar + workshop format (or critical discussion + sound design template application activity) will support students to connect critical themes to creative applications fruitfully. Student and staff feedback at the end of these sessions will be invited to further my understanding of engagement with accessibility methods and how these might enhance creativity and empathy, as well as key pedagogical challenges.

If you’d like to know more or would like to talk about this project, you can reach me at shweta.ghosh@reading.ac.uk or my personal website.

Overview

Our vocational postgraduate courses in Cognitive Behavioural Therapy include University Directed Learning (UDL) days that are completed within the placement setting (e.g. their NHS trust). A qualitative student feedback survey allowed us to collaboratively adapt this format, with favourable outcomes in how interesting, enjoyable and useful the students found the day.

Objectives

Our objectives were as follows:

-To ascertain how interesting, enjoyable and useful the UDL days were, as perceived by the students, based on pedagogical findings that students engage best and are most satisfied, if these characteristics are met (e.g. Ramsden, 2003).

-To make improvements to the UDL days based on qualitative student feedback.

-To ascertain whether our improvements had made the UDL days more interesting, enjoyable and useful, as perceived by the next cohort of students.

Context

The Educational Mental Health Practitioner (EMHP) and Children’s Wellbeing Practitioner (CWP) programmes are one-year vocational postgraduate courses. The students are employed by an NHS trust, local authority or charity, and study at UoR to become qualified mental health practitioners.

UDL days make up a small proportion of the teaching days. They are self-guided teaching days, usually containing elements of e-learning, designed to complement and consolidate face to face teaching (live or remote). A combination of learning methods, including e-learning, is shown to be effective in increasing clinical skills (e.g. Sheikhaboumasoudi et al., 2018).

UDL days had been poorly received by our two 2019/2020 cohorts, according to feedback in the student rep meetings and Mentimeter feedback after each UDL e.g. comments included: ‘there was too much [content] for one day’, ‘I felt pressured to fill [the form] out rather than focussing on the readings themselves’ and ‘[the reflective form] was too long and too detailed’. Whilst this gave us some ideas on changes to make, I was aware of the low completion rates of the Mentimeter feedback. Therefore, to hear from more voices, we decided to create a specific feedback survey about the UDLs to help us make amendments in a collaborative way.

Implementation

We started by creating a survey for the current students to ascertain their views on how interesting, enjoyable and useful the UDL days were. We also had qualitative questions regarding what they liked and disliked and ideas for specific improvements.

I then led a meeting with my course team to explore the key findings. We agreed to make several changes based on the specific feedback, such as:

– variety of activities (not purely e-learning, but roleplays, videos, self-practice self-reflection tasks, group seminars run by lecturers, etc, to provide a more engaging day)
– fewer activities (we aimed for one main activity for every 1-1.5 hours to manage workload demands)
– an option to complete the programme’s reflective form (designed to be more simple, by asking them to provide their own notes on each task) or provide their notes in a format of their choice (e.g. mindmaps, vlogs, etc) to increase accessibility.
– share these reflections on a discussion board for other students and the lecturer to comment on.

We were unable to implement these changes to the current cohort as they had finished all their UDL days in the timetable, so made the changes for the following cohorts in 2020/2021.

We then sought their feedback via a new survey to ascertain their views on how interesting, enjoyable and useful the UDLs are, with additional questions relating to specific feedback on the new elements.

Impact

The survey results for the newer cohorts were much more positive than the original cohort, after changes were made to the UDL format.

There was a significant increase in how interesting, enjoyable and useful the students found the days.

The trainees also largely agreed that the UDLs had an appropriate workload, e.g. one task per 1-1.5 hours.

They also largely agreed that UDLs included interactive and varied tasks. This finding is in contrast to some of the aforementioned literature of the importance of e-learning, and it must be remembered that too much e-learning can be less engaging for trainees.

The students also praised the simple reflective form as a helpful tool, and many appreciated the option to submit notes in their own preferred way.

Although we neglected to explore the role of the lecturer feedback in the new UDL survey, research shows that this makes for a more engaging e-learning session (Dixson, 2010), and may explain why the UDLs were now more favourable.

Moreover, the process of collecting data from the students via a feedback form seemed effective, in that we used feedback to adapt the specific teaching method, thus improving student satisfaction. Pedagogical research shows the importance of using qualitative questions (instead of, or as well as, quantitative methods) to elicit student feedback (Steyn et al., 2019).

Reflection

Overall, this redesign was successful, which may be down to the fact we used the student voice to make meaningful changes. This is in line with Floden’s (2017) research that student feedback can help to improve courses.

Furthermore, the changes we have made are in line with effective practice amongst other courses and universities, e.g. appropriate workload (Ginn et al., 2007), student choice of discussion format (Lin & Overbaugh, 2007), accessibility of resources (Mahmood et al., 2012) and lecturer interaction (Dixson, 2010).

There is a possible limitation in this case study, in that our more recent cohorts are generally happier on the course, and therefore may be more positive about the UDL. In future projects, it would be useful if we can notice themes within module evaluation/student rep meetings earlier, to then elicit specific survey feedback earlier in the course and make amendments sooner, allowing feedback from the same cohort.

In future variations of the survey, I would also wish to explicitly ask how trainees find sharing reflections on the Blackboard discussion groups, as this is one change we had not elicited feedback on.

Follow Ups

We have continued to utilise these changes in the UDL format with future cohorts, e.g. reduced workload, variety of activities, simplified forms, choice of discussion format and lecturer interaction. We no longer receive concerns about these days in the student rep meetings since the original cohort. The Mentimeter feedback at the end of each UDL is generally positive, with one person recently commenting: ‘this was a very engaging day’.

References

References:

Dixson, M. D. (2010). Creating effective student engagement in online courses: What do students find engaging?. Journal of the Scholarship of Teaching and Learning, 1-13.

Flodén, J. (2017). The impact of student feedback on teaching in higher education. Assessment & Evaluation in Higher Education, 42(7), 1054-1068.

Ginns, P., Prosser, M., & Barrie, S. (2007). Students’ perceptions of teaching quality in higher education: The perspective of currently enrolled students. Studies in higher education, 32(5), 603-615.

Lin, S. Y., & Overbaugh, R. C. (2007). The effect of student choice of online discussion format on tiered achievement and student satisfaction. Journal of Research on technology in Education, 39(4), 399-415.

Mahmood, A., Mahmood, S. T., & Malik, A. B. (2012). A comparative study of student satisfaction level in distance learning and live classroom at higher education level. Turkish Online Journal of Distance Education, 13(1), 128-136.

Ramsden, P. (2003). Learning to teach in higher education. Routledge.

Sheikhaboumasoudi, R., Bagheri, M., Hosseini, S. A., Ashouri, E., & Elahi, N. (2018). Improving nursing students’ learning outcomes in fundamentals of nursing course through combination of traditional and e-learning methods. Iranian journal of nursing and midwifery research, 23(3), 217.

Steyn, C., Davies, C., & Sambo, A. (2019). Eliciting student feedback for course development: the application of a qualitative course evaluation tool among business research students. Assessment & Evaluation in Higher Education, 44(1), 11-24.

Links

CWI website: https://sites.reading.ac.uk/charlie-waller-institute/

The One Where a Timetable Merger Gives Rise to a Curriculum Implementation Review

4 March 2022 By aaroncooper Comment Closed

Emma-Jayne Conway, James Kachellek and Tamara Wiehe

t.wiehe@reading.ac.uk

Overview

Staff and students in CWI collaborated on a project initially designed to merge two timetables of sister programmes to aid cross programme working (objective 1) but gave rise to the perfect opportunity to review the way our PWP curriculum is implemented following the pandemic (objective 2). In this blog, we reflect on achieving both objectives within our original timeframe!

Objectives

1. To create a single timetable to aid cross-programme working for teaching and administrative staff.

2. To review curriculum implementation including structure and modality on a modular and programme level with all key stakeholders.

Context

In response to a departmental restructure, we required more efficient ways of working across programmes starting with a uniform timetable. Early on, the project evolved to also review the structure and modality of the curriculum. Our two sister PWP training programmes (one undergraduate and one postgraduate) are virtually identical with a few exceptions but historically had been managed separately.

Over the course of 2021, we planned, designed, and implemented a timetable merger for our September cohorts. This impacted on 3 modules (4 for undergraduates) that form the PWP training year for the MSci Applied Psychology (Clinical) students and the Postgraduate/graduate Certificate in Evidence-Based Psychological Treatments (IAPT Pathway).

Taking both Higher Education and Mental Health Care processes into consideration was no easy feat, including those specific to University of Reading (e.g., term dates), our national PWP curriculum specifying the content and learning outcomes for our 26 teaching days and 19 study days, and British Psychological Society (BPS) accreditation requirements. Modality was a particularly important topic throughout this project, taking key learnings from remote delivery during the pandemic as well as awaiting guidance from our professional accrediting body.

Overall, it served as an excellent opportunity to work collaboratively with staff and students to review the implementation of PWP training at the University of Reading.

Implementation

Early 2021: The PWP team met on several occasions to discuss the possibility of merging the two timetables, including transitioning to a “blended” format of online and face-to-face teaching post-Covid. We set out a timeline for planning, designing, and implementing the project.
Advice was sought from the Director of Training in CWI and colleagues in Academic Development, CQSD based on their experience of timetable mergers and a green light was given based on our draft plans!
Several options were considered before the final format was arrived at: Face-to-face teaching is weighted towards the first module/term with progressive increase to the online taught content as the course progresses. (Rationale supplied elsewhere in this blog).
The educator team were able to draw on feedback from online teaching to gauge the attitude of the student body to online learning, as well as expectations and concerns related to a return to the classroom (see Impact, below). The student voice was important in terms of utilising partnership to create meaningful change to curriculum implementation. However, engaging professional practice students via the course reps was a challenge due to time constraints, therefore, we were able to engage graduate instead. This is something we would consider earlier on in future projects.
The educator team unanimously agreed that the externally taught content of the VEC module could be effectively taught with mixed cohorts from the Core-PWP and MSci cohorts using an online approach.
Information on the changes was disseminated to Program Administrators to enable efficient implementation. External Educators were made aware of the retention of online lecture sessions, and the mixed-cohort approach, by the VEC module convenor.
Timetables were updated by the Program Director, in collaboration with Module Convenors; consideration has been given to the potential Workflow impact of closely aligning multiple cohorts (see below). Timetables have been looked at by the team ‘side-by-side,’ to ensure that Workflow balance is maintained for educators across all cohorts. We can continue to monitor the impact on workload while adjustments are made to teaching (such as with the Working Document mentioned in the Follow-Up section, below).
IAPT Services were made aware of the changes to the timetables

Impact

As of October 2021, the merged timetables are proving effective, with no major barriers, having been detected. Predicted barriers included those to effective teaching of (previously face-to-face) content, student/staff dissatisfaction with a blended approach, and significant administrative/technical difficulties.

Face-to-Face teaching resumed in September 2021 and has been a successful return to the classroom. Educators report being able to switch between live sessions and face-to-face teaching days without significant barriers.

The educator team plan to continue to gather feedback on the student experience of the blended and merged approach. We will be able to assess feedback when the first cohorts fully complete in June 2022.

Feedback will be sought from module convenors, educators, and program administrators using “menti” feedback forms, bi-weekly team meetings and informal qualitative discussion, to gauge the impact of the changes on workflow. Student feedback will also be monitored through end-of-module feedback collated by the School.

Reflection

The challenge of engaging professional practice students and utilising graduates to overcome this. We will consider setting up graduate/patient advisory group for future projects.
Using feedback from a MSci graduate led to timetable changes to ensure readability and clarity for students. This included points such as colour coding F2F v online teaching days, explaining acronyms, etc.
Involving all members of the team (especially Module Convenors) felt like a much more meaningful and collaborative process than Programme Director decisions alone. It gave Module Convenors autonomy over their modules as well as aligning learning outcomes across the 3 modules of the programme which is particularly important for clinical training. Other courses may wish to replicate this approach to build team cohesion and allow all colleagues to make meaningful contributions to programme changes and delivery.

Follow up

Working document has been created for the educator team to comment on the teaching they have just delivered i.e., was there enough time to deliver all content? This has allowed changes to be made within a matter of weeks as the same day is delivered across the programmes. As a result, we can fine-tune the timetable and delivery of the programme quicky and efficiently to improve the student experience.
We will review module by module and at the end of each cohort to continue making any necessary adjustments. Module and programme evaluations, student-staff rep meetings and any feedback from individual teaching days will also help to inform this.

Driving programme development in the IOE: student focus groups and paper writing

15 November 2021 By aaroncooper Comment Closed

Jo Anna Reed Johnson – Institute of Education

j.a.reedjohnson@reading.ac.uk

Overview

This article outlines the thinking to drive programme development through student focus groups across three IOE programmes. The outcome to write a paper and present at a conference helped me to frame this project with a team of academics focusing on changes made during Covid-19 (2020-2021). This article will share reflections on setting up and running of the focus groups, the delivery of the conference presentation and the final paper writing. Finally, it will discuss what we have learnt from this and what we will continue to do.

Objectives

Share 4 academic perspectives on the redesigning of three modules (SKE, STEAM, PGCE Sec Science) that all have practical elements (laboratory or school), due to Covid-19, by sharing what we did and exploring the student perspectives
Show how we designed opportunities for discussion and collaboration when conducting practical work or school related work online
Consider the use of student focus groups for programme co-development
Reflect on the collaborative nature of paper writing and co-programme reflections

Context

At the IOE there are a range of teacher education programmes, with a practical focus. The four colleagues engaged in this article were involved with Skills in Schools (ED2TS1 – March to July 2020), SKE (IESKEP and PFTZSKEMATHSA– March to Aug 2020) and PGCE Secondary Science (GFSTSCIENCE – September 2020 to June 2021). These programmes all require students to work in schools and engage in a science laboratory (if science focused). As COVID hit in March 2020 we had to think quickly and imaginatively, transforming our provision to be online where required. Having worked across all three programmes I felt it was pedagogically appropriate to engage our students in the ways we had throughout their learning during the pandemic, where they worked in online communities of practice to reflect. Thus, we decided to set up a series of focus groups with students reflecting on the impact of the changes and to provide insights for future programme innovations. This culminated in a conference presentation and paper.

Implementation

The focus was to drive programme development through reflections and shared experiences of academics and students. I set up a project timeline and MS Team to manage and drive the deliverables, with the end goal to engage students as co-programme developers and to culminate in a conference presentation and paper. It required framing the project, seeking ethical approval and funding, setting up focus groups to collect data, then reflections and writing up.

Framing the project allowed me to maintain the focus for the redesigning of three modules that all had practical elements (laboratory or school), due to Covid-19. And then exploring how that had impacted on students through focus groups. It was the conference and paper deadlines that drove this activity and timeline. At first colleagues wondered why we were writing a paper for a submission related to the School of Architecture (Manchester and Manchester Metropolitan University), but in fact it was because it was about ‘place’. The remit was a paper related to ‘online education: teaching in a time of change’.

Seeking ethical approval and funding all required knowing where to go and what to do. Ethical approval required submission of an ethical approval form (including consent form, interview schedule, focus group details) to the IOE ethics committee. Then applying for funding through the University Travel Grants Scheme – Tasha Easton – e.saxon@reading.ac.uk

Data Collection was initially carried out using MS Forms, for the initial feedback request. Consent was also required, so where this could not be achieved in person, there was a need to have consent approval attached to the top of the MS Form. Once participants had consented and those who were willing had indicated taking part in the focus groups, I could set up a series of focus groups across the three programmes, to take place on MS Teams. We decided to split the four sets of interviews into subject specific groups so that the conversations and reflections could be driven by the students. One student was nominated as the chair, and they had a template of questions to guide their discussions.

Paper Writing was a challenge as we needed to fit this around our Teaching Focused roles. I created a writing template after attending an IOE Research and Scholarship Writing Workshop with Professor Alan Floyd. I scheduled meetings to review, discuss and allocate sections of writing.

The whole process began in December 2020 and continued through to 30 May 2021, with the conference in 21-23 April 2021 (July 2021- Paper Publication).

Impact

There were several elements of impact:

Working collaboratively with colleagues to reflect on programme development
Engaging students as co-programme developers
Attending a conference (where funding allowed)
Conference paper presentation
Conference paper publication

Reflection

In terms of the setting up of focus groups and driving data collection, we learnt that we needed to be organised, and the timeline/plan really helped to keep that focus. There were times where we were too busy, but we had to create time as we had deliverables to meet. If we had not have had those deliverables of a conference presentation and paper, we may have let this slip and do it ‘next year’.

Writing the paper was a challenge in that we had not done this together before, and some colleagues had not written an academic paper in a very long time, or even an educational one. So, creating that writing template and allocating tasks worked.

Gaining conference funding can always be a challenge. But reaching out and asking was the first thing to do. Then finding out what could be offered at the University/School Level. Next time, we would all like to attend the conference. Being an online conference made it more difficult to engage, and I think next time we would plan to all get funding an attend a face-to-face conference so that we too can benefit from being part of the Community of Practice.

What we will continue to do….

Develop students as programme co-developers through focus groups, engaging them in the paper writing.
Use focus groups to help us (academics) reflect on our own practice and discuss developments across programmes.
Drive programme development through the sharing of practices, building communities of practice with timelines and deliverables.

What else will we do…

Engage students in the paper writing and conference process.
Seek funding to attend a F2F conference with colleagues to allows us time and space to continue to reflect on practice.

Links

Research and Travel Grants Committee: https://www.reading.ac.uk/closed/research/committees/res-researchtravelgrantsubcommittee.aspx

AMPS Conference 21-23 April 2021 – https://architecturemps.com/online-ed-conference/

Merging the Academic Tutor System into Compulsory Core Skills Modules

28 October 2021 By aaroncooper Comment Closed

Lizzy Lander – School of Chemistry Food and Pharmacy

e.r.lander@reading.ac.uk

Overview

This blog will outline the successful integration of core (compulsory) skills modules with the academic tutor system via a curriculum of tutorials designed in this project to be delivered by tutors. This project involved the successful design and scheduling of tutorials for tutors to deliver that supported content (e.g. writing and referencing) in core skills modules to allow better support for student academic skill development and also more closely link tutors into modular taught material.

Objectives

Link new academic tutor system with existing Key Skills modules through newly designed academic tutorials discussing core skills to be delivered by tutors.
Design this curriculum of tutorials to improve engagement and development of skills at relevant points in the academic year.
Design tutorial resources for tutors to ensure consistent support for tutees.
Schedule tutorials so tutors and tutees have a place/time to meet in their timetable to facilitate engagement.

Context

SBS had been delivering core skill “Key Skills” modules in parts 1 & 2 for a number of years (since 2015) that focused on academic skills (e.g. writing, referencing,). The introduction of the academic tutor system (2018) with greater focus on academic skills development was closely aligned with the learning outcomes of these modules therefore it was proposed to link the two together.

Implementation

Firstly, an audit of the core skills taught in the Key Skills modules took place to identify which would be most impactful for student development to be reinforced by being integrated into tutorials with academic tutors. Then assignment timetabling was examined to create a schedule of tutorials for the identified skills which allowed practice and formative feedback before assignments, as well as post-assignment feedback to allow students to identify areas of development.

Next, tutorials were formally timetabled, so students viewed these sessions as part of their “normal” academic schedule rather than optional meetings with their tutors.

Afterwards, resources for tutors were created so they could facilitate these tutorials. This consisted of a one to two page pro-forma to inform tutors about the running of the session. Other resources created included materials for activities such as essays to critique as a group. This would also help improve consistency between tutors delivering these sessions as all tutors would have the same session to deliver.

Finally, this project was presented to staff along with details on how resources and information would be disseminated (initially email). Throughout the year tutorials were run by academic tutors directed by the pro-formas and resources with support if needed. Tutors also marked their tutees’ assignments in Key Skills and gave them feedback in their tutorials. Outside of the scheduled tutorials tutors gave one-to-one support for tutees as needed.

Impact

The implementation and influence of these structured and timetabled tutorials was highly effective in supporting tutees in improving academic skills and improving the consistency of tutors engaging with their tutees. Positive impact was clear from the students (surveying parts 1 + 2 in 2019); 53% felt supported/very supported by their tutor and overall, 65% were satisfied/very satisfied with their sessions with 62% finding the summative feedback from tutors helpful and 72% found it useful/very useful to have tutorials in their timetable. Tutors also fed back they have a much clearer idea of what do at tutorials and how best to support student development, whilst valuing the resources provided in this project.

Overall staff and student experience was positively impacted with staff being led and guided to successfully support student development more effectively and consistently.

Reflection

This activity was successful in the way in which it blended together academic tutors, compulsory modules, as well as assessment and feedback. This generated a platform from which students could learn and practise academic skills for success at university in both a compulsory module and with their tutor through formative and summative feedback. This also helped formalise the role of the tutor for both staff and students giving both groups direction, which ultimately benefitted the students’ academic development. Given that each tutorial had a pro-forma of discussions and activities this helped all tutorials stay consistent so all students got broadly the same development opportunities. Finally, the timetabling of meetings made the tutorials like a normal part of the academic calendar encouraging engagement.

Implementation when supplying information and resources via email to academics was shown to not be the most efficient distribution method and ultimately, some students did not attend tutorials despite reminders of the purpose of these sessions, meaning not all students benefitted from this project.

Follow up

The core outcome of the project in that Key Skills modules would be linked by academic tutorials run by tutors and assessments marked by tutors has continued to be implemented in SBS. However, some alterations have been made for more efficient accessing of materials, by placing resources on a OneDrive that could be accessed at any time. This then evolved into an MS Team to store these resources and also allow tutors to ask questions.

References

Xerte: engaging asynchronous learning tasks with automated feedback

9 June 2021 By aaroncooper Comment Closed

Overview

Jonathan Smith: j.p.smith@reading.ac.uk

International Study and Language Institute (ISLI)

This article describes a response to the need to convert paper-based learning materials, designed for use on our predominantly face-to-face summer Pre-sessional programme, to an online format, so that students could work independently and receive automated feedback on tasks, where that is appropriate. A rationale for our approach is given, followed by a discussion of the challenges we faced, the solutions we found and reflections on what we learned from the process.

Objectives

The objectives of the project were broadly to;

rethink ways in which learning content and tasks could be presented to students in online learning formats
convert paper-based learning materials intended for 80 – 120 hours of learning to online formats
make the new online content available to students through Blackboard and monitor usage
elicit feedback from students and teaching staff on the impacts of the online content on learning.

It must be emphasized that due to the need to develop a fully online course in approximately 8 weeks, we focused mainly on the first 3 of these objectives.

Context

The move from a predominantly face-to-face summer Pre-sessional programme, with 20 hours/week contact time and some blended-learning elements, to fully-online provision in Summer 2020 presented both threats and opportunities to ISLI. We realised very early on that it would not be prudent to attempt 20 hours/week of live online teaching and learning, particularly since most of that teaching would be provided by sessional staff, working from home, with some working from outside the UK, where it would be difficult to provide IT support. In addition, almost all students would be working from outside the UK, and we knew there would be connectivity issues that would impact on the effectiveness of live online sessions. In the end, there were 4 – 5 hours/week of live online classes, which meant that a lot of the core course content had to be covered asynchronously, with students working independently.

We had been working with Xerte, an open-source tool for authoring online learning materials, for about 3 years, creating independent study materials for consolidation and extension of learning based round print materials. This was an opportunity to put engaging, interactive online learning materials at the heart of the programme. Here are some of the reasons why we chose Xerte;

It allows for inputs (text, audio, video, images), interactive tasks and feedback to be co-located on the same webpage
There is a very wide range of interactive task types, including drag-and-drop ordering, categorising and matching tasks, and “hotspot” tasks in which clicking on part of a text or image produces customisable responses.
It offers considerable flexibility in planning navigation through learning materials, and in the ways feedback can be presented to learners.
Learning materials could be created by academic staff without the need for much training or support.

Xerte was only one of the tools for asynchronous learning that we used on the programme. We also used stand-alone videos, Discussion Board tasks in Blackboard, asynchronous speaking skills tasks in Flipgrid, and written tasks submitted for formative or summative feedback through Turnitin. We also included a relatively small number of tasks presented as Word docs or PDFs, with a self-check answer key.

Implementation

We knew that we only had time to convert the paper-based learning materials into an online format, rather than start with a blank canvas, but it very quickly became clear that the highly interactive classroom methodology underlying the paper-based materials would be difficult to translate into a fully-online format with greater emphasis on asynchronous learning and automated feedback. As much as possible we took a flipped learning approach to maximise efficient use of time in live lessons, but it meant that a lot of content that would normally have been covered in a live lesson had to be repackaged for asynchronous learning.

In April 2020, when we started to plan the fully-online programme, we had a limited number of staff who were able to author in Xerte. Fortunately, we had developed a self-access training resource which meant that new authors were able to learn how to author with minimal support from ISLI’s TEL staff. A number of sessional staff with experience in online teaching or materials development were redeployed from teaching on the summer programme to materials development. We provided them with a lot of support in the early stages of materials development; providing models and templates, storyboarding, reviewing drafts together. We also produced a style guide so that we had consistent formatting conventions and presentation standards.

The Xerte lessons were accessed via links in Blackboard, and in the end-of-course evaluations we asked students and teaching staff a number of open and closed questions about their response to Xerte.

Impact

We were not in a position to assess the impact of the Xerte lessons on learning outcomes, as we were unable to differentiate between this and the impacts of other aspects of the programme (e.g. live lessons, teacher feedback on written work). Students are assessed on the basis of a combination of coursework and formal examinations (discussed by Fiona Orel in other posts to the T&L Exchange), and overall grades at different levels of performance were broadly in line with those in previous years, when the online component of the programme was minimal.

In the end-of-course evaluation, students were asked “How useful did you find the Xerte lessons in helping you improve your skills and knowledge?” 245 students responded to this question: 137 (56%) answered “Very useful”, 105 (43%) “Quite useful” and 3 (1%) “Not useful”. The open questions provided a lot of useful information that we are taking into account in revising the programme for 2021. There were technical issues round playing video for some students, and bugs in some of the tasks; most of these issues were resolved after they were flagged up by students during the course. In other comments, students said that feedback needed to be improved for some tasks, that some of the Xerte lessons were too long, and that we needed to develop a way in which students could quickly return to specific Xerte lessons for review later in the course.

Reflections

We learned a lot, very quickly, about instructional design for online learning.

Instructions for asynchronous online tasks need to be very explicit and unambiguous, because at the time students are using Xerte lessons they are not in a position to check their understanding either with peers or with tutors. We produced a video and a Xerte lesson aimed at helping students understand how to work with Xerte lessons to exploit their maximum potential for learning.

The same applies to feedback. In addition, to have value, automated feedback generally (but not always) needs to be detailed, with explanations why specific answers are correct or wrong. We found, occasionally, that short videos embedded in the feedback were more effective than written feedback.

Theoretically, Xerte will track individual student use and performance, if uploaded as SCORM packages into Blackboard, with grades feeding into Grade Centre. In practice, this only works well for a limited range of task types. The most effective way to track engagement was to follow up on Xerte lessons with short Blackboard tests. This is not an ideal solution, and we are looking at other tracking options (e.g. xAPI).

Over the 4 years we have been working with Xerte, we had occasionally heard suggestions that Xerte was too complex for academics to learn to use. This emphatically was not our experience over Summer 2020. A number of new authors were able to develop pedagogically-sound Xerte lessons, using a range of task types, to high presentation standards, with almost no 1-to-1 support from the ISLI TEL team. We estimate that, on average, new authors need to spend 5 hours learning how to use Xerte before they are able to develop materials at an efficient speed, with minimal support.

Another suggestion was that developing engaging interactive learning materials in Xerte is so time-consuming that it is not cost-effective. It is time-consuming, but put in a situation in which we felt we had no alternative, we managed to achieve all we set out to achieve. Covid and the need to develop a fully-online course under pressure of time really focused our minds. The Xerte lessons will need reviewing, and some will definitely need revision, but we face summer 2021 in a far more resilient, sustainable position than at this time last year. We learned that it makes sense to plan for a minimum 5-year shelf life for online learning materials, with regular review and updating.

Finally, converting the paper-based materials for online learning forced us to critically assess them in forensic detail, particularly in the ways students would work with those materials. In the end we did create some new content, particularly in response to changes in the ways that students work online or use technological tools on degree programmes.

Follow up

We are now revising the Xerte lessons, on the basis of what we learned from authoring in Xerte, and the feedback we received from colleagues and students. In particular, we are working on;

ways to better track student usage and performance
ways to better integrate learning in Xerte lessons with tasks in live lessons
improvements to feedback.

For further information, or if you would like to try out Xerte as an author and with students, please contact j.p.smith@reading.ac.uk, and we can set up a trial account for you on the ISLI installation. If you are already authoring with Xerte, you can also join the UoR Xerte community by asking to be added to the Xerte Users Team.

Links and References

The ISLI authoring website provides advice on instructional design with Xerte, user guides on a range of page types, and showcases a range of Xerte lessons.

The international Xerte community website provides Xerte downloads, news on updates and other developments, and a forum for discussion and advice.

Finally, authored in Xerte, this website provides the most comprehensive showcase of all the different page type available in Xerte, showing its potential functionality across a broad range of disciplines.