Co-Creating a Programme Focussed Assessment Strategy in BSc Food Science

Julia Rodriguez Garcia: Department of Food and Nutritional Science

j.rodriguezgarcia@reading.ac.uk

Link back to case studies on the T and L Exchange website

Overview

This student-staff partnership involved students from all year groups, academics and the programme administrator of the BSc Food Science. In this PlanT project the Transforming the Experience of Students through Assessment (TESTA) methodology was used to develop programme focus assessment strategy that could lead to a reduction in volume and improved distribution of assessment, and overall enhance student and staff experience

Objectives

The main aim of this project was to take an evidence-based approach to enabling a cultural shift required from ‘my module’ to ‘our programme’ to engage staff in an integral restructuration of the assessment design, volume, and distribution in the programme.

The main objectives of the project were:

  • To assess if there is assessment is evenly distributed in terms of volume and weighting and type across the programme
  • To explore and propose changes in the design of assessment tasks to move to a programme level assessment strategy that could improve student and staff experience

Context

Restructuration of programmes have been usually performed at modular level, resulting in limited coordination of the learning (including assessment) strategy at programme level. In the BSc Food Science this led to student complains about high volume of assessment tasks due in a short period of time. External examiners also commented on the high volume of coursework components. And staff was overload with marking and feedback tourn around deadlines.

Having small assessment tasks set within a module, results in only a small number of concepts assessed as part of each task which leads to losing the holistic perspective of the subject area creating the effect of fragmenting knowledge and promoting surface learning, such as memorisation of content. Thus, a change from assessment of learning to assessment for/as learning will facilitate a change in behaviours improving students’ motivation and rewarding staff efforts

Implementation

Figure 1. Structure of the Transforming the Experience of Students through Assessment’ (TESTA method)

We developed Programme Level Assessment Maps that allowed us to calculate the number of assignments per credits, the weighting distribution per week and per term. These maps helped to create a visual interpretation of the students’ experience to increase staff awareness of the real situation at programme level (not just in their modules).

Assessment Map of Part 1 of an BSc in Food Science. Assessment activities are labelled in colours: purple for formative activities, pink for summative activities from compulsory modules, yellow for summative activities from optional modules, red for exams. The type of assessment activities and the weighting are also presented in the map.

In collaboration with students, we modify the Assessment Experience Questionnaire to give students more space to reflect in their experience when performing certain type of assessments. This questionnaire was run two consecutive years.

One of the students leading the PlanT project carried out a focus groups with students from all year groups to reflect and discuss on the way they learn, the skills the develop and the challenges they faced completing assessment tasks.

We combined all the data, discussed it in light of published literature and draw some final suggestions to modify our assessment design practices. The results and future strategies were presented in a staff meeting lead by a student.

Impact

This project has facilitated and achieved a change in staff mindset from module level to programme level. This has been reflected in a 9% reduction on assessment activities from 2018-2019 to 2019-2020.

Figure 3: Principles and practices that will inform and define the implementation of a programme focused assessments trategy

This shift in focus will allow us to perform more holistic changes following a common set of principles that underpin the development of authentic assessment to promote higher order thinking skills, integration of knowledge (horizontally and vertically), and students’ motivation and independence.

The conclusions from this project and the proposed strategies to develop a programme level assessment approach have been implemented in the department through the Portfolio Review

Reflection

The development of a cohesive staff community of Programme Directors, Module Convenors, and support staff from the teaching hub with strong communication links is crucial for the design and delivery of a high-quality programme.

Moreover, student voice and partnership are crucial for the co-development of teaching and assessment approaches, working collaborations are transformational both for the community wellbeing and for achieving highly successful outcomes. Development of a sense of community within the department is something distinctive in Food Science.

Follow up

Brown, Sally, and Kay Sambell. 2020a. “Changing assessment for good: a major opportunity for educational developers.” Assessment, Learning and Teaching in Higher Education; Sally Brown. https://sally-brown.net/kay-sambell-and-sally-brown-covid-19-assessment-collection/.

Murphy, Vanessa, James Fox, Sinéad Freeman, and Nicola Hughes. 2017. ““Keeping It Real”: A Review of the Benefits, Challenges and Steps Towards Implementing Authentic Assessment.”. The All Ireland Journal of Teaching and Learning in Higher Education (AISHE-J) 9 (3):2801.

O’Neil, Geraldine Mary. 2019. “Why don’t we want to reduce assessment?”  All Ireland Journal of Higher Education 11 (2):1-7.

TESTA. Transforming the Experience of Students Through Assessment. https://www.testa.ac.uk/

Villarroel, V.; Bloxham, S.; Bruna, D.; Bruna, C.; Herrera-Seda, C. Authentic assessment: creating a blueprint for course design. Assessment & Evaluation in Higher Education 2018, 43, 840-854, doi:10.1080/02602938.2017.1412396.

Merging the Academic Tutor System into Compulsory Core Skills Modules

Lizzy Lander – School of Chemistry Food and Pharmacy

e.r.lander@reading.ac.uk

Link back to case studies on the T and L Exchange website

Overview

This blog will outline the successful integration of core (compulsory) skills modules with the academic tutor system via a curriculum of tutorials designed in this project to be delivered by tutors. This project involved the successful design and scheduling of tutorials for tutors to deliver that supported content (e.g. writing and referencing) in core skills modules to allow better support for student academic skill development and also more closely link tutors into modular taught material.

Objectives

  • Link new academic tutor system with existing Key Skills modules through newly designed academic tutorials discussing core skills to be delivered by tutors.
  • Design this curriculum of tutorials to improve engagement and development of skills at relevant points in the academic year.
  • Design tutorial resources for tutors to ensure consistent support for tutees.
  • Schedule tutorials so tutors and tutees have a place/time to meet in their timetable to facilitate engagement.

Context

SBS had been delivering core skill “Key Skills” modules in parts 1 & 2 for a number of years (since 2015) that focused on academic skills (e.g. writing, referencing,). The introduction of the academic tutor system (2018) with greater focus on academic skills development was closely aligned with the learning outcomes of these modules therefore it was proposed to link the two together.

Implementation

Firstly, an audit of the core skills taught in the Key Skills modules took place to identify which would be most impactful for student development to be reinforced by being integrated into tutorials with academic tutors. Then assignment timetabling was examined to create a schedule of tutorials for the identified skills which allowed practice and formative feedback before assignments, as well as post-assignment feedback to allow students to identify areas of development.

Next, tutorials were formally timetabled, so students viewed these sessions as part of their “normal” academic schedule rather than optional meetings with their tutors.

Afterwards, resources for tutors were created so they could facilitate these tutorials. This consisted of a one to two page pro-forma to inform tutors about the running of the session. Other resources created included materials for activities such as essays to critique as a group. This would also help improve consistency between tutors delivering these sessions as all tutors would have the same session to deliver.

Finally, this project was presented to staff along with details on how resources and information would be disseminated (initially email). Throughout the year tutorials were run by academic tutors directed by the pro-formas and resources with support if needed. Tutors also marked their tutees’ assignments in Key Skills and gave them feedback in their tutorials. Outside of the scheduled tutorials tutors gave one-to-one support for tutees as needed.

Impact

The implementation and influence of these structured and timetabled tutorials was highly effective in supporting tutees in improving academic skills and improving the consistency of tutors engaging with their tutees. Positive impact was clear from the students (surveying parts 1 + 2 in 2019); 53% felt supported/very supported by their tutor and overall, 65% were satisfied/very satisfied with their sessions with 62% finding the summative feedback from tutors helpful and 72% found it useful/very useful to have tutorials in their timetable. Tutors also fed back they have a much clearer idea of what do at tutorials and how best to support student development, whilst valuing the resources provided in this project.

Overall staff and student experience was positively impacted with staff being led and guided to successfully support student development more effectively and consistently.

Reflection

This activity was successful in the way in which it blended together academic tutors,  compulsory modules, as well as assessment and feedback. This generated a platform from which students could learn and practise academic skills for success at university in both a compulsory module and with their tutor through formative and summative feedback. This also helped formalise the role of the tutor for both staff and students giving both groups direction, which ultimately benefitted the students’ academic development. Given that each tutorial had a pro-forma of discussions and activities this helped all tutorials stay consistent so all students got broadly the same development opportunities. Finally, the timetabling of meetings made the tutorials like a normal part of the academic calendar encouraging engagement.

Implementation when supplying information and resources via email to academics was shown to not be the most efficient distribution method and ultimately, some students did not attend tutorials despite reminders of the purpose of these sessions, meaning not all students benefitted from this project.

Follow up

The core outcome of the project in that Key Skills modules would be linked by academic tutorials run by tutors and assessments marked by tutors has continued to be implemented in SBS. However, some alterations have been made for more efficient accessing of materials, by placing resources on a OneDrive that could be accessed at any time. This then evolved into an MS Team to store these resources and also allow tutors to ask questions.

References

A Partnership in SLL to Enhance Blended Learning Practices: an Analysis of the Process and Findings

Michael Lyons- School of Literature and Languages

m.lyons@reading.ac.uk 

Link back to case studies on the T and L Exchange website

Overview

PEBLSS (a Partnership to Enhance Blended Learning – between Staff and Students) was a project seeking to strengthen teaching and learning in SLL. It particularly focuses on enhancing methods of teaching and learning amidst the extraordinary conditions faced by staff and students which has resulted in the transition into blended and online learning.

Context

A survey was designed and sent to students within the School of Literature and Languages, to identify members of staff who have excelled at blended learning during the Covid-19 pandemic 2020-2021. This was done by identifying three key areas of learning (pre-recorded lectures, live sessions, and assessment) and asking students to name the members of staff whom they believe stood out based on each of these core areas and their reasons behind this. Interviews were conducted to complete case studies based on these responses. Overall, 76 students responded to the survey.

We aimed to reduce “survey fatigue” due to the substantial number of surveys that students have received throughout the course of the Covid-19 pandemic. Therefore, by keeping the questions to short answers and multiple-choice, students were more likely to respond and contribute their ideas within the survey.

Once the surveys were collected, the results were analysed to identify the members of staff that proved to have stood out among the students in each of the core areas. From this, interviews were arranged and conducted to allow for the chosen members of staff to expand on their experience of teaching during the blended learning approach.

Objectives

The questions were designed to obtain specific examples of good teaching practices, and for the members of staff to

  1. elaborate on their successful methods.
  2. discover what teaching practices were successfully put into place and the ways they could be adapted into future teaching.
  3. retrieving information from nominated staff regarding how they continued to carry out assessments and ensure students felt supported in their assessments in the transition to the blended learning environment.

Implementation

Thirty minutes was designated per interview; however, many of the interviews extended beyond the time frame due the detail of the answers. Interviews were recorded and saved onto Microsoft Stream, for the researchers to conduct further analysis.

Impact

Summary of findings for the pre-recorded lectures

  • Pre-recorded lectures offered the opportunity to bring in new, innovative audio-visual materials to support and enhance sessions. It was noted that different modes of delivery are important.
  • Pre-recorded lectures were received well when language and style of speech was concise but exaggerated. More emphasis is needed on the speech style rather than gestures (and other paralinguistic features), as these are lost online. Additionally, potentially politically sensitive topics need extra consideration for online sessions.
  • Guides about timeframes (like a week-by-week schedule) are useful for the organisation of pre-recorded lectures. Students know what they are learning when; examples of these practices were demonstrated in the interviews by a word document calendar, or a weekly bitesize email.
  • It was advised that longer segments for the pre-recorded lectures should be avoided. 20-minute segments were recommended, as this makes the sessions concise but the content for the segment is still detailed,thus maintaining student engagement online.
  • It is also advisable to have weekly folders, clearly labelled, with the pre-recorded lectures (with an embedded link), PowerPoint presentation, and any other specific resources or worksheets relevant to the screencast.
  • It was noted that it was advantageous to have short videos of around 2-3 minutes in length to go over the concepts of the lecture; this makes it easier for students to revise. It also ensures that staff have covered important materials more than once e.g., prioritisation of information.
  • Useful to have several check-ins with students near the beginning and middle of the module so that feedback about the pre-recorded lectures can be acted on whilst students are learning.
  • Discussion boards were useful for the screencast exercises and promoted more student engagement. It was another useful tool for students to feedback about the screencasts, or if they had any questions.
  • Timestamping information was beneficial. Outlining to students when certain parts or concepts are going to show in the pre-recorded lecture was advantageous because students can use the timestamps to revise specific information.
    • It was noted that pre-recorded lectures were challenging because of the constraints of timing, editing and personalisation of content. Additionally, use of the pre-recorded lectures for the future was a topic under discussion in the interviews. Staff recognised the benefits of having the pre-existing screencasts; however, ensuring the information was up-to-date and engaging for current students was a concern. It was also noted that if the pre-recorded lectures were personalised this year for current students, the session would need to be updated or reproduced for the next year group.

 

Summary of findings for the live sessions

  • The advantages of live sessions:
  • Attendance is strong – reflected in the marking
  • Online sessions easier than socially distanced face-to-face sessions for discussions and facilitating dialogue
  • Sharing of files – OneNote/Blackboard/Collaborate
  • The number of methods identified to help increase student participation

The disadvantages of live sessions:

  • Technology – Learning new technology – reliance of technology, difficult if not working

Reflection

This project has been widely beneficial, not only in identifying current good teaching practices, but also finding methods to help for future pre-recorded lectures, live sessions, and assessments. The project was also beneficial for us as students. In identifying good practices during the pandemic, we were able to feed this information back to staff (and students) at the Teach Share event on the 29th of June 2021.

Overall, it is evident from this project that there are many teaching practices used by staff both before and during the Covid-19 pandemic that are beneficial to other members of staff (as well as the students) and therefore this project highlighted how important it is to share teaching practices, not only within individual departments, but school and university-wide.

Using OneDrive to Address the Challenges of Online Groupwork

Natalie Drake, Gemma Peacock, Rachel Rushton – ISLI

n.drake@reading.ac.uk; g.peacock@reading.ac.uk;  r.rushton@reading.ac.uk

Link back to case studies on the T and L Exchange website

Overview

Moving two ISLI academic skills modules (IF0ACA and IF0RAS) to online delivery presented certain challenges for group interaction. In online learning environments building interpersonal relationships, and dealing with both connectivity issues and different time-zones can prove problematic for international students and their tutors. To address this, group OneDrive folders were set up as a repository for collaborative documents, such as research findings or presentation slides. Students interacted with these during online workshops and asynchronously, while tutors monitored group engagement and provided instantaneous feedback to students. It resulted in a positive student response and recognition of its transferability as a study skill and to face-to-face learning.

Objectives

The aims of this activity were:

  1. To facilitate online group work by providing collaborative spaces through OneDrive
  2. To develop transferable IT literacy skills
  3. To improve communication and interpersonal skills
  4. To develop independent study skills and transferable IT literacy skills
  5. To improve student engagement
  6. To improve the time efficiency of online activities

Context

Both skills modules are compulsory 20-credit modules for home Foundation students and focus on developing group work skills. In term two, in both modules, students are required to complete summative assessments that rely heavily on group work, which forced tutors to reconsider how to facilitate this online.

Implementation

The theory underpinning the use of OneDrive to facilitate groupwork is the Community of Inquiry model (Garrison, Anderson and Archer, 2001) where knowledge is formed and embedded within a social context. Since classes moved online it has become more important for us as educators to find ways to encourage interpersonal relationships online, not only between students and peers but also between students and their teachers. Using OneDrive addresses some of these social issues and helps students to build relationships while developing knowledge and skills online. Lipman applied the Community of Inquiry model to education and stated that ‘education is the outcome of participation in a teacher-guided community of inquiry’ (2003, pp.18–19)​. Using OneDrive allows teachers to scaffold tasks and to clearly direct students, offering an alternative to knowledge transmission whereby students can arrive at more higher ordered thinking critical for Higher Education. Using OneDrive synchronously or asynchronously in online learning satisfies the three essential elements of the Community of Inquiry model: social presence (how participants identify with their community), cognitive presence (how much learners can construct meaning through reflection together) and teaching presence (how we design and facilitate processes for realising learning outcomes).

Setting up OneDrive for activities requires several steps. Firstly, module folders are set at the top level of the OneDrive folders, then within those class folders are created.

 

The next level contains specific assignment folders and within those folders are group folders which contain any necessary files or folders from teachers, course directors or students themselves.

 

Once students are added to groups within their class for a particular assignment, a link is generated for the relevant folder. Students are introduced to their group OneDrive repository and link via an email and encouraged to save the link as a bookmark for ease of access.

 

There are three main ways in which we implemented OneDrive in our courses. Firstly, we used it in asynchronous group tasks through Blackboard. Tutors could check student engagement and prepare feedback before the live session after the task deadline. This worked well when students needed to prepare information or brainstorm ideas before a synchronous task in the live session.

 

The second way was through synchronous group tasks in the live sessions themselves. Students accessed their OneDrive group document during the live session and worked collaboratively on documents. The tutor could monitor the students’ work and comment live.

 

The final way was through asynchronous group tasks and weekly group meetings for assignments such as a group presentation, where students continued to collaborate on group OneDrive documents such as a PowerPoint file and the tutor monitored and commented on the documents as appropriate.

Impact

The use of OneDrive documents and folders achieved our main objective of facilitating groupwork while satisfying the essential elements of the Community of Inquiry model.  Students were able to construct social presence and cognitive presence by working collaboratively on, for example, a group ground rules document (see snip 7) and a research repository document.

 

Being able to synchronously edit the ground rules document meant that students could hold a faster discussion on the relative merits of rules, along with agreement on the day/time of a weekly group meeting. Tasks where students added to a collaborative document, such as in the research of key terms, enhanced the sense of group identity and also served to highlight levels of participation. This often negated the need for tutor intervention in the early stages. The third essential element of teaching presence was realised in synchronous sessions, through immediate tutor input on documents which gave advice to help the group work towards the learning outcomes in a friendly, non-judgmental way.

 

This, and asynchronous tutor input, also provided a permanent record of tutor feedback which the group could revisit. This is perhaps more effective than the usual tutor comments given orally, useful in the moment but unrecorded.  One outcome which we had not anticipated was that for many students this was the first time they had worked simultaneously on one document. This led to students viewing groupwork in a more positive light as an opportunity to develop communication and IT skills which was reported in end-of-course student feedback. Furthermore, the pass rate for the assignment was high and most students achieved the learning outcomes, which indicates impact and effectiveness.

Reflection

Using OneDrive is not without challenges, notably the initial set-up procedure, implementing live sessions with large classes, and filling the digital gap for tutors and students. Also, the current practice is for a member of staff to host the OneDrive folders. We believe, however, that the benefits far outweigh these primary hurdles.

These benefits included improved IT skills and increased familiarity with UoR tools which is transferable to other modules, as well as building confidence for Foundation students by providing a scaffolded approach to groupwork. Using OneDrive made collaboration online easier and tutor comments were still available after the live sessions so students could reflect on these. Students reported feeling supported with groupwork and commented on how using OneDrive developed group communication skills as well as editing and computer skills.

Benefits for tutors in the asynchronous learning environment include that it was easier to monitor student engagement in tasks as it was clear to see which group members had participated.

 

Likewise, in the synchronous learning space, tutors were able to monitor student participation in live sessions and it allowed tutors to be ‘present’ in collaborative documents (see snips 9 – 10 above). We believe this emulated the F2F classroom in terms of when teachers monitor students and move from desk to desk checking students’ work. One downside of teacher presence in online student documents could be potential inhibition of less confident students, however we saw all student names participating so believe this may just be a necessary learning curve students need to make.

Follow up

Since trialling the use of OneDrive, these ideas have been disseminated to colleagues at the ISLI-based Learning & Teaching Research Forum via a presentation and Q&A session. In this academic year (2021-22) IFP plan to introduce OneDrive earlier and be more explicit on its transferability to other modules and assessments, thus encouraging students to become more autonomous. OneDrive may also be used in Pre-sessional year-round English courses this academic year to support online students’ participation in Academic Reading Circle activities and to share their Independent Reading Records with both peers and tutors.

 

References

Garrison, D. R., Anderson, T., Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education.  American Journal of Distance Education15(1).

Lipman, M. (2003). Thinking in Education. (2nd ed.). Cambridge: Cambridge University Press. pp.18-19

Learning to Interpret and Assess Complex and Incomplete Environmental Data

Andrew Wade a.j.wade@reading.ac.uk

Department of Geography and Environmental Sciences

Overview

Field work is well known to improve student confidence and enhance skills and knowledge, yet there is evidence for a decline in field work in Secondary Education, especially amongst A-level Geography students. This is problematic as students are entering Geography and Environmental Science degree programmes with reduced skills and confidence around field-based data collection and interpretation, and this appears to be leading to an apprehension around data collection for dissertations. A simple field-based practical where 47 Part 2 Geography and Environmental Science students tested their own hypotheses about factors that control water infiltration into soils was developed. Improved confidence and appreciation of critical thinking around environmental data was reported in a survey of the student experience. Student coursework demonstrated that attainment was very good, and that skills and critical thinking can be recovered and enhanced with relatively simple, low-cost field-based practical classes that can be readily embedded to scaffold subsequent modules, including the dissertation.

Context

The importance of field work is well established in Geography and Environmental Science as a means of active and peer-to-peer learning. However, students appear to have little confidence in designing their own field work for hypotheses testing when they arrive for Part 1, probably due to a decline in field work in Secondary Education (Kinder 2016, Lambert and Reiss 2014). Within the Geography and Environmental Science programmes, there is a part two, 20 credit ‘Research Training’ module that develops the same skills. However, this research training module and the dissertation are seen by the students as being of high risk in that they perceive a low mark will have a significant negative impact on the overall degree classification. Consequently, students are seemingly risk adverse around field-based projects. The idea here is to make field-based training more commonplace throughout multiple modules through inclusion of relatively simple practical training, so that hypotheses testing, critical thinking and confidence with ‘messy’ environmental data become intuitive and students are at ease with these concepts. In parallel, GES module cohorts have increased in recent years and this is an additional reason to develop simple, low-cost practical classes.

Objectives

The aim of the project was to determine if a simple, field-based practical would help boost student confidence around field data collection and interpretation, and hypotheses testing. The objective was to give the students a safe and supportive environment in which to develop their own hypotheses and method for field data collection, and to learn to interpret often ‘messy’ and ‘complex’ environmental data.

Figure 1: The practical class took place on the hill-slope on campus between the Atmospheric Observatory and Whiteknights Lake on the 28 October 2019 over 4 hours in total.

 

Figure 2: Students used a Decagon Devices Mini-Disc Infiltrometer to measure unsaturated hydraulic conductivity to test their own hypotheses about the factors controlling infiltration

Implementation

A practical was designed where 47 Part 2 students, working in groups of four or five, developed their own hypotheses around the factors controlling rainfall infiltration on a hill-slope in the class room following an in-class briefing, and then tested these hypotheses in the field using Mini Disc infiltrometers (Figs. 1, 2 and 3). There was a further follow-up session where each student spent two hours processing the data collected and was briefed on the coursework write-up.

Figure 3: The students tested hypotheses around distance from the lake, vegetation and soil type, soil moisture and soil compaction. Each student group spent two hours in the field.

Impact

Of 40 students who responded to an on-line survey:

  • 37 agreed the practical helped develop their critical thinking skills around complex and incomplete environmental data;
  • 36 agreed they were now better able to deal with uncertainty in field-based measurements;
    and 38 feel more confident working in the field.

Student quotes included:

  • “The practical was very useful in helping to understand the processes happening as well as being more confident in using the equipment.”
  • “I thought the practical was good as it was another way to process information which tends to work better for me, doing and seeing how it works allows me to gain a higher understanding in the processes”

The majority of students gained first class and upper second-class marks for the project write-up and the reports submitted demonstrated good critical thinking skills in the interpretation of the infiltration measurements. There has been a noticeable increase in the number of students opting for hydrology-based dissertations.

Reflections

Confidence and critical thinking skills can be enhanced with relatively simple, low-cost field-based practicals that scaffold subsequent modules including Research Training for Geographers and Environmental Science, and the dissertation, and focus on hypotheses testing in addition to knowledge acquisition. Each student spent 2 hours in the field on campus and 2 hours processing their data, with further time on the coursework write-up. This seems a reasonable investment in time given the benefits in confidence, skills and knowledge. Embedding such practicals should not replace the larger skills-based modules, such as Research Training, nor should such practical classes replace entirely those that focus more on knowledge acquisition, but these practical classes, where students explore their own ideas, appear to be a useful means to boost student confidence and critical thinking skills at an early stage. The practical was also an excellent means of encouraging peer to peer interaction and learning, and this and similar practical classes have good potential for the integration of home and NUIST students.

Follow up

Embed similar practical classes in part one modules to build confidence at the outset of the degree programme and, at part three, to further enable integration of home and NUIST students.

Links and References

Kinder A. 2016. Geography: The future of fieldwork in schools. Online: http://www.sec-ed.co.uk/best-practice/geography-the-future-of-fieldwork-in-schools/ (Last accessed: 03 Jan 2020).

Lambert D and Reiss MJ. 2014, The place of fieldwork in geography and science qualifications, Institute of Education, University of London. ISBN: 978-1-78277-095-4. pp. 20

The impact of COVID upon practical classes in Part 1 chemistry – an opportunity to redevelop a core module

Philippa Cranwell p.b.cranwell@reading.ac.uk, Jenny Eyley, Jessica Gusthart, Kevin Lovelock and Michael Piperakis

Overview

This article outlines a re-design that was undertaken for the Part 1 autumn/spring chemistry module, CH1PRA, which services approximately 45 students per year. All students complete practical work over 20 weeks of the year. There are four blocks of five weeks of practical work in rotation (introductory, inorganic, organic and physical) and students spend one afternoon (4 hours) in the laboratory per week. The re-design was partly due to COVID, as we were forced to critically look at the experiments the students completed to ensure that the practical skills students developed during the COVID pandemic were relevant for Part 2 and beyond, and to ensure that the assessments students completed could also be stand-alone exercises if COVID prevented the completion of practical work. COVID actually provided us with an opportunity to re-invigorate the course and critically appraise whether the skills that students were developing, and how they were assessed, were still relevant for employers and later study.

Objectives

• Redesign CH1PRA so it was COVID-safe and fulfilled strict accreditation criteria.
• Redesign the experiments so as many students as possible could complete practical work by converting some experiments so they were suitable for completion on the open bench to maximise laboratory capacity
• Redesign assessments so if students missed sessions due to COVID they could still collect credit
• Minimise assessment load on academic staff and students
• Move to a more skills-based assessment paradigm, away from the traditional laboratory report.

Context

As mentioned earlier, the COVID pandemic led to significant difficulties in the provision of a practical class due to restrictions on the number of students allowed within the laboratory; 12 students in the fumehoods and 12 students on the open bench (rather than up to 74 students all using fumehoods previously). Prior to the redesign, each student completed four or five assessments per 5-week block and all of the assessments related to a laboratory-based experiment. In addition, the majority of the assessments required students to complete a pro-forma or a technical report. We noticed that the pro-formas did not encourage students to engage with the experiments as we intended, therefore execution of the experiment was passive. The technical reports placed a significant marking burden upon the academic staff and each rotation had different requirements for the content of the report, leading to confusion and frustration among the students. The reliance of the assessments upon completion of a practical experiment was also deemed high-risk with the advent of COVID, therefore we had to re-think our assessment and practical experiment regime.

Implementation

In most cases, the COVID-safe bench experiments were adapted from existing procedures, allowing processing of 24 students per week (12 on the bench and 12 in the fumehood), with students completing two practical sessions every five weeks. This meant that technical staff did not have to familiarise themselves with new experimental procedures while implementing COVID guidelines. In addition, three online exercises per rotation were developed, requiring the same amount of time as the practical class to complete therefore fulfilling our accreditation requirements. The majority of assessments were linked to the ‘online practicals’, with opportunities for feedback during online drop-in sessions. This meant that if a student had to self-isolate they could still complete the assessments within the deadline, reducing the likelihood of ECF submissions and ensuring all Learning Outcomes would still be met. To reduce assessment burden on staff and students, each 5-week block had three assessment points and where possible one of these assessments was marked automatically, e.g. using a Blackboard quiz. The assessments themselves were designed to be more skills-based, developing the softer skills students would require upon employment or during a placement. To encourage active learning, the use of reflection was embedded into the assessment regime; it was hoped that by critically appraising performance in the laboratory students would remember the skills and techniques that they had learnt better rather than the “see, do, forget” mentality that is prevalent within practical classes.

Examples of assessments include: undertaking data analysis, focussing on clear presentation of data; critical self-reflection of the skills developed during a practical class i.e. “what went well”, “what didn’t go so well”, “what would I do differently?”; critically engaging with a published scientific procedure; and giving a three-minute presentation about a practical scientific technique commonly-encountered in the laboratory.

Impact

Mid-module evaluation was completed using an online form, providing some useful feedback that will be used to improve the student experience next term. The majority of students agreed, or strongly agreed, that staff were friendly and approachable, face-to-face practicals were useful and enjoyable, the course was well-run and the supporting materials were useful. This was heartening to read, as it meant that the adjustments that we had to make to the delivery of laboratory based practicals did not have a negative impact upon the students’ experience and that the re-design was, for the most part, working well. Staff enjoyed marking the varied assessments and the workload was significantly reduced by using Blackboard functionality.

Reflections

To claim that all is perfect with this redesign would be disingenuous, and there was a slight disconnect between what we expected students to achieve from the online practicals and what students were achieving. A number of the students polled disliked the online practical work, with the main reason being that the assessment requirements were unclear. We have addressed by providing additional videos explicitly outlining expectations for the assessments, and ensuring that all students are aware of the drop-in sessions. In addition, we amended the assessments so they are aligned more closely with the face-to-face practical sessions giving students opportunity for informal feedback during the practical class.

In summary, we are happy that the assessments are now more varied and provide students with the skills they will need throughout their degree and upon graduation. In addition, the assessment burden on staff and students has been reduced. Looking forward, we will now consider the experiments themselves and in 2021/22 we will extend the number of hours of practical work that Part 1 students complete and further embed our skill-based approach into the programme.

Follow up

 

Links and References

Improving student assessment literacy & engaging students with rubrics

Dr. Allan Laville

School of Psychology & Clinical Languages Sciences

In this 14 minute video, early rubrics adopter Dr. Allan Laville shares how he and colleagues in Psychology have sought to improve student assessment literacy, and have successfully engaged students with their assessment rubrics by embedding analysis of them into their in-class teaching and by using screencasts, discussion boards and student partnership. Lots of useful ideas and advice – well worth a watch.

Promoting and Tracking Student Engagement on an Online Undergraduate Pre-sessional Course

Sarah Mattin: International Study and Language Institute

Overview

This case study outlines approaches to fostering an active learning environment on the University’s first fully online Undergraduate Pre-sessional Course which ran in Summer 2020 with 170 students. It reports staff and student feedback and reflects on how lessons learnt during the summer can inform ISLI’s continued online delivery this autumn term and beyond.

 

Objectives

  • To design and deliver an online Pre-sessional Course to meet the needs of 170 students studying remotely, mostly in China
  • To promote student engagement in learning activities in an online environment
  • To devise effective mechanisms for tracking student engagement and thus identify students who may require additional support

 

Context

The Pre-sessional Programme (PSE) is an English for Academic Purposes (EAP) and academic skills development programme for degree offer holders who require more study to meet the English Language requirements of their intended programme. The programme runs year-round and, in the summer, has separate UG and PG courses. We would usually expect to welcome around 700 students to the campus for the summer courses (June-September); in summer 2020 we took the courses fully online in response to the COVID crisis. This case study focuses on the Undergraduate Course.

 

Implementation

Due to the constraints of the time difference between the UK and China, where most students were based, we knew learning on the course would need to be largely asynchronous. However, we were keen to promote active learning and so adopted the following approaches:

  • Use of the online authoring tool Xerte to create interactive learning materials which enabled students to have immediate feedback on tasks.
  • Incorporation of asynchronous peer and student-teacher interaction into the course each week through scaffolded tasks for the Blackboard Discussion Boards.
  • Setting up of small study groups of 3-4 students within each class of 16 students. Each group had fortnightly tutorials with the teacher and were encouraged to use the group for independent peer support.
  • Live online interactive sessions which took a ‘flipped’ approach, so students came prepared to share and discuss their work on a set task and ask any questions.

In order to track engagement with the learning materials we used Blackboard Tests to create short (4-5 questions) ‘Stop & Check’ quizzes at regular intervals throughout the week. We used the Grade Centre to monitor completion of these. We also made use of other student engagement monitoring features of Blackboard, in particular the Retention Centre within Evaluation and Blackboard Course Reports which enable instructors to track a range of user activity.

 

Impact

Our tracking showed that most students were engaging with the tasks daily, as required. We were very quickly able to identify a small group of students who were not engaging as hoped and target additional communication and support to these students.

Student feedback demonstrated that students perceived improvements in their language ability across the four skills (reading, writing, speaking and listening) and this was confirmed by their results at the end of the course. Student outcomes were good with over 90% of students achieving the scores they needed to progress to their chose degree programme. This compares favourably with the progression rate for the on-campus course which has run in previous years.

Feedback from teachers on the learning materials was very positive. One teacher commented that ‘The videos and Xerte lessons were excellent. As a new teacher I felt the course was very clear and it has been the best summer course I have worked on’. Teachers highlighted Xerte, the Discussion Boards and the interactive sessions as strengths of the course.

The materials and overall design of the course have informed the Pre-sessional Course (PSE 1) which is running this Autumn Term.

 

Reflections

Overall, we designed and delivered a course which met our objectives. Some reflections on the tools and approaches we employed are as follows:

Xerte lessons: these were definitely a successful part of the course enabling us to provide interactive asynchronous learning materials with immediate feedback to students. We also found the Xerte lessons enabled us to make coherent ‘packages’ of smaller tasks helping us to keep the Blackboard site uncluttered and easy to navigate.

Discussion Boards: teacher feedback indicated that this was a part of the course some felt was an enhancement of the previous F2F delivery. Points we found were key to the success of Discussion Board tasks were:

  • Creation of a new thread for each task to keep threads a manageable size
  • Linking to the specific thread from the task using hyperlinks
  • Detailed and specific Discussion Board task instructions for students broken down into steps of making an initial post and responding to classmates’ posts with deadlines for each step
  • Teacher presence on the Discussion Board
  • Teacher feedback on group use of the Discussion Board in live sessions to reinforce the importance of peer interaction

Small study groups: these were a helpful element of the course and greater use could have been made of them. For example, one teacher developed a system of having a rotating ‘group leader’ who took responsibility for guiding the group through an assigned task each week. In the future we could incorporate this approach and build more independent group work into the asynchronous learning materials to reinforce the importance of collaboration and peer learning.

Live sessions: student feedback showed clearly that this was an aspect of the course they particularly valued. Both students and teachers felt there should be more live contact but that these do not need to be long sessions; even an additional 30 minutes a day would have made a difference. Teachers and students commented that Teams provided a more stable connection for students in China than Blackboard Collaborate.

Blackboard Tests and monitoring features of Blackboard: these were undoubtedly useful tools for monitoring student engagement. However, they generate a great deal of data which is not always easy to interpret ‘at a glance’ and provides a fairly superficial account of engagement. Most teachers ended up devising their own tracking systems in Excel which enabled them to identify and track performance on certain key tasks each week.

 

Follow up

Taking into account the feedback from this year, materials developed could be used in future to facilitate a flipped learning approach on the course with students studying on campus or remotely. This would address the calls for more teacher-student interaction and enable the course to respond flexibility to external events. Currently, we are applying lessons learnt from the summer to the delivery of our Pre-sessional and Academic English Programmes running this term.

 

Links

The Pre-sessional English and Academic English Programme webpages give more details about the Programmes

Pre-sessional: http://www.reading.ac.uk/ISLI/study-in-the-uk/isli-pre-sessional-english.aspx

Academic English Programme: http://www.reading.ac.uk/ISLI/enhancing-studies/isli-aep.aspx

Introducing group assessment to improve constructive alignment: impact on teacher and student

Daniela Standen, School Director of Teaching and Learning, ISLI  Alison Nicholson, Honorary Fellow, UoR

Overview

In summer 2018-19 Italian and French in Institution-wide Language Programme, piloted paired Oral exams. The impact of the change is explored below. Although discussed in the context of language assessment, the drivers for change, challenges and outcomes are relevant to any discipline intending to introduce more authentic and collaborative tasks in their assessment mix. Group assessments constitute around 4% of the University Assessment types (EMA data, academic year 2019-20).

Objectives

  • improve constructive alignment between the learning outcomes, the teaching methodology and the assessment process
  • for students to be more relaxed and produce more authentic and spontaneous language
  • make the assessment process more efficient, with the aim to reduce teacher workload

Context

IWLP provides credit-bearing language learning opportunities for students across the University. Around 1,000 students learn a language with IWLP at Reading.

The learning outcomes of the modules talk about the ability to communicate in the language.  The teaching methodology employed favours student–student interaction and collaboration.  In class, students work mostly in pairs or small groups. The exam format, on the other hand, was structured so that a student would interact with the teacher.

The exam was often the first time students would have spoken one-to-one with the teacher. The change in interaction pattern could be intimidating and tended to produce stilted Q&A sessions or interrogations, not communication.

Implementation

Who was affected by the change?

221 Students

8 Teachers

7 Modules

4 Proficiency Levels

2 Languages

What changed?

  • The interlocution pattern changed from teacher-student to student-student, reflecting the normal pattern of in-class interaction
  • The marking criteria changed, so that quality of interaction was better defined and carried higher weight
  • The marking process changed, teachers as well as students were paired. Instead of the examiner re-listening to all the oral exams in order to award a mark, the exams were double staffed. One teacher concentrated on running the exam and marking using holistic marking criteria and the second teacher listened and marked using analytic rating scales

Expected outcomes

  • Students to be more relaxed and produce more authentic and spontaneous language
  • Students to student interaction creates a more relaxed atmosphere
  • Students take longer speaking turns
  • Students use more features of interaction

(Hardi Prasetyo, 2018)

  • For there to be perceived issues of validity and fairness around ‘interlocutor effects’ i.e. how does the competence of the person I am speaking to affect my outcomes. (Galaczi & French, 2011)

 Mitigation

  • Homogeneous pairings, through class observation
  • Include monologic and dialogic assessment tasks
  • Planned teacher intervention
  • Inclusion of communicative and linguistic marking criteria
  • Pairing teachers as well as students, for more robust moderation

Impact

Methods of evaluation

Questionnaires were sent to 32 students who had experienced the previous exam format to enable comparison.  Response rate was 30%, 70% from students of Italian. Responses were consistent across the two languages.

8 Teachers provided verbal or written feedback.

 Students’ Questionnaire Results

Overall students’ feedback was positive.  Students recognised closer alignment between teaching and assessment, and that talking to another student was more natural. They also reported increased opportunities to practise and felt well prepared.

However, they did not feel that the new format improved their opportunity to demonstrate their learning or speaking to a student more relaxing.  The qualitative feedback tells us that this is due to anxieties around pairings.

Teachers’ Feedback

  • Language production was more spontaneous and authentic. One teacher commented ‘it was a much more authentic situation and students really helped each other to communicate’
  • Marking changed from a focus on listening for errors towards rewarding successful communication
  • Workload decreased by up to 30%, for the average student cohort and peaks and troughs of work were better distributed

Reflections

Overall, the impact on both teachers and students was positive. Student reported that they were well briefed and had greater opportunities to practise before the exam. Teachers reported a positive impact on workloads and on the students’ ability to demonstrate they were able to communicate in the language.

However, this was not reflected in the students’ feedback. There is a clear discrepancy in the teachers and students’ perception of how the new format allows students to showcase learning.

Despite mitigating action being taken, students also reported anxiety around ‘interlocutor effect’.  Race (2014) tells us that even when universities have put all possible measures in place to make assessment fair they often fail to communicate this appropriately to students. The next steps should therefore focus on engaging students to bridge this perception gap.

Follow-up

Follow up was planned for the 2019-20 academic cycle but could not take place due to the COVID-19 pandemic.

References

Galaczi & French, in Taylor, L. (ed.), (2011). Examining Speaking: Research and practice in assessing second language speaking. Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo, Dehli, Tokyo, Mexico City: CUP.

Fulcher, G. (2003). Testing Second Language Speaking. Ediburgh: Pearson.

Hardi Prasetyo, A. (2018). Paired Oral Tests: A literature review. LLT Journal: A Journal on Language and Language Teaching, 21(Suppl.), 105-110.

Race, P. (2014) Making Learning happen (3rd ed.), Los Angeles; London: Sage

Race, P. (2015) The lecturer’s toolkit : a practical guide to assessment, learning and teaching (4th ed.), London ; New York, NY : Routledge, Taylor & Francis Group

 

How ISLI’s Assessment Team created an online oral exam for the Test of English for Educational Purposes (TEEP)

Fiona Orel– International Study and Language Institute (ISLI)

 

Overview

ISLI’s Test of English for Educational Purposes (TEEP) is administered at the end of pre-sessional courses as a measure of students’ academic English proficiency. The speaking test has traditionally been an academic discussion between two students that is facilitated by an interlocutor and marked by an observer.

This case study outlines the process of creating a version of the TEEP speaking test for 1-1 online delivery.

Objectives

  • To create an online TEEP speaking test that could be administered at the beginning of June to 95 students
  • To ensure reliability and security of results
  • To support students and teachers with the transition

Context

The Pre-sessional English course 3 (PSE 3) started in April during the period of lockdown.  At the end of the course all students sit a TEEP test which includes a test of speaking skills. We realised that we wouldn’t be able to administer the usual two student + two teachers test given the constraints with technology and the changes in teaching and learning which reduced to a certain degree the students’ opportunities for oral interaction and that we would need to develop a new 1-1 test that maintained the validity and reliability of the original TEEP Speaking test.

Implementation

We had two main objectives: to create a valid online 1-1 speaking test, and to make sure that the technology we used to administer the test was simple and straight-forward for both teachers and students, and would have reasonably reliable connectivity in the regions where students were based (China, Middle East and UK).

The first thing we needed to do was to return to our test specifications – what exactly were we hoping to assess through the oral exam? The original face-to-face test had five criteria: overall communication, interaction, fluency, accuracy and range, and intelligibility. We knew that interaction had been impacted by the move online, but decided that the aspect of responding appropriately to others was a crucial aspect of interaction that needed to remain and included this in the ‘overall communication’ criteria. Recognising also that interlocutors would also need to be examiners, we worked on streamlining the criteria to remove redundancy and repetition and to ensure that each block contained the same type of description in the same order thereby making it easier for tutors to skim and recall.

We then worked out exactly what functions and skills in speaking that we wanted to test and how we could do that while mostly working with existing resources. We aligned with the original test specifications by testing students’ ability to:

  • Provide appropriate responses to questions and prompt
  • Describe experiences and things
  • Give and justify an opinion by, for example, stating an opinion, explaining causes and effects, comparing, evaluating.

The test format that enabled this was:

  • Part one: an interview with the student about their studies and experience of studying online
  • Part two: problem solving scenario: Students are introduced to a problem which the teacher screen shares with them and they are given three possible solutions to compare, evaluate and rank most to least effective
  • Part three: abstract discussion building on the talk given in part two

The final stage was trialling a platform to conduct the tests. We had considered Zoom due to its reliability but discounted it due to security concerns. BB Collaborate had connectivity issues in China so we decided to use Teams as connectivity was generally better and students and teachers were familiar with the platform as they had been using it for tutorials. Due to the spread of students over time zones, we decided to spread the speaking tests over three mornings finishing by 11:00 BST on each day. We kept the final slot on Friday free for all teachers to enable rescheduling of tests for any student experiencing issues with connectivity on the day.

Finally, we needed to help teachers and students prepare for the tests. For students, learning materials were produced with videos of a sample test, there was a well-attended webinar to introduce the format and requirements, and the recording of this webinar was made available to all students along with a document on their BB course. This instructed them what to do before test day and what to expect on test day.

The test format and procedures were introduced to teachers with instructions for tasks to do before the test, during the test, and after the test. There was also an examiner’s script prepared with integrated instructions and speech to standardise how the tests were administered. Each test was recorded to ensure security and to enable moderation. All students had to verify their identity at the start of the test. The test recording caused some problems as we knew that the video would have to be downloaded and deleted from Stream before anyone else or the student in the Team meeting who had been tested could access it. For this reason we allowed 40 minutes for each 20 minute interview as downloading was sometimes a lengthy process depending on internet speeds. We had 2 or 3 people available each day to pick up any problems such as a teacher being unwell or having tech issues, and/or a student experiencing problems. This worked well and on the first two days we did have to reschedule a number of tests, fortunately, all worked well on the final day. The teachers were fully committed and worked hard to put students at ease, informal feedback from students was the appreciation of an opportunity to talk 1-1 with a tutor, and tutors said that the test format allowed for plenty of evidence upon which to base a decision.

Impact

The test was successful overall and there were fewer technical issues than we had anticipated. Teachers and students were happy with it as an assessment measure and we were able to award valid and reliable grades.

Working together collaboratively with the teachers and the Programme Director was incredibly rewarding and meant that we had a wide resource base of talent and experience when we did run into any problems.

Reflections

Incredibly detailed planning, the sharing of information across Assessment and Pre-sessional Teams, and much appreciated support from the TEL team helped to make the test a success. Students and teachers had very clear and detailed instructions and knew exactly what was expected and how the tests would be conducted. The sharing of expertise across teams meant that problems were solved quickly and creatively, and it is good to see this practice becoming the norm.

We need to work on the issue of downloading and deleting the video after each test as this caused some anxiety for some teachers with slower internet connection. We also need to have more technical support available, especially on the first day. Most students had tested their equipment as instructed but some who hadn’t experienced issues. It would be even better if a similar activity could be built into the course so that teachers and students experience the test situation before the final test.

Follow up

ISLI’s Assessment Team is now preparing to administer the same tests to a much larger cohort of students at the end of August. We will apply the lessons learned during this administration and work to make the process easier for teachers.