Capturing and Developing Students’ Assessment Literacy

By edwardwhite Comment Closed

Hilary Harris, Maria Danos, Natthapoj Vincent Trakulphadetkrai, Stephanie Sharp, Cathy Tissot, Anna Tsakalaki, Rowena Kasprowicz – Institute of Education

hilary.a.harris@reading.ac.uk

Overview

The Institute of Education’s (IoE) T&L Group on Assessment Literacy worked collaboratively with 300+ students to ascertain the clarity level of assessment criteria used in all programmes across the IoE.  The findings were used to develop a report containing key findings and recommendations, which were then shared with programme directors. The findings also fed into the development of a Glossary of Common Assessment Terms to help develop students’ assessment literacy. SDTLs and DDTLs of almost all UoR Schools and the Academic Director of the UoR Malaysia campus have now either had one-to-one meetings with us or contacted us to explore how our group’s work could be adopted and adapted in their own setting.

Objectives

The aims of the activity were to:

  • Develop students’ assessment literacy, specifically in terms of their understanding of assessment criteria which are used in marking rubrics
  • Engage students in reviewing the clarity of assessment criteria and terms used in marking rubrics
  • Engage programme directors in reflecting on the construction of their marking rubrics
  • Develop an IoE-wide glossary of common assessment terms

Context

The IoE has set up T&L Groups to enhance different aspects of our teaching and learning practices as part of the peer review process. The T&L Group on Assessment Literacy has been meeting since 2017, and is made up of seven academics from a wide range of undergraduate and postgraduate programmes.

As marking rubrics are now used for all summative assessments at the IoE (and to some extent across the University), ensuring that students have a good understanding of the embedded assessment terms matters as the criteria inform students of what is expected of them for a particular assessment. Moreover, the marking rubrics can also be used by students to develop their draft work before submission.

Implementation

The Group asked 300+ students across all the IoE programmes to indicate the clarity level of their programme’s assessment criteria by circling any terms on the marking rubric that they were confused by. The Group collated the information and created a summary table for each programme, ranking assessment terms according to how often the terms were highlighted by the students.  Each group member then wrote a brief report for each programme with key findings and recommendations on alternative assessment terms that are clearer (e.g. to replace ‘recapitulation’ with ‘summary’; ‘perceptive’ with ‘insightful’, etc.). In some other cases where the use of specific terminology is essential (e.g. scholarship or ethics), the Group’s advice is for module convenors to spend some time within classroom to explain such terms to students and refer students to the assessment glossary for further support and examples. Both the Report and the Glossary were disseminated to programme directors and their teams, who were then able to use the evidence in the report to reflect on their programme’s assessment criteria and consider with their team any changes that they would like to make that would make the marking rubric more accessible and easier to understand by the students.

Impact

At the IoE, the work has already made an impact in that programme directors have reflected on their assessment criteria alongside their teams and have acted on the Group’s recommendations (e.g. replacing problematic terms in their marking rubrics with terms that are easier to understand by students.) The Glossary has been used by IoE programme directors and module convenors when introducing the assessment and their marking rubrics. The Glossary has also been uploaded onto Blackboard for students to consult independently. The feedback from students on the Glossary has also been very positive. For example, one student commented that “The definitions were useful and the examples provided were even more helpful for clarifying exactly what the terms mean. The glossary is laid out in a clear and easy to follow way for each key term”.

Beyond the IoE, impact is being generated. Specifically, SDTLs and DDTLs of almost all UoR Schools and the Academic Director of the UoR Malaysia campus have now either had one-to-one meetings with us or contacted us to explore how our group’s work could be adopted and adapted in their own setting. The Group has been invited to give talks on its work at CQSD events and the School of Law’s T&L seminar. The Group is also currently working with academic colleagues at other universities (nationally and internationally) to replicate this Group’s work and generate impact beyond the UoR.

Reflections

The activity was very successful as:

  • The Group had a clear focus of what it wanted to achieve
  • The Group was given time to carry out its work
  • There was strong leadership of the team, with each member being allocated specific contributions to the project

The process of involving students in reviewing terms on marking rubrics has empowered them to treat the documents critically and start a conversation with their lecturers about the purpose of marking rubrics, as well as being involved in as partners in making the marking rubric work for them.

There were some challenges that needed to be overcome/ ideas for improving the project:

  • When presented to colleagues at the Staff Day, some members of staff expressed the view that ‘tricky’ terms should be retained as developing an understanding of these terms is part of the transition to HE study. This was recognised in our report which suggests that technical terms (e.g. methodology) could be retained provided that they are explained to students.

Follow up

The Group plans to spend the 2019/2020 academic year generating and capturing the impact of its work across and beyond the UoR.

Reframing Identity 360

By edwardwhite Comment Closed

Kate Allen, Department of Art, k.allen@reading.ac.uk

Overview

An investigative artwork that explores identity using 360 cameras developed through practical, alumni led workshops and socially engaged art with current art students, school groups and the general public. Part of ArtLab Movement’ at Tate Exchange (TEx) 2019 at the Tate Modern on March and be archived on the ArtLab website.

Objectives

- Contribute to live art event/out-reach work experience led by Alumni at Tate Exchange 1-3 March 2019

- Explore identity capture with 360 cameras

- 360 cameras experimentation including designing, capturing, printing and editing.

- Create portraits with purpleSTARS, people with learning disabilities and children from Widening Participation schools in Reading.

Context

Reframing Identity explored self-portraits in shot in 360, developed as a response to Tania Bruguera’s Turbine Hall Commission concerning institutional power, borders and migration. Can 360 self-portraits raise awareness of how interconnected we are, when no person is ever behind the 360 camera, everyone is included.

Implementation

Alumni and Virtual Reality artist Kassie Headon researched ideas in response to Tania Bruguera installation at Tate Modern inspired by Bruguera’s ideas on inclusion, connecting to Kate Allen’s research with purpleSTARS a group of people with and with learning disabilities who aim to make museums more inclusive. Kassie demonstrated to students and purpleSTARS how to use the GoPro Fusion Camera and the app to edit 360 content. Activities to share the 360 self portrait concept with visitors were developed including drawing cylindrical self-portraits which they could then wear on their heads for a 360 selfie. Students facilitated the Reframing Identity 360 workshop as part of ArtLab Movement at TEx. Using 360 cameras was a new experience and concept for our students and most people visiting the TEx. The 360 self-portraits were exhibited via live video stream from the 360 cameras on an iPad displayed at the Tate and let participants explore the views, which they could manipulate and distort to create the desired effect. Participants 360 self-portraits were also printed or sent to the visitors phone.

Impact

The impact of Reframing Identity 360 created access and inclusion with new technologies for students and the public. Experiencing the live video stream frequently gave visitors an ‘Oh Wow’ moment. TEx gave an opportunity for research led teaching with Dr Allen purpleSTARS, Alumni Kassie Headon and current BA students to explore the concept of 360 self-portraits gain professional practice experience facilitating the workshops and technical skills working, with the 360 camera. The 360 cameras are now part of the digital equipment available to students with a core team of ArtLab students now familiar with their potential and how to use them.

Reflections

Working with new technologies in collaboration with Alumni, ArtLab students and purpleSTARS led to new perspectives on ideas of inclusion and self -portraiture. The experimental research occurred in response to work at the Tate and in collaboration with visitors to TEx. The project built capacity and awareness of new technology being introduced into the Art Dept learning through research and practical experiences the potential to create artworks and inclusive engagements.

Follow up

Kassie Headen continued to work with the 360 camera collaborating with widening participation schools during the ArtLab summer workshops 2019 exploring spaces and manipulating 2d versions of 3d space.

We are developing further research collaborations and research led teaching opportunities for ideas exploring inclusion in museums and immersive virtual reality artworks/experiences using Oculus Rift technology.

Links and References

We created a 360 recording of our Reframing Identity event at the Tate https://www.thinglink.com/mediacard/1158753748827242499?autoplay=0&autorotate=0&displaytitle=1&rel=1

ArtLab documents the workshop

https://readingartlab.com/2019/04/25/artlab-tate-exchange-visual-diary-2nd-and-3rd-march-2019/

purpleSTARS web documentation

https://purplestars.org.uk/2017/11/12/purplestars-at-tate-gallery-2018/

Tate Exchange webpage

https://www.tate.org.uk/whats-on/tate-modern/tate-exchange/workshop/reading-assembly-movement

A ‘Sherlock’ Approach to Physician Associate Learning: Using Workshops to Promote Critical Thought

By edwardwhite Comment Closed

Dr Sarah Greenwood, Lecturer, Physician Associate Programme, School of Chemistry, Food and Pharmacy, s.l.greenwood@reading.ac.uk

Physician Associate (PA) students are talented life-sciences postgraduates who must quickly develop critical thinking skills in relation to medicine. Our PA programme focuses on the core skill of applying bioscientific and medical theory to skills of history taking, clinical examination, investigation diagnosis and treatment in order to produce safe, competent practitioners within two years.

Our student numbers have doubled in the five years since the programme began, and so as we strive to accommodate higher numbers, we witness greater diversity in learning styles. We recognised the need to promote advanced critical thinking amongst all our students in creative ways.

Firstly, funding secured access for all our students to McGraw Hill’s ‘Connect Online’, (which included an anatomy and physiology e-book, histology slides, media files, assessment tests and a cadaver dissection) for students to work though system by system. This online package proved very popular with the students whereby the overall average grade over 18 assignments was 94.47%.  Students’ engagement could be regularly monitored by the lead lecturer and areas of difficulty were successfully addressed.

Secondly, funding enabled us to develop in-house ‘PA workshop investigation packs’ – which were used by groups of PA students in our clinical skills suite, and online. The packs were themed according to body systems, and consisted of series of work stations containing instructions and various learning materials. Our PA students worked together to tackle core practical and theoretical concepts, working out solutions together in a systematic manner – hence using a ‘Sherlock’ detective approach to their learning!  The funding covered the cost of all our workshop materials, in particular laminated displays/charts, questions and visual guides; these are particularly valued because they are reusable for future cohorts of PA students.

The learning processes aimed to mirror the role of the Physician Associate in practice. As such, the learning packs provided engaging, challenging and motivational learning to develop essential skills safely and effectively.

The effectiveness of the workshops became apparent early on – as evidenced by the number of students passing their formative practical examinations at first attempt (shown below).

Formative results without workshops                        Formative results with workshops

Graph showing improved results following workshop

In the summative end-of-year objective structured clinical examinations (OSCEs): 28% of our workshop students achieved > 80% in these practical exams, with 5 students achieving 90% or above  – this exceeded the previous cohort’s results where only 8% of students scored over 80% and none scored 90% or above. There was an overall improvement in mean performance from 66% to 70%.Graph showing improved results following workshop

 

The student evaluations were very positive; all students were able to articulate what they had gained from the experience:

“Examination station was useful because I was able to practice examination skills in an -almost- clinical environment, with the help of teachers. Another station I found useful was the BNF station. It gave me an understanding of how to use the BNF in a given time frame, and find what I am looking for. The BNF station also helped me identify a lot of drugs for certain conditions, which I would not have known otherwise”

“The upper and lower neurological examinations were very useful. This is because I found the overlap and structure similar and reinforce the other. I also found the breast examination very useful because I am less likely to get patient experience with this as a male student”.

“Listening to the heart murmurs station with questions on hypertension – allowed us to work through different case examples”

The lecturers and students all recognised the value of the workshops, and this fun, interactive and relaxed teaching approach has now been formally integrated into the curriculum. We are most grateful for the support of the University’s teaching and learning enhancement scheme which funded this intervention.

Interdisciplinary teaching: Science in Culture

By edwardwhite Comment Closed

Professor Nick Battey, School of Biological Sciences
n.h.battey@reading.ac.uk

Overview

12402A module for Part Three students was created by a collaborative effort between the Department of English Literature, the Department of History, and the School of Biological Sciences (SBS), called Science in Culture. This module was well-received by students, who found value in obtaining the perspective of disciplines other than their own, and experiencing teaching and learning methods outside the norm of their previous study.

Objectives

  • Offer a truly disciplinary module allowing students from English Literature, History, and SBS to study alongside one another, learning through the diverse teaching methods of science and the humanities.
  • Develop in students a broader, critical understanding of the precepts of science.
  • Provide an integrated view of science (with emphasis on Biological Sciences) within culture.

Context

The development of this collaborative module grew out of an Arts and Humanities Research Council sponsored project which looked at the value of literary and historical study of biology to students of biological sciences. An element of this was a workshop, ‘Cultivating Common Ground’, which aimed to foster interdisciplinary discussion between biology and the humanities. One of the key findings of the scoping study was that it would be beneficial to develop at least one module that taught both biology and humanities students alongside one another in an interdisciplinary way.

Implementation

The module was developed over a number of years by staff from SBS, English and History. The module designers from the different disciplines were determined to ensure that what was developed was a truly interdisciplinary module, breaking down the perceived divide between the sciences and the humanities, and showing how the different approaches and bodies of knowledge bear on the same questions.

The module is taught over one term. Students receive lectures and partake in seminar discussions on a historical, literary, or scientific concept, and also conduct lab work on subjects related to those explored in the lectures. As an example of this, in lab work students will identify a mutated gene, and explore the use of mutations for understanding how genes work. This topic of mutation can then be explored in its literary and historic contexts. The difference that exists between the scientific, literary and historical approaches can then be explored as a cultural challenge. From the ‘Cultivating Common Ground’ workshop, consensus had emerged that interdisciplinary learning and teaching needed to be ‘narrow and deep’. As a result, the module focuses on a defined set of ‘problems’, rather than ‘grand themes’, allowing a deeper exploration thereof, and situation of this within the cultural dynamics and methods of science.

In order to ensure students experience different ways of learning, students were given a variety of tasks, ranging from interpreting poems or discussing the history of a scientific process, which they recorded in a learning journal, these being marked and receiving feedback from tutors each week. While the completion of this task over the course of the module was an aspect of the summative assessment, the weekly feedback provided regular formative feedback to students. A focus on formative assessment was recognised as being important by the scoping study, as students on such an interdisciplinary module would require greater opportunity to learn what was expected of them. Linking formative assessment to the summative assessment ensured that students would be motivated to engage and receive valuable feedback. Students taking the module as part of a History or English Literature degree, for whom the module was worth 20 credits, rather than 10, also wrote a summative essay.

Impact

The project was successful in delivering a truly interdisciplinary module, with collaboration between the School of Biological Studies, the Department of English and the Department of History. The module was well-received by students, who reported that they appreciated the value of getting different perspectives on their disciplines.

Reflections

The greatest challenge in creating this module was achieving interdisciplinarity, as the teaching and learning strategies best suited to the individual disciplines were not necessarily suited to the teaching of an interdisciplinary module. That the module was in development for a number of years reflects the difficulty that developing an interdisciplinary approach, and this was made increasingly difficult by the paucity of existing literature on the topic from which to draw suitable practices. As a result, there had to be a number of iterative developments in order to create a module that could be delivered in a way which best achieved its learning outcomes.

Interdisciplinarity also provided a challenge with regards marking of assessments. As each discipline has different expectations, it was necessary for marking to be a collaborative process, with compromise being reached between assessors.

While the provision of multiple opportunities for formative assessment and feedback had value, given that it helped introduce students to the other disciplines, and encouraged deep learning, the process was strenuous, for both students and staff.

As the module was interdisciplinary, this meant that students had to engage with topics and processes outside the norm of their previous academic study. As a result, despite their enjoyment and high attainment, students on the module did find it challenging.

Follow up

Following the successful running of the module during the 2014-15 academic year, the module has been offered again, with slight revisions. One of the revisions has been in assessment, with students producing a report at the end of the module, rather than creating a learning portfolio over the course of the module, thus somewhat reducing the workload of staff and students. A group presentation has also been introduced, providing a different type of assessment, and making interdisciplinary collaborative group work part of summative assessment.

Links

Reviewing assessment and feedback in Part One: getting assessment and feedback right with large classes

By edwardwhite Comment Closed

Dr Natasha Barrett, School of Biological Sciences
n.e.barrett@reading.ac.uk
Year(s) of activity: 2010/11
Overview

Objectives

  • Review the quantity, type and timing of assessments carried out in compulsory modules taken by students in the School of Biological Sciences.
  • Recommend better practices for assessment and feedback.

Context

The massification and marketisation of Higher Education means that it is increasingly important that the University of Reading perform well in term of student satisfaction and academic results. The National Student Surveys between 2005 and 2011 and the Reading Student Survey of 2008 and the National Student Survey both indicated that assessment and feedback were areas in which the University of Reading and the School of Biological Sciences needed to improve.

Implementation

An evaluation of online systems of peer assessment for group work

By edwardwhite Comment Closed

Cathy Hughes and Heike Bruton, Henley Business School
catherine.hughes@reading.ac.uk

Overview

Online peer assessment systems were evaluated for their suitability in providing a platform to allow peer assessment to be conducted in the context of group work.

Objectives

  • To establish the criteria against which peer assessment systems should be evaluated.
  • To evaluate the suitability of online systems of peer assessment.
  • To provide a way forward for Henley Business School to develop peer assessment for group work.

Context

There are many well-documented benefits of group work for students. Given the recognised issue that members of a group may not contribute equally to a task, and that it can be difficult for tutors to accurately judge the contributions made by individuals within a group, this presents a context in which peer assessment can be utilised, allowing students to assess the process of group work. Within Henley Business School, Cathy Hughes has utilised peer assessment for group work in Real Estate and Planning, and developed a bespoke web-based system to facilitate this. As this system was not sustainable, the project was funded to evaluate the suitability of other web-based peer assessment systems for use at the University.

Implementation

By first establishing how academics across the University use peer assessment in a range of subjects, it would be possible to establish the criteria against which available online systems of peer assessment for group work could be evaluated. This was done by performing a series of interviews with academics who already used peer assessment, these volunteering after a call for respondents was made through the T&L distribution list. The eleven interviewees were drawn from across seven departments. The interviews revealed that five separate peer assessment systems were in use across the University. These systems had, with one exception, been in use for four years or fewer. Peer assessment at the University of Reading has been utilised at all Parts, for a range of group sizes (between three and ten depending on the task being performed). While a range of credits were affected by peer assessment (between 1 and 20), no module used peer assessment to contribute 100% of the final mark, though in one case it did contribute 90% of the final mark.

With peer assessment of group work, students may be required to mark their peers against set criteria, or in a more holistic manner whereby students award an overall mark to each of the others in their group. Given the subjective nature of the marking process, peer assessment can be open to abuse, and so interviewees stressed the need for them to be able to check and moderate marks. All interviewees stated that they collated evidential material which could be referred in case of dispute.

All systems which were in use generated numerical data on an individual’s performance in group work, but with regard to feedback there were differences in what users required. Some users of peer assessment used the numerical data to construct feedback for students, and in one case students provided their peers with anonymised feedback.

It was apparent from interviews that performing peer assessment requires a large amount of support to be provided by staff.  Other than the system that was in use in Henley Business School and the Department of Chemistry, all systems had students fill out paper forms, with calculations then being performed manually or requiring data to be input into a spreadsheet for manipulation.  This high workload reflected a need to disseminate online peer assessment, in order to reduce the workload of those already conducting peer assessment, and to attempt to lower the barrier to entry for others interested in peer assessment, but unable to accept the increased workload.

With the input from interviewees, it was possible to put together criteria for evaluation of online peer assessment systems:

  1. Pedagogy:
    • Any systems must provide a fair and valid method for distinguishing between contributions to group work.
  2. Flexibility:
    • Peer assessment is used in different settings for different types of group work. The methods used vary on several dimensions, such as:
      1. Whether holistic or criteria based.
      2. The amount of adjustment to be made to the group mark.
      3. The nature of the grading required by students, such as use of a Likert scale, or splitting marks between the group
      4. Whether written comments are required from the students along with a numerical grading of their peers.
      5. The detail and nature of feedback that is given to students such as: grade or comment on group performance as a whole; the performance of the student against individual criteria; further explanatory comments received from students or given by academics.
    • Therefore any system must be flexible and capable of adapting to these environments.
  3. Control:
    • Academics require some control over the resulting marks from peer assessment. While the online peer assessment tool will calculate marks, these will have to be visible to tutors, and academics have to have the ability to moderate these.
  4. Ease of use:
    • Given the amount of work involved in running peer assessment of group work, it is necessary for any online system to be both easy to use by staff and reduce their workload. The other aspect of this is ease of use for the student. The current schemes in use may be work-intensive for staff, but they do have the benefit of providing ease of use for students.
  5. Incorporation of evidence:
    • The collection of evidence to support and validate marks provided under peer assessment would ideally be part of any online system.
  6. Technical integration and support:
    • An online peer assessment system must be capable of being supported by the University in terms of IT and training
  7. Security:
    • Given the nature of the data, the system must be secure.

Four online peer assessment systems were analysed against these criteria: iPeer, SPARKplus, WebPA, and the bespoke peer assessment system created for use in Real Estate and Planning.

Findings

A brief overview of the findings is as follows:

iPeer

While iPeer can be used to collect data for the purposes of evaluation, unlike other systems evaluated the manipulation and interpretation of said data is left to the tutor, thus maintaining some of the workload that it was hoped would be avoided. While its ease of use was good, for staff and students, there were limits to what it was possible to achieve using iPeer, and supporting documentation was difficult to access.

SPARKplus

SPARKplus is a versatile tool for the conduct of online peer assessment, allowing students to be marked against specific criteria or in a more holistic manner, and generating a score based upon their peer assessed contribution to group work and the tutor’s assessment of what the group produces. There were, however, disadvantages: SPARKplus does not allow for the gathering of additional evidential material, and it was difficult at the time of the evidence gathering to find information about the system. While SPARKplus is an online system, it is not possible to incorporate it into Blackboard Learn that might have clarified its suitability.

WebPA

For WebPA there was a great deal of documentation available, aiding its evaluation. It appeared to be easy to use, and is able to be incorporated into Blackboard Learn. The main disadvantages of using WebPA was that it does not allow evidential data to be gathered, and that there is no capacity for written comments to be shared with students, as these are only visible to the tutor.

Bespoke REP system

The bespoke online peer assessment system developed within Real Estate and Planning and also used in the Department of Chemistry is similar to WebPA in terms of the underpinning scoring algorithm, and has the added advantage of allowing the collection of evidential material. Its main disadvantage is that it is comparatively difficult to configure, requiring a reasonable level of competence with Microsoft Excel. Additionally, technical support for the system is reliant on the University of Reading Information Technology Services.

Reflections

Developing the use of the interactive whiteboard for initial teacher trainees (2011-12)

By edwardwhite Comment Closed

Catherine Foley, Institute of Education
c.m.foley@reading.ac.uk

Overview

With interactive whiteboards becoming a well-established feature of English primary schools classrooms over the last decade, it is vital that the primary Post-Graduate Certificate of Education (PGCE) programme taught at the University of Reading’s Institute of Education prepares it graduates to be confident and competent in using interactive whiteboard technology in the classroom, including making pedagogically sound, informed decisions about when, when not, and how the interactive whiteboard can enhance learning.

Objectives

    • Explore how trainees can be supported to use the interactive whiteboard in their teaching of mathematics.
    • Gain an informed view of the entry- and exit-level interactive whiteboard skills and understanding of trainees to inform future programme planning.
    • Ensure that the trainee voice is incorporated into developmental planning.
    • Make recommendations regarding embedding the use of interactive whiteboard technology into our wider initial teacher training provision.

Implementation

Initial data collection was conducted through a questionnaire, which was administered towards the end of the trainees’ first week on the programme. This questionnaire was used to gather data on skills and competencies with regards interactive whiteboard technology.

The results of the initial questionnaire revealed that trainees on the programme generally had little or no experience of using interactive whiteboard technology, and that confidence levels for using the interactive whiteboard for general teaching and learning, and specifically within mathematics lessons, were low. The questionnaire had also asked trainees to rank statements in order to indicate the most important to meet their needs. The most preferred statement was that trainees would like support for the skills of how to use an interactive whiteboard. Second was that the use of the interactive whiteboard for teaching and learning be modelled within sessions.

On the basis of the questionnaire results, the following action plan was discussed and agreed with the programme director:

  1. Modelling of interactive whiteboard use throughout taught mathematics sessions. Where interactive whiteboard use was modelled, the ‘stepping out’ technique, as described in Lunenberg et al., was used explicitly to focus trainee’ attention on how the interactive whiteboard has been used, and more importantly, why and to what effect.
  2. Optional workshops during free-time within Autumn and Spring Terms.  These were aimed to ensure a basic level of skills, tied in with the interactive functions most likely to have an impact.  These workshops were limited to 10 trainees, to allow greater access to the interactive whiteboard and less pressure on ‘getting it right’.  The skills addressed during these workshops were based on a combination of student requests, the experience of the project leader, and those outlined in Beauchamp and Parkinson.
  3. Provision for peer sharing of resources created on school experience later in the programme.  In workshops, trainees who had developed interactive whiteboard skills while on placement were invited to share their expertise with other trainees.
  4. Opportunities for peer modelling within starter activities.  Trainees were encouraged to use the interactive whiteboard where appropriate in the presentation of starter activities to their peers, which occurs on a rolling programme throughout the module.

At the end of the module a follow-up questionnaire was administered. This contained a mixture of identical questions to the initial questionnaire, to allow comparison with the results that were gained at the beginning of the programme, and items designed to evaluate the different forms of support that had been provided.

Reflections

Trainees had, by the conclusion of the module, improved their experience with the use of interactive whiteboards, their confidence in doing so, their preparedness to use interactive whiteboard technology for the teaching of mathematics, and increased the level of skill they possessed in writing, manipulating shapes or images, and inserting children’s work or photographs.

It was possible as a result of the project to make the following recommendations for the Institute of Education, which may be useful for related subjects across the University of Reading:

  1. If staff are expected to integrate modelling of appropriate use of interactive whiteboards into their practice, they will need both technical and peer support in order to develop their own confidence. This could be tackled through teaching and learning seminars, practical workshops, software provision and technician time, in much the same way as the project itself supported trainees.
  2. Some of the technical skills could be integrated into ICT modules, allowing subject modules to focus on the most effective pedagogy within their subject.
  3. Primary programmes could consider some kind of formative collaborative tasks to develop and review interactive whiteboard-based activities within subject areas.
  4. The interactive whiteboard provisions in schools could be audited in order to ensure that the Institute of Education’s software and hardware provision is appropriately matched to what trainees will encounter, and incorporated a request for supervising students to comment on their tutees’ interactive whiteboard use as a quality assurance check.
  5. Time support so that trainees reach a basic level of confidence with the use of interactive whiteboard technology before their first school placement.

Links and Resources

Mieke Lunenberg, Fred Korthagen, and Anja Swennen (2007): The teacher educator as role model.  Teaching and Teacher Education, 23 (5)
Gary Beauchamp and John Parkinson (2005): Beyond the ‘wow’ factor: developing interactivity with the interactive whiteboard.  School Science Review, 86 (316)

Engaging Diverse Learning Communities in Partnership: A Case Study Involving Professional Practice Students in Re-designing an Assessment

By edwardwhite Comment Closed

 

 

 

 

Lucy Hart (student – trainee PWP)- l.hart@student.reading.ac.uk 

Tamara Wiehe (staff – PWP Clinical Educator)- t.wiehe@reading.ac.uk

Charlie Waller Institute, School of Psychology and Clinical Language Sciences

Overview

This case study re-designed an assessment for two Higher Education programmes where students train to become Psychological Wellbeing Practitioners (PWP) in the NHS. The use of remote methods engaged harder to reach students in the re-design of the assessment tool. The project promotes the effectiveness of partnership working across diverse learning communities, by placing student views at the centre of decision making. In line with one of the University’s principles of partnership (2018) – shared responsibility for the process and outcome – this blog has been created by a student involved in the focus group and the member of teaching staff leading the project.

Objectives

  • Improve the design of an assessment across the University’s PWP training programmes.
  • Involve students throughout the re-design process, ensuring student voices and experiences are acknowledged.
  • Implement the new assessment design with the next cohorts.

Context

It was proposed by students in modular feedback and staff in a quarterly meeting that the design of an assessment on the PWP training programmes could be improved. These programmes are grounded in evidence-based, self-reflective and collaborative practice. Therefore, it was appropriate to maintain this style of working throughout the process. This was achieved through the students reflecting on their experiences when generating ideas and reviewing the re-designed assessment.

Implementation

Traditional methods of partnership were not suitable for our students due to the nature of the PWP training programmes. Their week consists of one teaching day running from 9:30-4:30, a study day and three days practising clinically as a trainee PWP in an NHS service. Location was another factor as many of our students commute to University and live closer to their workplace. The use of technology and remote working enabled us to overcome these barriers and work in partnership with our students.

The partnership process followed these three steps:

 

 

 

 

 

 

 

When generating ideas and reviewing the proposed assessment, we, the professional practice students, considered the following points:

  • Assessment design – consistency in using vignettes throughout the course meaning students will be familiar with this method of working. Word limit ensures concise responses.
  • Time frame – the release date of the essay in proportion to the examination date.
  • Feasibility – will there be enough study days to compensate for the change in design allowing trainees to plan their essays.
  • Academic support – opportunities within the academic timetable to provide additional supervision-style sessions later in the module to support students.
  • Learning materials – accessibility to resources on blackboard. Assigning study days to allow planning of essay.

Impact

  • It was agreed that the original ICT would be replaced with written coursework based on a vignette and implemented with our next cohorts.
  • The assessment aligned with the module learning outcomes and student experiences were considered in a meaningful way.
  • Harder to reach students were able to engage in the re-design of the assessment through effective communication methods.

Reflections

Student perspective:

“Being the expert of our experiences, it was refreshing to have our voices and experiences heard. We hope the re-design supports future cohorts and reduces anxieties around managing both university and service-based training. The focus group was a success due to the clear agenda setting and feasibility of remote online working. It can be proposed that a larger focus group would have beneficial during the review stage to remove biases associated with a small sample size.”

Staff perspective:

“Student input allowed us to hear more about their experiences during the training and took a lot of pressure off of staff to always be the ones coming up with solutions. The outcomes have a far reaching impact beyond that of the students and staff on the programme in terms of engaging diverse learning communities in Higher Education and forming more connections between Universities and NHS services. Although inclusivity and diversity was considered throughout, more participants in the virtual focus group would improve this further. Students could also have more power over the creation of the assessment materials themselves. Both of these reflections will inform my professional practice going forwards.”

Using Flipped Learning to Meet the Challenges of Large Group Lectures

By edwardwhite Comment Closed

Adopting a flipped classroom approach to meet the challenges of large group lectures

Name/School/ Email address

Amanda Millmore / School of Law / a.millmore@reading.ac.uk

Overview

Faced with double-teaching a cohort of 480 students (plus an additional 30 in University of Reading Malaysia), I was concerned to ensure that students in each lecture group had a similar teaching experience. My solution was to “flip” some of the learning, by recording short video lectures covering content that I would otherwise have lectured live and to use the time freed up to slow the pace and instigate active learning within the lectures. Students provided overwhelmingly positive feedback in formal and informal module evaluations, the introduction of flipped learning has aided the welfare of students, allowing those who are absent or who have disabilities or language barriers to revisit material as and when needed. For staff, it has aided the reduction in my workload and has the ongoing benefit of reducing workload of colleagues who have taken over teaching the module.

Objectives

  • Record short video lectures to supplement live lectures.
  • Use the time freed up by the removal of content no longer delivered live to introduce active learning techniques within the lectures.
  • Support the students in their problem-solving skills (tested in the end of year examination).

Context

The module “General Introduction to Law” is a “lecture only” first year undergraduate module, which is mandatory for many non-law students, covering unfamiliar legal concepts. Whilst I have previously tried to introduce some active learning into these lectures, I have struggled with time constraints due to the sheer volume of compulsory material to be covered.

Student feedback requested more support in tackling legal problem questions, I wanted to assist students and needed to free up some space within the lectures to do this and “flipping” some of the content by creating videos seemed to offer a solution.

As many academics (Berrett, 2012; Schaffzin, 2016) have noted, there is more to flipping than merely moving lectures online, it is about a change of pedagogical approach.

Implementation

I sought initial support from the TEL (Technology Enhanced Learning) team, who were very happy to give advice about technology options. I selected the free Screencast-O-Matic software, which was simple to use with minimal equipment (a headset with microphone plugged into my computer).

I recorded 8 short videos, which were screencasts of some of my lecture slides with my narration; 6 were traditional lecture content and 2 were problem solving advice and modelling an exemplar problem question and answer (which I had previously offered as straightforward read-only documents on Blackboard).

The software that I used restricted me to 15 minute videos, which worked well for maintaining student attention. My screencast videos were embedded within the Blackboard module and could also be viewed directly on the internet https://screencast-o-matic.com/u/iIMC/AmandaMillmoreGeneralIntroductiontoLaw.

I reminded students to watch the videos via email and during the lectures, and I was able to track the number of views of each video, which enabled me to prompt students if levels of viewing were lower than I expected.

By moving some of the content delivery online I was also able to incorporate more problem-solving tasks into the live lectures. I was able to slow the pace and to invite dialogue, often by using technology enhanced learning. For example, I devoted an hour to tackling an exam-style problem, with students actively working to solve the problem using the knowledge gained via the flipped learning videos and previous live lectures. I used the applications Mentimeter, Socrative and Kahoot to interact with the students, asking them multiple-choice questions, encouraging them to vote on questions and to create word clouds of their initial thoughts on tackling problem questions as we progressed.

Evaluation

I evaluated reaction to the module using the usual formal and informal module evaluations. I also tracked engagement with the videos and actively used these figures to prompt students if views were lower than expected. I monitored attendance to modules and didn’t notice any drop-off in attendance. Finally, I reviewed end of year results to assess impact on students results.

Impact

Student feedback, about the videos and problem solving, was overwhelmingly positive in both formal and informal module evaluations.

Videos can be of assistance if a student is absent, has a disability or wishes to revisit the material. Sankoff (2014) and Billings-Gagliardi and Mazor (2007) dismiss concerns about reduced student attendance due to online material, and this was borne out by my experience, with no noticeable drop-off in numbers attending lectures; I interpret this as a positive sign of student satisfaction. The videos worked to supplement the live lectures rather than replace them.

There is a clear, positive impact on my own workload and that of my colleagues. Whilst I am no longer teaching on this module, my successor has been able to use my videos again in her teaching, thereby reducing her own workload. I have also been able to re-use some of the videos in other modules.

Reflections

Whilst flipped learning is intensive to plan, create and execute, the ability to re-use the videos in multiple modules is a huge advantage; short videos are simple to re-record if, and when, updating is required.

My initial concern that students would not watch the videos was utterly misplaced. Each video has had in excess of 1200 views (and one video has exceeded 2500). Some of the material was only covered by the flipped learning videos, and still appeared within the examination; students who tackled those questions did equally well as those answering questions covering content which was given via live lecture, but those questions were less popular (2017/18 examination).

I was conscious that there may be some students who would just ignore the videos, thereby missing out on chunks of the syllabus, I tried to mitigate this by running quizzes during lectures on the recorded material, and offering banks of multiple choice questions (MCQs) on Blackboard for students to test their knowledge (aligned to the summative examination which included a multiple choice section). In addition, I clearly signposted the importance of the video recorded material by email, on the Blackboard page and orally and emphasised that it would form part of the final examination and could not be ignored.

My experience echoes that of Schaffzin’s study (2016) monitoring impact, which showed no statistical significance in law results having instituted flipped learning, although she felt that it was a more positive teaching method. Examination results for the module in the end of year summative assessment (100% examination) were broadly consistent with the results in previous academic years, but student satisfaction was higher, with positive feedback about the use of videos and active learning activities.

Follow Up

Since creating the flipped learning videos another colleague has taken over as convenor and continued to use the videos I created. Some of the videos have also been able to be used in other modules.  I have used screencast videos in another non-law module, and also used them as introductory material for a large core Part 1 Law module. Student feedback in module evaluations praised the additional material. One evolution in another module was that when I ran out of time to cover working through a past exam question within a lecture, I created a quick screencast which finished off the topic for students; I felt that it was better to go at a more sensible pace in the lecture and use the screencast rather than rush through the material.

Michelle Johnson, Module Convenor 2018-2019 commented that:

“I have continued to use and expand the flipped learning initiative as part of the module and have incorporated further screencasts into the module in relation to the contract law content delivered. This allowed for additional time on the module to conduct a peer-assessment exercise focussed on increasing the students’ direct familiarity with exam questions and also crucially the marking criteria that would be used to score their Summer exams. Students continue to be very positive about the incorporation of flipped learning material on the module and I feel strongly that it allowed the students to review the more basic introductory content prior to lectures, this allowing time for a deeper engagement with the more challenging aspects of the lectures during lecture time. This seemed to improve students understanding of the topics more broadly, allowing them to revisit material whenever they needed and in a more targeted way than a simple lecture recording.”

TEF

TQ1, LE1, SO3

Links

University of Reading TEL advice about personal capture – https://sites.reading.ac.uk/tel-support/category/learning-capture/personal-capture

Berrett, D. (2012). How “Flipping” the Classroom Can Improve the Traditional Lecture. – https://www.chronicle.com/article/how-flipping-the-classroom/130857. Chronicle of Higher Education..

Billings-Gagliardi, S and Mazor, K. (2007) Student decisions about lecture attendance: do electronic course materials matter?. Academic Medicine: Journal of the Association of American Medical Colleges, 82(10), S73-S76.

Sankoff, P. (2014) Taking the Instruction of Law outside the Lecture Hall: How the Flipped Classroom Can Make Learning More Productive and Enjoyable (for Professors and Students), 51, Alberta Law Review, pp.891-906.

Schaffzin, K. (2016) Learning Outcomes in a Flipped Classroom: A comparison of Civil Procedure II Test Scores between Students in a Traditional Class and a Flipped Class, University of Memphis Law Review, 46, pp. 661.