Improving student engagement with assessment and feedback through peer review

By edwardwhite Comment Closed

Professor Helen Parish, School of Humanities
h.l.parish@reading.ac.uk

Year of activity: 2014-15

Overview

9070

The project investigated recent research and practice in peer assessment and feedback in order to implement a peer assessment model for use within History, and develop a framework for the adoption of said model in cognate disciplines where evaluation of substantial text-based assignments is an important part of assessment.

Objectives

  • Present students with well-managed opportunities to engage in feedback and assessment and learn from it.
  • Present staff with access to tried and tested models for implementation that can be used and tailored across disciplines.

Context

The importance of increasing the impact of assessment in feedback and learning is recognised by the University’s teaching and learning enhancement priorities, and is evident in the ‘Engage in Assessment’ and ‘Engage in Feedback’ materials.  The requirement to pursue an agenda for feedback is also highlighted by the expectations of employers that graduates of the University of Reading will be able to assess and evaluate the work of others, by comments on feedback made by University of Reading students in the National Student Survey, and by discussions with potential students on Open Days.

Implementation

There were five stages to the project:

  1. A literature search on the topic and detailed engagement with recent scholarship, undertaken by the Principal Investigator.
  2. A ‘competitor analysis’, undertaken by a research assistant, looking at the extent that peer feedback is present on Humanities curricula at other institutions.
  3. Development of a model for the trial of peer assessment informed by the previous two stages.
  4. Implementation of this model as a ‘pilot project’ in the Department of History.
  5. Obtaining student feedback on the process and reflection by the Principal Investigator.

The feedback gained during the early stages of the project revealed that students were reluctant to allow their work to be reviewed by their peers, even when anonymised.   This necessitated the envisaged model to be altered, whereby the written work being ‘peer reviewed’ was either from previous cohorts within the Department or alternative sources.

Once the pilot project was developed, there were three stages:

  1. Development of an understanding of marking and assessment criteria. Students read the assessment criteria of their module, and were then tasked with rewriting these in their own words.
  2. Applying these criteria to written work. Students then read a sample essay (not taken from the group), and with reference to the marking criteria, were asked to give a mark to the essay, with a summary of reasons they had come to this judgment.  This was followed by a discussion of the written feedback provided.
  3. Focus group and project review.  It was intended that students would meet to talk about the project, and more general issues to do with assessment and feedback, in the presence of an experienced observer external to the department.

Impact

One of the principal benefits of the project was that students became more aware of the marking criteria by which their assignments were assessed, as although they found these clear, few students had actually taken the time to read these before. An additional benefit was that the activity helped develop students’ academic confidence, as they were impelled to adopt a critical attitude to writing within scholarship, and gained experience of promoting their point of view to their peers.

Reflections

Feedback from questionnaires suggested that students enjoyed the project; that they now had a better understanding of assessment and feedback; that the project had been helpful with the preparation of their own written work; and that they were now more confident in the assessment of their own work prior to submission.

The reluctance of students to submit their own work to review by their peers meant that there was a less direct link between the peer feedback provided and the specific assignment for each module.  By using work from previous cohorts or alternative sources, however, it was possible to get students to engage more willingly with the process of peer review.

The main disappointment was that it proved impossible to gather a large enough group of students to participate in the focus group stage of the project.  This may have been due to the proposed scheduling of the focus groups at a time when students had recently participated in a Departmental Periodic Review and submitted their final coursework of the academic year.  Nevertheless, valuable feedback on the pilot was provided through questionnaires and verbal communication.

It was interesting to observe that students held broad spectrum of ideas about what constituted good work, arising from a lack of understanding about the criteria against which work is marked. From this perspective, the project was valuable, as students were familiarised with the marking criteria and how these applied to written pieces Students were able to look ‘behind the scenes’ at the marking process, with student applying the marking criteria as individuals, but then needing to decide as a group upon a final mark for pieces they were reviewing.

Follow up

Following the pilot project, the use of peer review to engage students in assessment and feedback has been used by other members of staff within the Department of History, with similar success. Other than the specific pieces of work and criteria used for peer review purposes, there was nothing within this project that was specific to the Department of History or School of Humanities, and so this activity could easily be adapted for use in other Departments and Schools across the University.

The peer review approach has been successfully applied within the Department of History to student presentations in seminars. As student presentations are more ‘in the moment’ and designed with a peer audience in mind, students have not expressed the same reticence to have their peers review their work, and those presenting have appreciated receiving immediate feedback.

‘What did I do wrong?’ Supporting independent learning practices to avoid plagiarism

By edwardwhite Comment Closed

Helen Hathaway, Library; Clare Nukui, International Foundation Programme; Dr Kim Shahabudin, Study Advice; Dr Elisabeth Wilding, International Study and Language Institute
h.m.hathaway@reading.ac.uk
Year of activity: 2012-13

Overview

8905The development of an Academic Integrity Toolkit for academic tutors to draw on, which collated evaluated teaching and support resources for supporting the development of independent learning practices necessary to avoid plagiarism, and offered guidance for adapting and using them in subject teaching.

Objectives

  • To conduct research into current practices and needs for supporting the development of independent learning practices.
  • To develop a toolkit to provide academic tutors with resources for developing independent learning practices necessary to avoid plagiarism.

Context

The fundamental academic principles of independent critical thinking, supported by appropriate and properly cited evidence from evaluated sources, lie at the heart of Higher Education in the UK. A proper understanding of these principles and the independent learning practices needed to achieve them is especially crucial in avoiding unintentional plagiarism. Despite the availability of a range of advice, both internal and external to the University of Reading, students continually stated that they did not know when and how to use citations, or how to avoid unintentional plagiarism. Beyond simply learning the mechanisms of setting out a bibliography or when to include a citation, students need to understand associated practices, such as where to find appropriate sources of information in their subject, how to keep proper records, and how, when, and why to use references in their academic work.

Implementation

Data was collected from a variety of sources on current practices and perceived needs to inform the production of the toolkit. Team members were able to draw upon their professional communities for information about practices at other institutions, and on contacts at the University of Reading for practices and perceived needs. A research officer was appointed, and was tasked with collecting further data from academic tutors and students in a number of selected departments at the University, using semi-structured interviews and focus groups. These were set against the wider context of general observations gathered through separate online surveys offered to all staff and students. Existing pedagogical research into student referencing practices was also considered.

The research questions during this stage of inquiry were:

  • What are the main (perceived and actual) difficulties that students have with understanding referencing and avoiding plagiarism?
  • How do associated independent learning practices impact on this?
  • Why do students fail to engage with current teaching and guidance on referencing?
  • What teaching resources are currently available (at the University of Reading and elsewhere), and how might they be made more effective for teaching staff and students across the University?

The research was used to inform the content and production of the Academic Integrity Toolkit materials. These were generated by team members using a template, before going through an iterative process of revision and evaluation by other team members.  They were then edited by a single team member to ensure consistency.

Impact

The Academic Integrity Toolkit was successfully launched in June 2013 at an event attended by over 50 members of staff, with a visiting speaker presenting on the topic of student referencing practices. The Toolkit comprises:

    • 17 handouts giving guidance on key learning practices;
    • 8 exercise sheets with answers;
    • 13 sets of PowerPoint slides for use in teaching;
    • Links to screencasts produced by members of the Study Advice team;
    • An annotated list of useful websites.

The Academic Integrity Toolkit has been made available through the Blackboard Virtual Learning Environment, onto which staff can self-enrol. A paper version contained in a card folder was created for attendees at the launch event and to disseminate to key members of the teaching and learning community at the University of Reading.

The Toolkit has been a successful and well-used teaching resource. Departments and Schools have requested that their entire teaching staff get enrolled to the Toolkit through Blackboard. It was favourably received by the University Board of Teaching and Learning and the Sub-Committee for the Development and Enhancement of Teaching and Learning, with suggestions being made for its further development through a student-facing version in digital format.

There has been interest, both at the University of Reading and beyond, in the results of the project. Presentations were given at the Librarians’ Information Literacy Annual Conference, the Association for Learning Development in Higher Education conference, and the Association of Librarians and Information Professionals in the Social Sciences summer conference.

Reflections

While the project was successful, it was not without its difficulties. Difficulty was experienced with data collection at the University, particularly in recruiting students for focus groups, which may have been an unavoidable consequence of the necessary timing. Despite this difficulty, however, the data obtained by the focus groups that did run was supplemented by a good response to the wider University-wide survey, and by reported data from academic and support staff through their direct contact with students.

Another difficulty faced was the existing busy workloads of the team members. This was overcome by scheduling brief face-to-face lunchtime meetings once a month, and by setting up a wiki (using PBworks) to allow collation of data and joint working on documents, in addition to regular email communications. Despite the potential for difficulties caused by the project being a collaboration between the Library, International Study and Language Institute and Study Advice, the combined expertise and experience of team members proved particularly valuable, especially as it allowed the project to make use of team members’ involvement with various professional networks, and it was found to be very advantageous to have the different perspectives that were able to be provided by having a diverse team.

Follow up

It is planned that the resources developed through the project be adapted to be an Open Educational Resource, to allow them to be more widely shared for use in UK and global Higher Education teaching.

Guides to citing and avoiding plagiarism available on the Library website have been informed by the results of the project and updated appropriately.

It is hoped that more extensive resources will be developed, and that these will be mediated by members of staff.

As a result of the experience of collaborative working between different areas of the University, Helen Hathaway and Kim Shahabudin have had a chapter entitled ‘Terms of reference: working together to develop student citation practices’ accepted for publication within a forthcoming edited volume.

Links

MOOCs at Reading – what, why and where next?

By edwardwhite Comment Closed

Dr Clare Wright, School of Literature and Languages
c.e.m.wright@reading.ac.uk
Year(s) of activity: 2014-15

Overview

MARThis project funded a small team of researchers and teaching practitioners from the Department of English Language and Applied Linguistics to explore teaching and learning implications of the University of Reading’s pilot Massive Open Online Course (MOOC), A Beginner’s Guide to Writing in English for University Study, delivered on English language academic writing, designed and run by staff at the International Study and Language Institute (ISLI). The project team members focused on design, delivery and mentoring issues arising from the pilot, to be used to improve future MOOCs at Reading.

Objectives

  • Create a mentoring training brief.
  • Complete two research outputs.
  • Host a national workshop to share best practice and set up a community of practice.

Context

As the English language academic writing MOOC was in its initial piloting stage, and was an unusual combination-style MOOC (merging both content knowledge and skills in using knowledge), project team members were able to bring expertise in Applied Linguistics and Academic Writing to evaluate teaching and learning success and identify areas for improvement in future iterations.

Implementation

Project team members conducted interviews with educators and mentors and evaluated data on student evaluations obtained from the MOOC platform team (Future Learn) in order to ensure that a rigorous and thorough evaluation of the pilot MOOC could be conducted. Building upon these findings, a national workshop for over 30 participants drawn from various institutions was held at the University of Reading, where presentations, group discussions and a concluding round-table discussion, considered a number of key issues surrounding MOOCs.

Impact

The data gained from interviews with educators and mentors led to ISLI staff creating a specialised induction training pack for incoming mentors in further iterations of the MOOC, which has been successful in helping new mentors avoid some of the pitfalls and challenges identified by the pilot.
The national workshop was successful in meeting its aims, attracting over 30 participants from the UK and Ireland. Following the workshop, a blog entry for the University of Reading’s Centre for Quality Support and Development (CQSD) Engage in Teaching and Learning blog was written, which was also adapted for an Association for Learning and Teaching (ALT) newsletter highlighting the tips on best practice which emerged from the workshop discussions.

An invitation-based website was also set up for those attending the final project workshop to host the speakers’ slides and space to maintain an ongoing community of practice.

Project team members have contributed a chapter to be published in a forthcoming book on educator and mentor experiences of the MOOC, and a journal article in preparation on student evaluations of the MOOC, for academic dissemination of the project’s research aims.

Reflections

There was excellent teamwork between the three members of staff involved, with clear project aims and timely targeted support for the MOOC staff helping to ensure good buy-in from all stakeholders on the research part of the project. Good connections with the wider MOOC community ensured that the workshop was well planned, with good speakers, and ensured there was a good take up and as wide an impact as had been hoped.

The project did not have any evolving aims; given the success of the impact activities, however, especially the workshop, ongoing take-up within the online Community of Practice would be good, while better knowledge in how to set up and maintain such a network by project members and more time to keep momentum with the wider MOOC team at the University of Reading would further boost ongoing interest and further research impact and activities.

Links

http://blogs.reading.ac.uk/engage-in-teaching-and-learning/2015/06/15/education-online-en-masse-lessons-for-teaching-and-learning-through-moocs-by-clare-wright-clare-furneaux-and-liz-wilding/
Education online en-masse: lessons for teaching and learning through MOOCs

Using technology to find low-tech solutions by Mary Morrissey

By edwardwhite Comment Closed

Like a lot of people, I do not consider myself particularly savvy about technology: when I find that something is useful to me, I learn how to use it. That said, I think we can use learning technologies to come up with ‘low tech’ solutions to our teaching needs. Among the advantage is efficiency in terms of time and money: we already have the kit, and we know how to use it. I offer the following as an example.

It is often difficult to make sure that students are aware of detailed regulations that affect their work but which cannot be summarised or displayed easily. Conventions for writing and referencing are a good example in our department.  Last summer, Pat Ferguson (our Royal Literary Fund fellow whose role in the department is to help student improve their writing skills) observed that we had excellent advice on essay writing, but it was in our large Student Handbook, distributed at the start of the first year. Pat suggested that we make this information available separately.

I thought this was a great idea. I noticed there was other information in the handbook that students need through their degree too: there was information about our marking criteria; there were some very helpful examples that showed the difference between plagiarism and poor academic practice. I took these sections, and I created three separate documents with titles that I hoped would be self-explanatory: ‘Style Guide for English Literature students’; ‘Understanding Feedback – Marking Criteria’; and ‘Plagiarism’.

I uploaded all three documents to Blackboard’s ‘Fileshare’ area for the department, and I created links from the Blackboard courses for all our Part 1 and Part 2 modules. (I am working on the Part 3 modules, but there are over 50 of those!) I also posted the documents in our central ‘Information for English Literature Students’ Blackboard organisation, on which all staff, undergraduates and postgraduate students are enrolled. By keeping the documents in ‘Fileshare’ I can update them every year, to include new ‘standard paragraphs’ for example. I overwrite the old file with the newer version, and all the daughter versions linked to it update automatically.

This isn’t rocket science, but I think it has helped us make useful information more readily available. Having in posted in most of our Blackboard courses makes it more visible; having three small documents (in pdf format) makes them easier to download and print.

Where would I go from here? Students have told me that they like a website with exercises that help with grammar and writing skills that we recommended. It’s based in the University of Bristol:  http://www.bristol.ac.uk/arts/exercises/grammar/grammar_tutorial/index.htm

I would like to create an interactive resource like this, and I know it can be done. The University of Aberdeen took the paper-based ‘Guide to Written Work’ (on which we all relied when I worked there!) and turned it into an internet-based resource with exercises: http://www.abdn.ac.uk/writing/

If anyone knows any low-tech ways that I could do something similar, please let me know!