Online peer assessment of group work tools: yes, but which one? By Heike Bruton (a TLDF project)

By edwardwhite Comment Closed

A short while ago I wrote the post “Group work: sure, but what about assessment? This outlines a TLDF- funded project in which Cathy Hughes and I investigated tools for the peer assessment of group work. Cathy and I have now produced a full report, which is available for download here (Cathy Hughes and Heike Bruton TLDF peer assessment report 2014 07 02), and summarised below.

 

Aim and methods

The aim of the project was to evaluate available online systems for the assessment of students’ contribution to group work. In order to establish our criteria for evaluation of these systems, we conducted a series of interviews with academics across the university. This allowed us an understanding of how peer assessment (PA) is used in a range of subjects, and what the different perspectives on the requirements for a computer-based system are.

 

Systems in use and evaluation criteria

Among our eleven interviewees we found five different separate PA systems (including Cathy’s own system) in use by six departments. Notably, Cathy’s tool appeared to be the only entirely computer-based system. Based on the insights gained from the interviews, we developed a set of criteria against which we evaluated available PA systems. These criteria are pedagogy, flexibility, control, ease of use, incorporation of evidence, technical integration and support, and security.

 

Available online systems

We identified three online tools not in use at the university at the moment, which implement PA specifically to the process, not the product, of group work. These three systems are iPeer, SPARKplus and WebPA. In addition we also critically assessed Cathy’s own system, which is already being used in several departments across the university. After investigating PA systems currently in use at Reading and applying the above-named criteria to the four PA system under investigation, we came to a number of conclusions, which resulted in a recommendation.

 

Conclusion

There is a strong sense of commitment among staff to using group work in teaching and learning across the university. PA can serve as a mechanism to recognise hard work by students and also to provide feedback aimed at encouraging students’ to improve their involvement with group work. Whilst any PA system is simply a tool, which can never replace the need for active engagement by academics in their group work projects, such a tool can make PA more effective and manageable, especially for large groups.

 

Recommendation

Our recommendation then is that WebPA should be considered for use within the university. Our research suggests that it could be adopted with relative ease, particularly given the strong and active community surrounding this open-source software.   While it may not be appropriate for everyone, we believe it could be a useful tool to enhance teaching and learning, potentially improving the experience of group work assessment for both staff and students.

Cathy and I will be delivering a number of Teaching and Learning seminars on PA of group work in the near future. To download the full report, click here (Cathy Hughes and Heike Bruton TLDF peer assessment report 2014 07 02). To try out a stand-alone demo version of WebPA, follow this link: http://webpaos.lboro.ac.uk/login.php

Cathy and Heike will be presenting their project in a TEL Showcase event in the spring term. Please check http://www.reading.ac.uk/cqsd/TandLEvents/cqsd-ComingSoon.aspx.

Group work: sure, but what about assessment? By Heike Bruton (a TLDF project)

By edwardwhite Comment Closed

Group work has many well-documented benefits for students, but it also provides considerable challenges. A frequent complaint from students is that differences in contributions are not recognised when everyone in the group receives the same mark – the free loader issue. However, when students are working unsupervised, it is very difficult for the tutor to gauge who contributed to what extent. This is where peer assessment of group work can be a key part of the assessment framework.

What’s this project all about?
Cathy Hughes from Real Estate & Planning has developed and implemented her own online system of peer assessment of group work, and has given presentations about it at various T&L events. With the help of an award from the Teaching and Learning Development Fund, Cathy appointed me as Research Assistant. Our hope is to find a sustainable system for those colleagues who wish to use it. This may mean developing Cathy’s system further, or possibly adopting a different system.

What peer assessment systems are staff currently using?
The first step of the project was to find out what peer assessment (PA) of group work tutors at the University of Reading are currently using. We conducted a number of interviews with colleagues who are currently using such systems, and we found a variety of systems in use (both paper-based and digital).  Most systems seem to work well in increasing student satisfaction through the perception of fairer marking, and encourage reflection. However, all such systems require quite a lot of effort by those administering them. While lecturers are unanimous in their estimation that peer assessment of group should be done for pedagogic reasons, unsurprisingly they also say that a less labour-intensive system than they are currently using would be highly desirable.

What peer assessment systems are out there?
Cathy and I investigated available peer assessment systems. After examining several digital tools, we identified one system which seems to tick all the boxes on the wish list for peer assessment of group work. This system is called WebPA. WebPA is an open source online peer assessment system which measures contribution to group work. It can be used via Blackboard and seems to be very flexible.

Where to go from here?
You can try out a stand-alone demo version here: http://webpaos.lboro.ac.uk/login.php. This site also contains links leading to further information about WebPA. We are currently putting our findings together in a report, and we will disseminate the results throughout the University.