MOVING TOWARDS E-ASSESSMENT: The Use of Electronic Submission and Grading on the Academic Skills Module – Svetlana Mazhurnaya

By laurenmccann Comment Closed

Profile picture for Svetlana Mazhurnaya

I have been teaching English for over 15 years. I worked on EFL courses in Russia and the UK between 2000 – 2012. I started teaching English for Academic Purposes in 2013 when I joined the International Foundation Programme at the University of Surrey. I have been working as an EAP tutor at the University of Reading since 2014, first on the International Foundation Programme and now on the Pre-sessional English Programme. I have recently become part of the assessment group within ISLI, which creates and administers tests of EAP.

OBJECTIVES

• To familiarise Foundation level students with e-assessment practices as part of their preparation for Undergraduate Courses at UoR
• To simplify assessment administration procedure for multiple module subgroups with varied deadlines on a 20-credit module
• To reduce the marking workload associated with paper submissions
• To deliver more timely and accessible feedback to students

CONTEXT

The International Foundation Programme has a 15-module portfolio delivered by various UoR departments. International Students joining the course have to manage multiple assessment deadlines and follow academic assessment practices used within the departments delivering their core modules. In order to support them, IFP runs a 20-credit Academic Skills module taught over 2 hours per week and assessed through a combination of formative and summative oral and written assignments marked off-line. A combination of word documents, excel spreadsheet and online RISIS reports are used for assessment data administration. When I joined the programme in 2015, the team were looking for ways of:

• optimising the administration of a large volume of paper submissions with multiple sub-group deadlines

• reducing the tutors’ marking workload & simplifying the assessment data entry process

• gauging the level of learners’ engagement with feedback

IMPLEMENTATION

The trial

• Having previously used electronic marking tools, I was keen to introduce them on the IFP. With the Module Convenor’s support, I started trialling the Turnitin e-submission and grading tools with my sub-group in spring 2015. It was agreed that a formative assessment piece would be suitable for the trial to allow space for an error and that learner training could be integrated into the module syllabus as part of developing the students’ referencing and source integration skills. There were 3 classroom demonstrations: how to submit work, how to check originality reports and how to access electronic feedback. Learners were also signposted to the learner training resources available on Blackboard. Some students requested further guidance and were supported through a peer-led demonstrations in subsequent lessons. The fact that most students managed the e-submission with minimal training was an encouraging start.

• For the purposes of maintaining consistency in feedback delivery with other module subgroups I created a QuickMarks set based on the existing module error correction codes that all of the tutors used and hyper-linked them to the online practice materials we normally recommend to students when suggesting areas for improvement. I also uploaded our mark scheme as a Turnitin rubric. Similarly, I provided global feedback comments on submitted work. The only difference in the feedback delivery was its online mode and the fact that QuickMarks were associated with one of the 5 assessment criteria such as “organization” or “task completion”, hopefully making the rationale behind the grading more explicit.

• Students reacted favourably to receiving electronic feedback, saying that they liked having instant access to their grades through “My Grades” feature and that word-processed comments were easier to understand for international students than handwritten ones. They also like the fact that QuickMarks we use are hyperlinked to external practice materials. This allows them to work independently. For example, a comment on referencing issues is linked to the referencing guidelines page.

• Interestingly, the electronic assignment inbox showed that the students’ level of engagement with feedback varied: some viewed the marks but did not access the detailed feedback; others read the comments but did not explore the hyperlinks. This has prompted us to run follow-up tutorials that students have to prepare for using tutor’s feedback. Overall, the trial was largely successful but highlighted the need for some more learner training in how to process e-feedback.

Sharing practice

• Because the online marking procedure used with the trial group was largely replicating our existing off-line marking procedure in a less time-consuming way, other module tutors were keen to experiment with e-assessments. The Programme Director and the Module Convenor were very supportive and allowed me to spend time on one-to-one consultations with team members in order to demonstrate the benefits of using e-assessment tools and train them if they wished to trial them.

Wider implementation

• Over the next couple of terms it was decided to introduce e- submission for all written coursework assignment in order to optimise the administration process. However, tutors were allowed the flexibility of marking online or downloading e- submissions in order to mark them in Word or print papers. This approach met our staff training needs and working styles.
The challenge at this stage was that the e-feedback and grades had to be transferred into the official feedback forms and spreadsheets for consistency purposes. In order to avoid multiple data entry, we decided to start using the Turnitin rubric and the Blackboard Grade centre. Creating a Turnitin rubric was easy and eliminated the need for calculating grades in excel documents and transferring them to a master spreadsheet. We have not moved away from excel documents completely but have significantly reduced the manual data entry load.

• By autumn 2016 all Academic Skills written assignments were submitted and graded online

IMPACT

Effect on the students

• Students find the new submission procedure, with a single submission point and an electronic receipt system, easier to follow.

• Many IFP students have used the opportunity to submit work remotely while visiting their families abroad during holidays.

• Many students are using Turnitin Originality reports as a formative learning tool that helps them see how well they have paraphrased or referenced source material and revise their drafts independently more, which has resulted in fewer cases of unintentional plagiarism.

• There is a greater transparency to learners as to how their mark was arrived at because they can see the number and type of QuickMarks comments that are associated with each criterion their work has been graded on.

• Generally, they now view e-submission and feedback as part of the daily university activities, which prepares them for the reality of the academic studies on their future degree courses.
Effect on the Tutors

• Using e-submission has decreased the burden of assessment administration: instead of sorting large volumes of student papers into sub-groups manually tutors use GradeCentre SmartViews to filter out their students’ submissions.

• Non-submitters are identified and sent a reminder earlier. In the past non-submitters could only be identified after the anonymous marking process has been completed, which often resulted in a hefty penalty. Now a tutor or the module convenor uses the “e-mail non-submitters” button right after the deadline to chase the students (even if marking is anonymous). As a result, students who failed to submit their assignment or uploaded to the wrong submission point receive an early reminder. For many IFP students, it is a learning curve and getting an early reminder helps them.

• Marking has become easier with Turnitin: tutors can manage the 15 days turnaround time better because they can start marking straight after the deadline and not have to wait until the printed copies are distributed. Many find QuickMarks hyperlinked to external practice or reference materials helpful as a way to feed forward without giving a lengthy explanation. Some tutors reported being slowed down by internet connection issues. It also took us some time to adjust to the
need Feedback Studio Interface.

• Using electronic assessment tools has also prompted a professional dialogue about our current assessment practices and highlighted the need for protocols on e-submissions, e- moderation and external examining. So it is great news that such guidelines are being developed as part of the EMA work.

• We have gained a better overview of IFP students’ engagement levels because GradeMark allows us to identify and contact non-submitters at one click. It also shows us the number of submission attempts and whether students have accessed feedback prior to tutorials. This helps us to support at risk students better.

Effects on the Module Convenor

• The module convenor has gained a better real-time overview of the marking process: number of scripts marked so far, marking analytics (average, standard deviation, range), all displayed in the GradeCentre column statistics. This has allowed the module convenor to support the tutors by re-distributing scripts or helping to mark and moderate.

• The module convenor can also see how much feedback is given to students across the board, which is important for quality assurance purposes.

• Dealing with possible cases of academic misconduct and late submissions has become easier thanks to Turnitin originality reports and electronic receipt system.

REFLECTIONS

• Our team’s experience has shown that it is worthwhile trying to integrate electronic assessment literacy into the course syllabus. It would also be great if there were university-wide learner-training sessions, similar to CQSD sessions offered to staff.

• Moving our module toward e-assessment was manageable
because our approach to electronic tools has been selective: where our current assessment practices worked well, we only sought to replicate them. When a change was needed, we looked for ways technology could be used to implement it.

• Sharing best practice and providing peer support has proven to be a good way of encouraging more colleagues to use e- assessment tools, because it was not perceived as a top-down driven change.

• Having the programme management support has really helped our small community of e-practitioners to grow. Creating training opportunities and allowing some flexibility during the transition to e-practice have been key to its success. There was a point when our exploratory e-assessment practices needed to be more standardized and programme level decisions were key to maintaining consistency of practice.

FOLLOW UP

Following the successful trial of the e-assessment tools on the Academic Skills and International English Modules, the programme management is keen to encourage other IFP modules to trial them.

In Spring 2017, a member of the Blackboard Team delivered a Staff Development Session on GradeMark to the IFP team.

We are currently exploring the possibility of doing our internal and external moderation electronically.

Henley Business School staff guide to using Turnitin to aid identifying and dealing with academic misconduct when marking

By laurenmccann Comment Closed

Edith Rigby, Henley Business School
e.rigby@henley.ac.uk

Overview

A review of existing University of Reading assessment advice and two staff workshops at Henley Business School to inform a new marker’s guide on fair marking and managing Academic Misconduct. The marker’s guide will specifically cover what to look out for and when and how to use Turnitin Originality Reports. The guide can also be used at staff development workshops.

Objectives

To produce a staff guide that:

  1. Distinguishes plagiarism, poor academic practice and academic misconduct
  2. Outlines different staff roles in relation to marking and assessment feedback
  3. Provides a training tool for new staff
  4. Promotes consistent marking and feedback practice including when or how to use Turnitin Originality Reports.

Context

Turnitin Originality Reports are used widely to detect potential academic misconduct. While there is some information on how to use Turnitin to help identify academic misconduct, there are no sessions or workshops on best practice specific to Henley Business School. Discrepancies can therefore able to arise across programmes in how similarity reports are used to advise students or inform marking.

Developing a new marker’s guide to best practice within Henley Business School when marking could ensure a more consistent student experience.

Implementation

Two workshops with the School Director of Teaching and Learning, Directors of Studies and Programme Directors were held to first identify current processes and areas of concern around academic misconduct, and specific areas on which guidance was needed. Then the structure and core content for a new marker’s guide were agreed. The core content was to include: definitions of roles of admin teams, module convenors, markers and Directors of Studies; core definitions; processes; and advice on basic and best practice for new markers.

Then a basic but flexible guide with space for users to add more examples and narratives of best practice was developed. This was based on the results of the two workshops and a review of the Henley Good Academic Practice guide and test for students, and University of Reading advice and documentation around academic misconduct.

Core staff were invited to contribute to the guide as a work in progress, and this draft guide was used at staff workshops.

Impact

The basic guide achieved the project objectives.

Different disciplines across Henley Business School have different needs and collating contributions from busy academics has resulted in a guide that is currently best used for workshops only. Once more contributions are forthcoming an online version can be developed as required.

The key impact of this project has been to generate discussion and share practices around the

  • purpose and processes of marking
  • different types of assessment and the assessment literacies required for staff and students
  • handling large group assessment and marking.

Reflections

Being able to identify what new markers need to know about roles and processes has turned out to be essential given the changes to the Academic Misconduct policy over the last two academic years. Academics and programme administrators alike have found this part of the guide more than helpful.

Henley Business School Directors of Studies also found the workshop discussions useful and pertinent to assessment aspects of their roles.

More workshops with targeted academics across an academic year as part of the project plan would have elicited more content.

Overall the work on this project has informed other work on eAssessment and eFeedback at Henley Business School, and will be revisited as part of the Henley review of assessment and feedback.

Follow up

In time more contributions will be sought so that the guide can better illustrate the differences between undergraduate and postgraduate marking requirements. It can then be made more interactive for web self-access or use in workshops.

Links

 Henley Marking Guide to Academic Misconduct and using Turnitin Originality Reports
If you need to edit this, please contact Nicola Langton (nicola.langton@henley.ac.uk).

‘What did I do wrong?’ Researching student referencing practices by Helen Hathaway, Clare Nukui, Dr Kim Shahabudin and Dr Liz Wilding

By edwardwhite Comment Closed

‘What did I do wrong?’ may be a sadly familiar phrase to tutors when notifying students of poor academic practice in their written work. It was chosen for the title of our collaborative TLDF-funded project because it captures the confusion and lack of understanding which is characteristic in student responses to plagiarism and poor academic practice accusations. Such accusations can destroy the confidence and downgrade the final result for otherwise intelligent, committed and hardworking students. Our project aims to uncover some of the reasons why student difficulties with referencing arise, and collate good practice teaching materials for use in subject teaching and for self-development throughout the University.

Difficulties with referencing and plagiarism are an area of concern for all HE institutions in the UK. Problems go beyond knowing how and when to write a citation. Both tutors and students regularly report a failure to understand the purpose of referencing to support a critical discussion. This inability to apply principles of rigorous independent learning goes to the heart of poor academic writing.

There have been increasing efforts on the part of academic departments and central services to provide solutions to this problem through guides, taught sessions, exercises and digital tools like Turnitin. We are all far too busy to keep re-inventing the wheel! So the primary aim of the project is to collate best practice examples of teaching materials and make them available in a format which can be easily adapted for other departments.

We also hope to discover why it is that, even where there is plentiful, comprehensive and highlighted guidance, student difficulties persist. Why are students not using – or not understanding – the guidance available? How can we best persuade them to take referencing at university seriously? We plan to include suggestions with the ‘toolkit’ of teaching materials.

While we are limited by the scope of the current project to focusing close attention on a limited number of departments, we are also planning to briefly survey academics across the university on this topic. In the meantime, we would be very happy to receive any comments, experiences or examples of good practice. Please feel free to contact any member of the project team named above.

The sincerest form of flattery by Nadja Guggi

By edwardwhite Comment Closed

http://www.youtube.com/watch?feature=player_embedded&v=6OjCAq5u-TQ

Our Teaching & Learning Showcase Series continued yesterday with a session on ‘Sharing good practice in the use of Turnitin’. Turnitin is an online service which allows educators to check students’ work for similarity with other sources as a tool for plagiarism prevention and development of academic writing skills.

Turnitin automatically generates an ‘Originality Report’ with a ‘similarity index’ expressed as a percentage, and links to matched sources, including other students’ work, the internet, and other publications. It’s available at Reading through the University’s VLE Blackboard as well as a web portal.

As Associate Dean Orla Kennedy, who has been chairing these informal lunchtime gatherings this term, pointed out, the event coincidentally took place at the same time as a meeting of SCAM (the University’s aptly named Sub-Committee on Academic Misconduct) but still saw a good turnout of some 20 colleagues from academic and service departments across the University.

Speakers Virginie Ruiz (Systems Engineering), Sara Broad (Institute of Education) and Mary Morrissey (English Literature) are among those leading the use of Turnitin in teaching and learning at the University. Each shared their approaches and experiences, highlighting different aspects and issues surrounding the use of Turnitin, before addressing questions and concerns from colleagues.

Continue reading →