Using Quickmarks to enhance essay feedback in the department of English Literature – Dr Mary Morrissey

By laurenmccann Comment Closed

Within the department, I teach primarily in Early Modern and Old English. For more details of my teaching please see Mary Morrissey Teaching and Convening

My primary research subject is Reformation literature, particularly from London. I am particularly interested in Paul’s Cross, the most important public pulpit in sixteenth and seventeenth-century England. I retain an interested in early modern women writers, with a particular focus on women writers’ use of theological arguments. Further details of my research activities can be found at Mary Morrissey Research

OBJECTIVES

A number of modules within the Department of English Literature began using GradeMark as a new marking tool in the Autumn of 2015. I wanted to explore the use of the new QuickMarks function as a way of enhancing the quality of the feedback provided to our students and ensuring the ‘feedback loop’ from general advice on essay writing to the feedback on particular pieces of assessed work was completed.

CONTEXT

The Department developed extensive guidance on writing skills to support student assessment: this includes advice on structuring an argument as well as guidance on grammar and citations. This guide was housed on departmental handbooks and in the assignments folder in Blackboard. There was considerable concern that this resource was underused by students. We did know that the QuickMarks function was being used as part of our online feedback provision and that it was possible to personalise the comments we were using and to add links to those comments as a way of providing additional explanation to students.

IMPLEMENTATION

In order to allow relevant sections of the essay writing style guide to be accessed via QuickMarks I copied the document into a Google Doc, divided each section by using Google Doc bookmarks and assigned each bookmark an individual URL link. I then used Bitly.com to shorten the URL link assigned to each section by the Google Doc to make it more useable. I then created a set of Quickmarks that included these links to the Style Guide. In this way, students had direct access to the relevant section of the Guide while reading their feedback. So if a student hadn’t adopted the correct referencing format (the Modern Humanities Research Association style in the case of English Literature) the marker would pull a QuickMark across to the relevant point of the essay. When the student hovered over this comment bubble, they would see the text within it but were also able to click on the URL taking them directly to page 7 of the departmental writing style guide on MHRA citation and referencing. If other colleagues wanted to start adopting the same approach, I simply exported the QuickMark set to them which they incorporated into their own QuickMarks bank within seconds.

IMPACT

The Bitly.com tool, used to shorten the URL link, monitored the usage of each link included in our QuickMarks. This showed us how many times and on which date each individual link was used.

To complement this data I also ran a survey on the student response to online marking and feedback. 35 undergraduate students responded. This showed that students found feedback most useful when it came in forms that were familiar from paper marking, like general comments on the essay and marginal comments throughout the essay. Less familiar types of feedback (links to web-resources included in bubble comments accessed by hovering the cursor) were often missed. In the survey, 28 out of 35 students said that they did not receive any links to the writing style guide within their QuickMark comments even though more than this did receive them. 3 students did not click on the links. Of the 5 remaining students who did make use of the links, 3 responded positively, mentioning their value in terms of improving their writing skills:

“It was good to refer to alongside my work”
“They helped me to strengthen my writing overall”
“Yes motivational to actually look at them-whereas on a paper copy you might read he comment and forget but here you can click straight through so much easier!”

REFLECTIONS

Some of the new functions available to us on GradeMark allow us to improve our feedback. We shouldn’t just be using online marking tools to replicate existing off line marking processes. We can go much further! But if this is going to be successful it is really important to inform students about the range of options that online marking makes available so that they make the most of the systems we use.

Once we do this effectively, we can then explore other options. In English Literature, we are keen to ensure that our Department style guide is used effectively. But there are many other web resources to which we could link through Quickmarks: screencast essay writing guides in Politics and IWLP, as well as the new Academic Integrity toolkit by Study Advice, for example.

By including links within QuickMark comments we help to move students towards greater levels of assessment literacy.

LINKS

Academic Integrity Toolkit

http://libguides.reading.ac.uk/academicintegrity

Examples of assessment support screencasts created by colleagues

Screencast bank

Study Support Screencast Suite

https://www.reading.ac.uk/library/study-advice/guides/lib-sa- videos.aspx

Bitly URL shortener and link management platform

https://bitly.com/

MOVING TOWARDS E-ASSESSMENT: The Use of Electronic Submission and Grading on the Academic Skills Module – Svetlana Mazhurnaya

By laurenmccann Comment Closed

Profile picture for Svetlana Mazhurnaya

I have been teaching English for over 15 years. I worked on EFL courses in Russia and the UK between 2000 – 2012. I started teaching English for Academic Purposes in 2013 when I joined the International Foundation Programme at the University of Surrey. I have been working as an EAP tutor at the University of Reading since 2014, first on the International Foundation Programme and now on the Pre-sessional English Programme. I have recently become part of the assessment group within ISLI, which creates and administers tests of EAP.

OBJECTIVES

• To familiarise Foundation level students with e-assessment practices as part of their preparation for Undergraduate Courses at UoR
• To simplify assessment administration procedure for multiple module subgroups with varied deadlines on a 20-credit module
• To reduce the marking workload associated with paper submissions
• To deliver more timely and accessible feedback to students

CONTEXT

The International Foundation Programme has a 15-module portfolio delivered by various UoR departments. International Students joining the course have to manage multiple assessment deadlines and follow academic assessment practices used within the departments delivering their core modules. In order to support them, IFP runs a 20-credit Academic Skills module taught over 2 hours per week and assessed through a combination of formative and summative oral and written assignments marked off-line. A combination of word documents, excel spreadsheet and online RISIS reports are used for assessment data administration. When I joined the programme in 2015, the team were looking for ways of:

• optimising the administration of a large volume of paper submissions with multiple sub-group deadlines

• reducing the tutors’ marking workload & simplifying the assessment data entry process

• gauging the level of learners’ engagement with feedback

IMPLEMENTATION

The trial

• Having previously used electronic marking tools, I was keen to introduce them on the IFP. With the Module Convenor’s support, I started trialling the Turnitin e-submission and grading tools with my sub-group in spring 2015. It was agreed that a formative assessment piece would be suitable for the trial to allow space for an error and that learner training could be integrated into the module syllabus as part of developing the students’ referencing and source integration skills. There were 3 classroom demonstrations: how to submit work, how to check originality reports and how to access electronic feedback. Learners were also signposted to the learner training resources available on Blackboard. Some students requested further guidance and were supported through a peer-led demonstrations in subsequent lessons. The fact that most students managed the e-submission with minimal training was an encouraging start.

• For the purposes of maintaining consistency in feedback delivery with other module subgroups I created a QuickMarks set based on the existing module error correction codes that all of the tutors used and hyper-linked them to the online practice materials we normally recommend to students when suggesting areas for improvement. I also uploaded our mark scheme as a Turnitin rubric. Similarly, I provided global feedback comments on submitted work. The only difference in the feedback delivery was its online mode and the fact that QuickMarks were associated with one of the 5 assessment criteria such as “organization” or “task completion”, hopefully making the rationale behind the grading more explicit.

• Students reacted favourably to receiving electronic feedback, saying that they liked having instant access to their grades through “My Grades” feature and that word-processed comments were easier to understand for international students than handwritten ones. They also like the fact that QuickMarks we use are hyperlinked to external practice materials. This allows them to work independently. For example, a comment on referencing issues is linked to the referencing guidelines page.

• Interestingly, the electronic assignment inbox showed that the students’ level of engagement with feedback varied: some viewed the marks but did not access the detailed feedback; others read the comments but did not explore the hyperlinks. This has prompted us to run follow-up tutorials that students have to prepare for using tutor’s feedback. Overall, the trial was largely successful but highlighted the need for some more learner training in how to process e-feedback.

Sharing practice

• Because the online marking procedure used with the trial group was largely replicating our existing off-line marking procedure in a less time-consuming way, other module tutors were keen to experiment with e-assessments. The Programme Director and the Module Convenor were very supportive and allowed me to spend time on one-to-one consultations with team members in order to demonstrate the benefits of using e-assessment tools and train them if they wished to trial them.

Wider implementation

• Over the next couple of terms it was decided to introduce e- submission for all written coursework assignment in order to optimise the administration process. However, tutors were allowed the flexibility of marking online or downloading e- submissions in order to mark them in Word or print papers. This approach met our staff training needs and working styles.
The challenge at this stage was that the e-feedback and grades had to be transferred into the official feedback forms and spreadsheets for consistency purposes. In order to avoid multiple data entry, we decided to start using the Turnitin rubric and the Blackboard Grade centre. Creating a Turnitin rubric was easy and eliminated the need for calculating grades in excel documents and transferring them to a master spreadsheet. We have not moved away from excel documents completely but have significantly reduced the manual data entry load.

• By autumn 2016 all Academic Skills written assignments were submitted and graded online

IMPACT

Effect on the students

• Students find the new submission procedure, with a single submission point and an electronic receipt system, easier to follow.

• Many IFP students have used the opportunity to submit work remotely while visiting their families abroad during holidays.

• Many students are using Turnitin Originality reports as a formative learning tool that helps them see how well they have paraphrased or referenced source material and revise their drafts independently more, which has resulted in fewer cases of unintentional plagiarism.

• There is a greater transparency to learners as to how their mark was arrived at because they can see the number and type of QuickMarks comments that are associated with each criterion their work has been graded on.

• Generally, they now view e-submission and feedback as part of the daily university activities, which prepares them for the reality of the academic studies on their future degree courses.
Effect on the Tutors

• Using e-submission has decreased the burden of assessment administration: instead of sorting large volumes of student papers into sub-groups manually tutors use GradeCentre SmartViews to filter out their students’ submissions.

• Non-submitters are identified and sent a reminder earlier. In the past non-submitters could only be identified after the anonymous marking process has been completed, which often resulted in a hefty penalty. Now a tutor or the module convenor uses the “e-mail non-submitters” button right after the deadline to chase the students (even if marking is anonymous). As a result, students who failed to submit their assignment or uploaded to the wrong submission point receive an early reminder. For many IFP students, it is a learning curve and getting an early reminder helps them.

• Marking has become easier with Turnitin: tutors can manage the 15 days turnaround time better because they can start marking straight after the deadline and not have to wait until the printed copies are distributed. Many find QuickMarks hyperlinked to external practice or reference materials helpful as a way to feed forward without giving a lengthy explanation. Some tutors reported being slowed down by internet connection issues. It also took us some time to adjust to the
need Feedback Studio Interface.

• Using electronic assessment tools has also prompted a professional dialogue about our current assessment practices and highlighted the need for protocols on e-submissions, e- moderation and external examining. So it is great news that such guidelines are being developed as part of the EMA work.

• We have gained a better overview of IFP students’ engagement levels because GradeMark allows us to identify and contact non-submitters at one click. It also shows us the number of submission attempts and whether students have accessed feedback prior to tutorials. This helps us to support at risk students better.

Effects on the Module Convenor

• The module convenor has gained a better real-time overview of the marking process: number of scripts marked so far, marking analytics (average, standard deviation, range), all displayed in the GradeCentre column statistics. This has allowed the module convenor to support the tutors by re-distributing scripts or helping to mark and moderate.

• The module convenor can also see how much feedback is given to students across the board, which is important for quality assurance purposes.

• Dealing with possible cases of academic misconduct and late submissions has become easier thanks to Turnitin originality reports and electronic receipt system.

REFLECTIONS

• Our team’s experience has shown that it is worthwhile trying to integrate electronic assessment literacy into the course syllabus. It would also be great if there were university-wide learner-training sessions, similar to CQSD sessions offered to staff.

• Moving our module toward e-assessment was manageable
because our approach to electronic tools has been selective: where our current assessment practices worked well, we only sought to replicate them. When a change was needed, we looked for ways technology could be used to implement it.

• Sharing best practice and providing peer support has proven to be a good way of encouraging more colleagues to use e- assessment tools, because it was not perceived as a top-down driven change.

• Having the programme management support has really helped our small community of e-practitioners to grow. Creating training opportunities and allowing some flexibility during the transition to e-practice have been key to its success. There was a point when our exploratory e-assessment practices needed to be more standardized and programme level decisions were key to maintaining consistency of practice.

FOLLOW UP

Following the successful trial of the e-assessment tools on the Academic Skills and International English Modules, the programme management is keen to encourage other IFP modules to trial them.

In Spring 2017, a member of the Blackboard Team delivered a Staff Development Session on GradeMark to the IFP team.

We are currently exploring the possibility of doing our internal and external moderation electronically.

ELECTRONIC FEEDBACK AND GRADING METHODS – Dr Geoff Taggart

By laurenmccann Comment Closed

Profile picture for Dr Taggart

Dr Geoff Taggart is a lecturer in the Institute of Education and Programme Director for the Early Years Practice programme at Reading. As part of his secondment to the EMA programme, Geoff decided to run a focus group with students from the IoE to gather perspectives on electronic feedback and grading methods.

OBJECTIVES

To identify student views on:

• The perceived benefits of the three forms of most commonly- used feedback offered by Grademark (i.e. Quickmarks, rubrics and text comments)

• Preferences regarding the emphasis which each form of feedback should be given in a typical piece of work

• Views regarding the interrelationship of the different forms of feedback

CONTEXT

The focus group was composed of 4 MA students (2 international and 2 home), plus one Chinese academic visitor with recent experience of being a student. Their views were therefore representative of students engaged in social science disciplines and may not be transferable to other fields. Also in attendance were myself, Dr Maria Kambouri (engagement in feedback project) and Jack Lambert-Taylor (EMA). It took place at London Road campus between 5 and 6.30pm on Thurs 18th January.

IMPLEMENTATION

I provided participants with three copies of the same assignment, one marked exclusively with Quickmarks, one marked only with the final text comment and one marked solely according to the rubric. The purpose of this was to isolate and focus attention upon each of the three kinds of electronic feedback provided through the Feedback Studio.

The marking was not meant to be typical (nor as examples of best practice) but to highlight the positive and negative qualities of each kind of feedback. For example, there were a lot more quickmark comments appended to the assignment than would usually occur. The purpose of this was to emphasise both the positive benefits of maximised contextualised feedback and the negative impression of ‘overload’ which the comments could give. Additionally, the text comments amounted to over 2500 words and were extremely conversational and wide-ranging.

In a similar way, whilst this strategy deliberately emphasised the dialogical and personal nature of this feedback method, it was also not easy to straightforwardly pick out those points where the student needed to improve. By contrast, the rubric does this very clearly but is not a personal way of providing feedback.

REFLECTIONS

Quickmark feedback

• Students appreciated Quickmarks which contained hyperlinks (e.g. to Study Advice)

• One participant noted that they didn’t like the Quickmarks, on the basis that when printed the document does not have interactive links. The same participant suggested that excessive Quickmarks may be intrusive, and give the impression of ‘massacring’ a student’s work. They agreed that less excessive use would be preferable. The same participant noted that there was ‘no positive’ or ‘constructive’ feedback on the page- only problem points. This may be due to the nature of the sample work, which was deliberately of a poor standard; perhaps the same study should be conducted with a high quality piece of work.

• Another participant noted that narrative summaries can come across as more personal, particularly if negative, and that they preferred Quickmarks on the basis that they provided a more objective tone. Another participant suggested that Quickmarks may come across as more ‘humane’ on that basis, rather than a ‘rant at the end’.

• Another participant suggested that Quickmarks provide good evidence of the thoroughness of the marking process.

• One participant suggested that Quickmarks could indicate to which assessment criteria in the rubric it refers. The facility to do this was explained

• It was noted that Quickmarks should be written passively rather that directed at the author, as it can appear more accusatory. For example, ‘The point is not clear here’ as opposed to ‘you have not been clear here’.

Summary – Quickmarks should be limited in their use, include positive as well as negative comments, include relevant hyperlinks and be focussed on the assignment rather than the student and associated with rubric criteria where possible.

Text comments

• Two participants suggested that narrative summary can provide more detailed feedback and valued the conversational tone. It was also suggested that Quickmarks may be perceived as momentary thoughts without reflection, whilst narrative summary may come later after further thought.

• One participant noted that when you write an essay you aren’t ‘just trying to tick boxes in a rubric, you are trying to say something’. This was a really interesting point which emphasised the student expectation of a personal, dialogical relationship with their tutor (something which rich text comments support).

• Several participants noted that marking with more narrative summary would be more time-consuming, and expressed empathy for academics doing so.

• It was also noted that narrative summary would be better-fitted to a conversation in person, and that subtleties within the feedback would be better expressed through intonation in the voice and facial expressions of the marker. Absent those features, it can come across as very serious, and lacks intricacy.

• Students commented that this kind of feedback can also become too ‘waffly’ and lack focus.

Summary – This kind of feedback gives the strongest impression that the tutor has considered the assignment overall, mulled it over and arrived at a holistic impression, something that was highly valued (contrast with: ‘a marked rubric alone shows that the tutor perhaps didn’t think about it that much’). However, the writing needs to be clearly focussed on specific ways in which the student can improve (i.e. bullet points).

Rubric

• Students commented positively that the rubric showed very clearly how successful an assignment had been in general terms. However, they were concerned that it does not explain how to improve if you have not done very well.

• Students questioned how the final mark is actually calculated through the use of a qualitative rubric where the different elements are unweighted – this was considered to lack full transparency.

• It was unanimously agreed that a rubric without comments was not a preferable form of feedback on its own due to lacking feed-forward information, despite the fact that the adjacent rubric statements (i.e. in the next grade band up) also appear to students in the feedback.

• Students did not like the way in which the rubric statements were represented in a consecutive list (see below) when printed off. They much preferred the grid they were used to (i.e. with grade boundaries as the columns and rubric criteria as the rows).

Summary – a rubric is useful in showing how successful an assignment has been in a broad and general sense. The only way in which it could be more useful would be if the rubric were more specific to this particular assignment (and so have multiple rubrics across programmes/the School)

CONCLUSIONS

1. All forms of feedback, taken together, were considered to be useful.

2. The three different forms of feedback need to support each other (e.g. the rubric needs to reflect the written comments, tutors could use the same language in their text comments as that used in the rubric statements)

3. No matter the means by which feedback is given, students want to feel as though their work has made an impression on their tutor.

4. If tutors want to mark mostly through Quickmarks and rubrics (and provide greatly reduced written comments), this may be perceived negatively by students who expect a more personalised response.

FOLLOW UP

The following points may require consultation from Blackboard:

• One participant suggested that different colours may be used to indicate whether quickmark feedback is positive or negative.

• A tutor suggested that it would be helpful if tutors could have flexibility about where to position their Quickmarks in their set, otherwise they just appear rather randomly. This is an issue when marking at speed. )

• All participants suggested that they like the use of ticks in marking, but no alternative was suggested. Can a tick symbol be included in the quickmark set?

• Tutors are able to expand the rubric when marking. Can it be presented to students in this format?

LINKS

Quickmarks:

https://guides.turnitin.com/01_Manuals_and_Guides/Instructor_Guides/Turnitin_Classic_(Deprecated)/25_GradeMark/QuickMark

Rubrics:

https://guides.turnitin.com/01_Manuals_and_Guides/Instructor_Guides/Turnitin_Classic_(Deprecated)/25_GradeMark/Rubrics_and_Grading_F orms

Text comments:

https://guides.turnitin.com/01_Manuals_and_Guides/Instructor_Guides/Feedback_Studio/Commenting_Tools/Text_summary_comments

Using rubrics to transform the marking and feedback experience – Professor Will Hughes

By laurenmccann Comment Closed

Professor Will Hughes has extensively used rubric grids within Grademark across of all of his modules to significantly enhance student engagement with his feedback, student understanding of his marking criteria and student attainment in subsequent essays whilst making his own experience of marking more efficient.

Profile picture for Prof. Hughes

My research interests include the control and management of building contracts, the management of design in construction, the analysis of organizational structure, and the analysis of procurement systems. The focus of my work is the commercial processes of structuring, negotiating, recording and enforcing business deals in construction. I have developed a framework for modelling and describing the myriad permutations of procurement variables, to aid in analysis and understanding of the complexities of organizing the procurement of built facilities. This has been incorporated into a British Standard (2011) on construction procurement.

OBJECTIVES

As convenor of a range of modules typically enrolling 120 students submitting around 3 pieces of work each year, I wanted to ensure that I had a really effective approach to marking and the provision of feedback. I wanted all of my students to engage fully in the feedback I provided, to thoroughly understand exactly what they had done well and where they could be making improvements after reading each piece that I provided. But I needed to achieve all of this in an effective and efficient way.

CONTEXT

National Student Survey results suggest that a significant number of students do not feel that they have access to marking criteria prior to submission and do not understand how to improve their performance based on the comments provided. Often the provision of more and
more free text feedback doesn’t appear to feed into higher levels of student attainment and satisfaction. At the same time, increasing student numbers and broader workload demands have increased pressures on all lecturers across the sector. In response I decided to adopt the use of rubric grids as one way to start to address these key issues.

IMPLEMENTATION

In 2015 I created a rubric grid in which I listed criteria along the left hand side and then unpicked what performance levels against each of those criteria might look like, describing different levels of performance in the lower 10 or 20 range all the way up to outstanding performance in the 90 or 100 range. It was extremely interesting to attempt a clear articulation of the differences between grades of failure and grades of excellence. Explaining, for example, the difference between 90 and 100% for a specific criterion is not something I had ever done before. A screenshot of a typical grid is shown below.

I actually created a slightly different grid for each piece of assessment but it would be equally possible to create a slightly less assessment specific grid that could be used across a whole module or even a whole programme.

Crucially, I shared the criteria for assessment with my students in the assignment brief so they knew, well ahead of submission, what the marking criteria themselves looked like.

I created all of this content in a standard Excel spreadsheet first and then clicked on the ‘rubric manager’ button and then ‘import’ to transfer my grid into Grademark. I could have created it directly within Grademark, in an incredibly simple process, by clicking on the ‘rubric’ icon, ‘rubric manager’, ‘rubric list’ and then ‘create new rubric’. I could then populate my grid with specific criteria and scales. By attaching the rubric grid to one assignment, Grademark automatically attaches the grid to all the assignments within the submission point.

This meant that each time I opened a piece of work in Grademark, I could click on the rubric icon to display the grid. I could then simply click the box against each criteria that applied to the particular assessment I was marking to show the student how they had performed in that particular skill.

In addition to using the rubric grid to classify student performance against individual marking criteria I would also provide in text comments and general comments in the free text feedback box to ensure really tailored and specific feedback content was also provided to all of my students. As I have become more experienced, I have tried to stop myself from adding in-text comments as it tends to result in detailed editing comments, which are not as helpful as feedback.

IMPACT

From the first time I used this approach, students have been enthusiastic. They have emailed my personally, as well as commenting in module evaluation forms, that they found the feedback more useful than anything they has received in their education to date. I no longer have students complaining that their mark is too low and asking whether I have made a mistake. Rather, those who would have complained begin by acknowledging that the mark is clear and well- justified and that they would like to discuss how to improve. This positive approach from students is refreshing.

REFLECTIONS

One of the things that made this activity successful was the prior development of a feedback library, which provided a wide-ranging list of comments to draw from and summarise. Another has been the move towards making comments positive rather than negative. It can be very difficult to focus on what students have done well in a poor submission. But it has proved to be the single most valuable thing. The performance of weak students improves significantly when they are given encouragement rather than discouragement. And strong students appreciate being given indications about how they could improve, as well, which, they tell, me is rare but welcome. I still have a way to go in making all of the comments positive and encouraging. If I were starting over, I would begin by spending time on thinking seriously about how to sound encouraging and positive when students submit very low-quality work. One thing to be careful about is that once the rubric has been attached to an assignment, it cannot be edited without being detached and losing all the grading. At first, I copied every mark into an Excel spreadsheet in case there were errors or omissions in my rubric that I hadn’t noticed until using it.

FOLLOW UP

Every piece of work I set up in Turnitin gives me the opportunity to fine tune the approach. Each piece of work has its own criteria for assessment, so I tend to develop the rubrics in Excel, making them easier to adapt for the next piece of work. This also makes it easy to share with colleagues. If anyone would like further examples, I would be happy to share more recent ones as Excel files.

LINKS

Simple illustrated instructions to create similar qualitative rubric within Grademark, as well as standard or custom rubric and free response grading forms, can be found here:

https://guides.turnitin.com/01_Manuals_and_Guides/Instructor_Guides/Turnitin_Classic_(Deprecated)/25_GradeMark/Rubrics_and_Grading_F orms

A six minute Grademark video demonstrating the creation of a similar rubric and other key features is available here:

https://www.youtube.com/watch?v=BAG44Fpm55o

Connecting with the Curriculum Framework: Using focus groups to diversify assessment (Part 2)

By laurenmccann Comment Closed

Dr Madeleine Davies and Michael Lyons, School of Literature and Languages

Overview

The Department of English Literature (DEL) has run two student focus groups and two whole-cohort surveys as part of our Teaching and Learning Development Fund‘Diversifying Assessments’ project. This is the second of two T&L Exchange entries on this topic. Click here for the first entry which outlines how the feedback received from students indicates that their module selection is informed by the assessment models that are used by individual modules. Underpinning these decisions is an attempt to avoid the ‘stress and anxiety’ that students connect with exams. The surprise of this second round of focus groups and surveys is the extent to which this appears to dominate students’ teaching and learning choices.

Objectives

  • The focus groups and surveys are used to gain feedback from DEL students about possible alternative forms of summative assessment to our standard assessed essay + exam model. This connects with the Curriculum Framework in its emphasis on Programme Review and also with the aims of the Assessment Project.
  • These forms of conversations are designed to discover student views on the problems with existing assessment patterns and methods, as well as their reasons for preferring alternatives to them.
  • The conversations are also being used to explore the extent to which electronic methods of assessment can address identified assessment problems.

Context

Having used focus groups and surveys to provide initial qualitative data on our assessment practices, we noticed a widespread preference for alternatives to traditional exams (particularly the Learning Journal), and decided to investigate the reasons for this further. The second focus group and subsequent survey sought to identify why the Learning Journal in particular is so favoured by students, and we were keen to explore whether teaching and learning aims were perceived by students to be better achieved via this method than by the traditional exam. We also took the opportunity to ask students what they value most in feedback: the first focus group and survey had touched on this but we decided this time to give students the opportunity to select four elements of feedback which they could rank in order or priority. This produced more nuanced data.

Implementation

  • A second focus group was convened to gather more detailed views on the negative attitudes towards exams, and to debate alternatives to this traditional assessment method.
  • A series of questions was asked to generate data and dialogue.
  • A Survey Monkey was circulated to all DEL students with the same series of questions as those used for the focus group in order to determine whether the focus group’s responses were representative of the wider cohort.
  •  The Survey Monkey results are presented below. The numbers refer to student responses to a category (eg. graphic 1, 50 students selected option (b). Graphic 2 and graphic 5 allowed students to rank their responses in order or priority.

Results

  • Whilst only 17% in the focus group preferred to keep to the traditional exam + assessed essay method, the survey found the aversion to exams to be more prominent. 88% of students preferred the Learning Journal over the exam, and 88% cited the likelihood of reducing stress and anxiety as a reason for this preference.
  • Furthermore, none of the survey respondents wanted to retain the traditional exam + assessed essay method, and 52% were in favour of a three-way split between types of assessment; this reflects a desire for significant diversity in assessment methods.
  • We find it helpful to know precisely what students want in terms of feedback: ‘a clear indication of errors and potential solutions’ was the overwhelming response. ‘Feedback that intersects with the Module Rubric’ was the second highest scorer (presumably a connection between the two was identified by students).
  • The students in the focus group mentioned a desire to choose assessment methods within modules on an individual basis. This may be one issue in which student choice and pedagogy may not be entirely compatible (see below).
  • Assessed Essay method: the results seem to indicate that replacing an exam with a second assessed essay is favoured across the Programme rather than being pinned to one Part.

Reflections

The results in the ‘Feedback’ sections are valuable for DEL: they indicate that clarity, diagnosis, and solutions-focused comments are key. In addressing our feedback conventions and practices, this input will help us to reflect on what we are doing when we give students feedback on their work.

The results of the focus group and of the subsequent survey do, however, raise some concerns about the potential conflict between ‘student choice’ and pedagogical practice. Students indicate that they not only want to avoid exams because of ‘stress’, but that they would also like to be able to select assessment methods within modules. This poses problems because marks are in part produced ‘against’ the rest of the batch: if the ‘base-line’ is removed by allowing students to choose assessment models, we would lack one of the main indicators of level.

In addition, the aims of some modules are best measured using exams. Convenors need to consider whether a student’s work can be assessed in non-exam formats but, if an exam is the best test of teaching and learning, it should be retained, regardless of student choice.

If, however, students overwhelmingly choose non-exam-based modules, this would leave modules retaining an exam in a vulnerable position. The aim of this project is to find ways to diversify our assessments, but this could leave modules that retain traditional assessment patterns vulnerable to students deselecting them. This may have implications for benchmarking.

It may also be the case that the attempt to avoid ‘stress’ is not necessarily in students’ best interests. The workplace is not a stress-free zone and it is part of the university’s mission to produce resilient, employable graduates. Removing all ‘stress’ triggers may not be the best way to achieve this.

Follow up

  • DEL will convene a third focus group meeting in the Spring Term.
  • The co-leaders of the ‘Diversifying Assessments’ project will present the findings of the focus groups and surveys to DEL in a presentation. We will outline the results of our work and call on colleagues to reflect on the assessment models used on their modules with a view to volunteering to adopt different models if they think this appropriate to the teaching and learning aims of their modules
  • This should produce an overall assessment landscape that corresponds to students’ request for ‘three-way’ (at least) diversification of assessment.
  • The new landscape will be presented to the third focus group for final feedback.

Links

With thanks to Lauren McCann of TEL for sending me the first link which includes a summary of students’ responses to various types of ‘new’ assessment formats.

https://www.facultyfocus.com/articles/online-education/assessment-strategies-students-prefer/

Conclusions (May 2018)

The ‘Diversifying Assessment in DEL’ TLDF Mini-Project revealed several compelling reasons for reflecting upon assessment practice within a traditional Humanities discipline (English Literature):

  1. Diversified cohort: HEIs are recruiting students from a wide variety of socio-cultural, economic and educational backgrounds and assessment practice needs to accommodate this newly diversified cohort.
  2. Employability: DEL students have always acquired advanced skills in formal essay-writing but graduates need to be flexible in terms of their writing competencies. Diversifying assessment to include formats involving blog-writing, report-writing, presentation preparation, persuasive writing, and creative writing produces agile students who are comfortable working within a variety of communication formats.
  3. Module specific attainment: the assessment conventions in DEL, particularly at Part 2, have a standardised assessment format (33% assessed essay and 67% exam). The ‘Diversifying Assessment’ project revealed the extent to which module leaders need to reflect on the intended learning outcomes of their modules and to design assessments that are best suited to the attainment of them.
  4. Feedback: the student focus groups convened for the ‘Diversifying Assessment’ project returned repeatedly to the issue of feedback. Conversations about feedback will continue in DEL, particularly in relation to discussions around the Curriculum Framework.
  5. Digitalisation: eSFG (via EMA) has increased the visibility of a variety of potential digital assessment formats (for example, Blackboard Learning Journals, Wikis and Blogs). This supports diversification of assessment and it also supports our students’ digital skills (essential for employability).
  6. Student satisfaction: while colleagues should not feel pressured by student choice (which is not always modelled on academic considerations), there is clearly a desire among our students for more varied methods of assessment. One Focus Group student argued that fees had changed the way students view exams: students’ significant financial investment in their degrees has caused exams to be considered unacceptably ‘high risk’. The project revealed the extent to which Schools need to reflect on the many differences made by the new fees landscape, most of which are invisible to us.
  7. Focus Groups: the Project demonstrated the value of convening student focus groups and of listening to students’ attitudes and responses.
  8. Impact: one Part 2 module has moved away from an exam and towards a Learning Journal as a result of the project and it is hoped that more Part 2 module convenors will similarly decide to reflect on their assessment formats. The DEL project will be rolled out School-wide in the next session to encourage further conversations about assessment, feedback and diversification. It is hoped that these actions will contribute to Curriculum Framework activity in DEL and that they will generate a more diversified assessment landscape in the School.

Using grademark to write high quality feedback more rapidly in the school of law – Dr Annika Newnham

By laurenmccann Comment Closed

Profile picture for Dr Newnham

Dr Newnham is the module convenor for LLB Family Law. Her areas of interest include, Child Law, Autopoietic Theory, The Common Intention Constructive Trust.

Since 2015, Annika has gradually personalised the ‘Quickmarks’ function within Turnitin Grademark to be both discipline specific and also assignment specific. In addition, Dr Newnham has also developed a lengthy comments bank which she can draw on and personalise to ensure that she can write high quality feedback more quickly, speeding up the entire marking process.

OBJECTIVES

The School of Law currently operates online submission, marking and feedback for the vast majority of assessed work. As part of this process it makes extensive use of Turnitin Grademark and some of the functionality on offer, including Quickmarks. Given the large numbers of students submitting work within the School and the need to provide high quality feedback quickly, I wanted to use these new tools to speed up the entire marking process and support the quality and quantity of feedback offered.

CONTEXT

The School of Law recruits strongly, makes extensive use of summative assessment and maintains a large number of core modules. Online assessment has been adopted, in part, to help support the continued provision of high quality feedback in this context while ensuring that feedback is returned to students within 15 working days.

IMPLEMENTATION

Grademark allows for the customisation of Quickmarks by individuals markers. I very quickly began to customise the Quickmarks that were available to me by adding comments that I make frequently. Gradually, over time, my Quickmarks section has expanded to include a whole series of comments which range from just a few words to more lengthy sections of text. Dragging these across to relevant sections of text saves me a considerable amount of time because I’m not writing out the same type of comment again and again. I’ve even developed my set of Quickmarks to be specific not only to each module I teach but to each assignment I mark within that module. I carefully save each set with a different name so I can easily access them again. Grademark even remembers my Quickmarks sets from one year to the next so my collection appears automatically when I open each new essay.

I wanted to explore the possibilities of reducing marking time whilst maintaining the quality and quantity of feedback in other areas.
This approach worked very well for targeted in text comments throughout the essay but, like most markers, I also leave summative text in the general comments section in the Grademark sidebar so that students have a sense of my overall thoughts. I started to compile a lengthy list of comments that I use extensively in a simple and separate word document. I ordered each set under key headings. Some of these are generic for all essays: writing style, referencing, structure and so on. There are also sets of comments on how students have tackled a particular issue in law, for example how well they have presented balanced arguments on commercial surrogacy, or have understood the different stages of a cohabitant’s claim for a share in her ex-partner’s house. Each heading contains 8-10 different sentences or longer sections covering a wide range of different areas I may want to comment on. I am then able to cut and paste the most relevant into the Grademark comment box and, if needed, rewrite to suit the exact statement for the specific essay I’m working on. This process has become even more efficient since the arrival of a second screen. I can list my commonly used statement on the left hand screen, cut and paste or drag over to the actual essay on my right hand screen. Although I might then want to personalise the statement I still save a significant amount of time in comparison to typing everything out repeatedly for each essay.

IMPACT

I maintain a balance between the use of Quickmarks, my comments bank and specific comments written for each piece of work. Students should not receive exactly the same comments time and time again. Feedback should not become a highly mechanised process. But Quickmarks and comments banks can be used as a starting point or work alongside very specific comments written for a particular piece of work. In this way I can maintain the quality and quantity of my feedback whilst speeding up the marking process considerably. In particular, this approach seems to ensure greater consistency between essays in terms of the amount of feedback that each student receives because it is so much quicker and easier to insert comments. More broadly it feels like a far more efficient process and is certainly a more fulfilling task to undertake.

REFLECTIONS

Quickmarks and cut and paste comments have made marking feel much less like a chore; and removes the irritation often felt if you have to correct the same misunderstanding again and again to different students.

LINKS

Turnitin Quickmark

https://guides.turnitin.com/01_Manuals_and_Guides/Instructor_Gu ides/Turnitin_Classic_for_Instructors/25_GradeMark/QuickMark

Feedback via audiofiles in the Department of English Literature – Professor Cindy Becker

By laurenmccann Comment Closed

Profile picture for Prof. Becker

Cindy Becker is the Director of Teaching and Learning for the School of Literature and Languages and also teaches in the Department of English Literature. She is a Senior Fellow of the Higher Education Academy and has been awarded a University of Reading Teaching Fellowship. She is an enthusiastic member of several University Communities of Practice: Placement Tutors, University Teaching Fellows, Technology Enhanced Learning, and Student Engagement Champions.

Cindy is a member of Senate and has sat on university steering committees and working parties; she is also a member of the Management Committee for the School of Literature and Languages and chair the School Board for Teaching and Learning. She is the convenor of Packaging Literature and Shakespeare on Film.

In September 2015 she started to trial the use of the audio feedback function within Turnitin’s online marking tool (GradeMark). This innovative approach did present some initial challenges but, overall, it proved to be a great success for both Cindy and her students.

OBJECTIVES

GradeMark was introduced to the University in the Summer of 2015. I wanted to use this new marking tool to explore different ways of providing feedback for students. In particular, I wanted to adopt a more personal approach and provide more in-depth feedback without significantly increasing the time I spend marking each essay.

CONTEXT

GradeMark allows you to produce typewritten feedback for assessment work and this is what most of us are used to. However, it will also let you click on an icon that allows you to create an audio file of up to three minutes of spoken feedback instead.

IMPLEMENTATION

I started off by making notes as I marked the essay and then talking through them on the audio file. It did not work very well because my feedback became stilted, took longer than three minutes and was time consuming to prepare. I think I lacked confidence at the outset.

Now I take a more relaxed approach. I make no more than a couple of notes (and often not even that) and then I simply press the record button. As I talk to the student I scroll down the assignments on the split screen and this is enough to jog my memory as to what I want to say. Taking a methodical approach has helped me. I always begin with an overview, then work on specific challenges or praiseworthy elements, then end with a brief comment summing up my thoughts. If it
goes wrong, I simply scrap the recording and begin again. I save myself time with the uploading by setting it to upload and then begin to work on the next assignments. This saves the frustration of staring at an upload symbol for ages when you want to get on with it.

IMPACT

It is worth the effort.

For now, students love it. I asked students to let me know whether they would prefer written or audio file feedback and those who responded voted for audio file. The novelty factor might wear off, but I think at the moment it is a useful way to engage students in our assessment criteria and module learning aims, in class and beyond.

For now, I love it. It is a pleasant change; it is quicker and fuller than written feedback. It seems to allow me to range more widely and be more personally responsive to students through their assignments. Because I am ‘talking to them’ I have found myself more ready to suggest other modules they might like, or some further reading that they might enjoy.

REFLECTIONS

It can take a few attempts to ensure that your headphones are working within the system. This is most usually a problem with GradeMark or Blackboard more generally – restarting Blackboard or even your computer will fix it. You might not have headphones already to hand, and that sounds like another investment of time and money, but it’s good idea to buy cheap headphones – they cost around £20 from a supermarket and are perfectly adequate for the job. You feel like a twit talking to your computer. Of course you do – who wouldn’t? After your first few audio files it will feel perfectly natural.

For the future, I can see it having an impact on assignment tutorials. I believe I can have an equal impact via a tutorial or a three minute audio file, and everyone actually listens to their audio file. I am going to have to decide what to do with the extra ‘spare’ contact time this might give
me…

Changing the assessment experience of professional staff in SAPD – Emily Parsons

By laurenmccann Comment Closed

Profile picture for Emily Parsons

Emily Parsons is a Senior Programme Administrator in the School of Agriculture, Policy and Development (SAPD). Online assessment has been adopted throughout the SAPD, impacting academic and non- academic colleagues. In this case study, Emily outlines the experiences of her Support Centre team working with SAPD as an Early Adopter School.

OBJECTIVES

To reduce the administrative burden of assessment and improve the overall assessment experience for staff within the Support Centre whilst supporting change within the School.

CONTEXT

The University has a long-term vision to move toward online assessment, where practical, and improve underlying processes. SAPD became an Early Adopter School in May 2017 which allowed the EMA Programme to support a significant shift away from a mixture of online and offline marking to the full provision of online marking where practical. The SAPD Support Centre was involved right from the start working collaboratively with the EMA, TEL, CQSD and senior school leadership team during the change process. The Support Centre was one of the first to experience the impact on their working practices of a shift towards greater online marking throughout 2017-2018.

IMPLEMENTATION

As an Early Adopter School, SAPD undertook a full change programme to support online submission, feedback and grading as well as support for all underlying processes. A series of meetings and three major workshops lasting between three and four hours were held throughout the Summer involving all collaborating teams.

Initially only two members of the Support Centre team were involved but representation quickly expanded to include at least four members. It was really important to make sure that a range of professional staff views were being heard during the change planning stage particularly because all of these colleagues would play a role in implementing new processes and delivering change.

Each of these collaborative workshop meetings drew everyone together, in person, in one room instead of relying on e-mail correspondence. This proved far more effective. Relying on e-mail could have significantly delayed the process and may not have led to the kind of in depth, rich discussion around assessment practice, process and policy within the School that was seen at each meeting.

One of the triggers for these debates was the creation of a series of highly detailed process flow diagrams showing the end to end assessment process within the University. These process maps outlined who does what and when in four main diagrams – anonymous marking using the Blackboard marking tool, named marking using the Blackboard tool, anonymous marking using Turnitin, and named marking using Turnitin. These maps were essential to understanding the end to end process and for allowing the School to start thinking about consistent practices.

Following this approach to consistent practice professional staff also created a manual containing essential information such as how to set up submission points or Turnitin similarity reports in the way that the School wanted. All professional staff could then follow this detailed guidance. This proved essential to ensure that all colleagues were working in a similar way.

IMPACT

Two key areas of impact have been experienced within the Support Centre – the first surrounds the adoption of more consistent processes to deal with the submission, receipt, marking and moderation of coursework, and the second surrounds the significant increase in amount of work marked online.

The adoption of more consistent processes was made possible by the creation of the detailed process diagrams outlined above. These show the 45-50 steps involved from submission to final exam board agreement and confirmation, including who does what exactly, and when. The creation of these process diagrams during the Summer workshops, informed by all of the groups involved was, in itself, a useful exercise. We could take a step back and really think about how we could make this process as efficient and as effective as possible whilst keeping an element of flexibility to cover any type of submission or new requirement that we collectively hadn’t thought of!

During the workshops, the Support Centre, in collaboration with the School, was also asked to create a large assessment spreadsheet listing all submissions due to be submitted during the academic year. The creation of this detailed assessment spreadsheet, in itself, provided an opportunity for colleagues to pause and review the amount of assessment and the School’s use of different assessment types.

This was also a crucial starting point from which we could categorise assessment types (such as group work, individual essay, video submission) and then think through which of the two marking tools – Blackboard or Turnitin – would be most appropriate for each type. Both the process diagrams together with these spreadsheets helped to support workflow and planning within the Support Centres who then knew exactly what they had to do and when, for the full academic year.

Under the new, more consistent. processes, academic colleagues were no longer required to create submission points. This role was transferred to professional staff and actually represented one of the most significant changes undertaken. All submission points are now created in the same way -for example there is no longer any variation within the School surrounding student views of Turnitin reports as all students only see similarity reports after the submission deadline. In general, academic colleagues were happy to transfer the set-up of submission points to professional staff and just had to inform the Support Centre, in advance, when assessment was due. Around 400 pieces of assessment were due during 2017- 2018.

Alongside increased consistency surrounding processes, the School has seen significant increases in the amount of work submitted and marked online. Overall this change has improved the assessment experience for colleagues within the Support Centre in a number of ways:

• Previously, using a rota system, colleagues were allocated a time slot to sit in the front office to receive hard copies and process each paper coming in. This was an intense role and so reduced the time available to undertake any other supporting role. There is no need to do this in the current system as submission is managed online for almost all work. This represents a significant time saving for colleagues.

• At the end of the marking process, each paper would also have to be sorted alphabetically and placed in individual envelopes, ready for collection by students. This doesn’t happen now for the vast majority of pieces which are accessed online. In the past this role might have taken half a day. Now it takes an estimated 30 minutes for the small amount of assessment still marked in hard copy. The time saved has been described by professional staff within the team as “extraordinary”.

• This also means that the assessment process has become much more scalable. Support Centres can cope with increases in students without seeing significant increase in workload.

• The Support Centre used to ask academic colleagues to return marked work to them within 14 working days of submission to allow time for processing. There is no need to do this anymore because the marks and feedback are returned online so academic colleagues now have the full 15 working days to mark submitted work,

• The Support Centre is no longer drowning in a sea of paper leaving much more room and saving storage space. This was a particular problem when students failed to come back to collect their work.

• Some of the functions of the marking tools are saving a significant amount of time for the Support Centre. One example surrounds non-submission. It took a considerable amount of time to contact students who had failed to submit work when they were submitting hard copies. Now Turnitin allows professional staff to send one e-mail to all non-submitters easily and very quickly.

• Previously, in order to undertake internal moderation, Support Centre staff would release marks but keep the hardcopy coursework, which included their feedback, back from the students until internal moderation had taken place. After this point, the full feedback would also be released. In order to undertake external moderation, Part 2 and Part 3 students were asked to create a portfolio of their work, including marks and feedback, and submit this at the end of the academic year so that external examiners could review the work. Student engagement in this process was variable with some students having lost their work by this point. In addition, these processes generated a huge amount of paper and took a large number of working hours to manage. This isn’t necessary anymore, aside from a very small amount of fieldtrip work. Internal and external moderators can access both marks and feedback quickly and easily online, from wherever they are in the country.

REFLECTIONS

Moving the School towards more consistent approaches to managing assessment and increasing online marking and feedback has largely been a very positive experience for the Support Centre. We are now enjoying a range of benefits which have made our role within the assessment cycle much more manageable.

We had worried that some areas of work might increase – for example, we might have seen more reported cases of academic misconduct as a result of much greater use of Turnitin similarity reports. This has not occurred but the School had been undertaking a significant amount of work in this area including the introduction of a formative piece of work at Part 1 and at the start of the MSc programmes which is then analysed during follow on seminars.

As we move forward into the next academic year, there are still some areas that we need to think about a little more. We’ve discovered through this processes, for example, that there are multiple different ways in which academic colleagues assess and give feedback on presentations. We need to work on understanding the processes in this area more in 2018-2019.

This year we will also be able to start the process of collecting new assessment data and deadlines much earlier. This will enable us to create submission points around July and August. This will place us in a better position to plan ahead for 2018-2019.

Reflecting on change and the management of non-standard submissions in Typography – Dr Jeanne-Louise Moys

By laurenmccann Comment Closed

Profile picture for Dr Moys

Jeanne-Louise teaches design practice, theory and research skills across a range of genres and platforms. She is the Programme Director for the MA Creative Enterprise and the Pathway Lead for the MA Communication Design (Information Design Pathway).

OBJECTIVES

Typography has been keen to continue to support the move from offline to online submission, feedback and grading, where possible. In particular, the Department has wanted to ensure a more consistent and streamlined approach to managing assessment, especially given the range of diverse submission types within Typography programmes. The Department were also very keen to ensure that online marking tools allowed colleagues to provide feedback that supports students’ design literacy. In this respect, markers aim to give feedback designed to allow for openness in the ways students think and that builds students’ confidence to develop their own design judgement.

CONTEXT

The University has a long-term vision to move toward online assessment, where practical, and improve underlying processes. In 2015–6, the Department of Typography adopted a policy of either online submission or dual submission (where students are asked to submit both an online digital ‘copy’ and in material form as relevant to the particular deliverables of different design briefs) across the undergraduate degree. Paper-based feedback forms were replaced with online rubrics. The Department mainly made use of Blackboard as a marking tool but with some further use of
Turnitin, particularly for essay based assessment. The Department has undertaken this change in the context of growing student numbers, increasing diversity of student cohorts and growing numbers of international students. The trends have increased the need to adopt more efficient and streamlined assessment processes.

IMPLEMENTATION

Over the past four years the Department has supported student online submission and the increased use of marking tools. In 2014, The Head of Department and I initially worked together to explore different online tools to find sustainable assessment practices for increasing cohorts. We liaised with our IT partners who encouraged us to work with Maria Papaefthimiou – as they were aware that the University was setting up a new TEL team. Maria introduced us to Blackboard rubrics, which we piloted for both practical and written forms of assessment.

These early initiatives were reviewed ahead of our decision to adopt online assessment for all undergraduate coursework (with a few exceptions such as technical tasks, examinations and tasks where self or peer assessment plays a particular role in the learning process). I then translated our paper-based forms into a set of Blackboard rubric templates for colleagues to work with and provided a workshop and video resources to support the transition.

For almost every submitted piece of work, students receive feedback from colleagues using either Turnitin or the Blackboard marking tool. Each piece has an online submission point so that colleagues can provide feedback online, often using the rubrics function within the Blackboard marking tool.

One of the challenges faced by the Department has been managing non-standard types of submission. Typography employs a particularly broad range of assessment types including self- and peer-assessment and group work. It also handles a range of different physical submissions such as books or posters and assessment involving creating designs like websites and app prototypes that exist only in digital form.

Because of the nature of the work, dual submission is common. Our policy of online submission for written work and dual submission for practical work ensures that – regardless of the nature of the work – students receive feedback and grades in a consistent manner throughout their degree.

More recently, we have introduced some new practices that support the development of professional skills and enhance the transparency of group work. For example, professional practice assignments use a project management app, Trello. Students are assessed on their usage and the content (including reflection) they input into the app. The tutor can, for example, set up a Trello group and monitor group activity. Some practical modules require students to use prototyping software or create videos. In these cases, it might be easier for students to share links to this content either by submitting the link itself online to Blackboard or to a dedicated Typography submission e-mail address monitored by administrative colleagues (although this second approach may change as we work with the EMA Team).

A second issue faced by the Department during implementation, as a result of the significant diversity of assessment, is that the management of online submission can become confusing for students in terms of what exactly they should submit and how. The diversity of assessment allows students to demonstrate a range of learning outcomes and broad skills base but the Department has had to ensure that students fully understand the range of submission practices. This challenge exists both in Part 1 when students are being introduced to new practices and in Parts 2 and 3 where a single design brief may have multiple deliverables. We are continually working to find the best balance between ensuring the kind of submission is always appropriate to the learning outcomes, provides students with experience in industry standard software and tools, and is accompanied by clear guidance about submission requirements.

IMPACT

The shift from offline to online assessment within the Department has led to a range of changes to the staff and student experience:

1. Online feedback for students has meant that they now always know where their feedback is. There is no need for them to contact their tutors to access content.

2. For some staff, the use of online marking and feedback has meant spending some time getting used to the interface and learning about the functionality of the tools, particularly the Blackboard marking tool. There have been some issues surrounding the accessibility of rubrics within Blackboard and their consistent use, which the Department has had to work through. In general colleagues are now reporting that online marking has significantly reduced marking time, especially where more detailed rubrics have
been developed and trialled in the current academic year.

3. The Department has spent time thinking carefully about the consistency of the student assessment experience and making the most of the functionality of the tools to make marking easier and, potentially, quicker. As a result, there is a sense that the practices adopted are more sustainable and streamlined, which has been important given rising student numbers and increasingly diverse cohorts.

REFLECTIONS

Over the last year, following recommendations from Periodic Review, the Department has been trialling different practices such as the creation of much more detailed rubrics. As noted above, detailed rubrics seem to reduce marking and feedback time, while providing students with more clarity about the specific criteria used to assess individual projects. However, these do not always accommodate the range of ways in which students can achieve the learning outcomes for creative briefs or encourage the design literacy and independent judgment we want students to develop.
We are also working on ensuring that the terminology used in these rubrics is mapped appropriately to the level of professional skill expected in each part of the degree. The Department is currently looking at the impact of this activity to identify best practice.

Typography is keen to continue to provide a range of assessment options necessary for developing professional skills and industry- relevant portfolios within the discipline. We are committed to complementing this diversity with an assessment and feedback process that gives students a reassuring level of consistency and enables them to evaluate their performance across modules.
There is some scope to develop the marking tools being used. It would, for example, be very helpful if Blackboard could develop a feature where students can access their feedback before they can
see their marks or if it allowed colleagues to give a banded mark (such as 60-64), which is appropriate formative feedback in some modules. In addition, Typography students have reported that the user experience could be improved and that the interface could be more intuitive. For example, it could contain less layers of information and access to feedback and marks might be more direct.

More broadly, the shift from offline to online practices has been one driver for the Department to reflect on existing assessment practices. In particular, we have begun to consider how we can better support students’ assessment literacy and have engaged with students to review new practices. Their feedback, in combination with our broader engagement with the new Curriculum Framework and its impact on Programme Level Assessment, is informing the development of a new set of rubric templates to be adopted in autumn 2018.

LINKS

For further information please see the short blog, ‘Curriculum Review in Practice Aligning to the Curriculum Framework-first steps started at:
http://blogs.reading.ac.uk/engage-in-teaching-and- learning/2018/04/09/curriculum-review-in-practice-aligning-to- the-curriculum-framework-first-steps-started-by-jeanne-louise- moys-rob-banham-james-lloyd/

Pre-sessional English use of Turnitin’s online marking tool – Rob Playfair, IFP Course Tutor

By laurenmccann Comment Closed

OBJECTIVES

I was interested in improving the efficiency of my marking, and liked the idea of having a digital record of written feedback to students. During the PSE induction for new tutors we were told that the University is moving towards e-feedback over the next few years so it seemed like a useful skill to acquire.

CONTEXT

My group of international students were on a 9 week course to improve their level of English before starting their postgraduate studies. They needed to write three 500 word essays and one 1500 word project. For each of these, students wrote two drafts. I needed to provide written feedback on both drafts and the final version of each essay, i.e. a lot of marking!

IMPLEMENTATION

Jonathan Smith, PSE course director and ISLI TEL Director, gave all teachers a one-hour workshop on how to use Turnitin and Grademark, during which we had a chance to get hands on with the software. Each year Jonathan runs a training session for new members of PSE staff who will work on the PSE courses during the summer.

Later, Jonathan shared the PSE ‘QuickMarks’, with those of us who had opted to use e- feedback. We could download these, via our QuickMarks library, into our own personal QuickMarks set. These comments were then available each time we opened an essay.

The QuickMarks focussed on common student errors with explanations and links to relevant sources. ‘Quickmarks’ are based not only on common grammar and lexical errors but also on the complexity of the structures used and coherence and cohesion in the texts.

Students grew accustomed to submitting work, accessing feedback and seeing their progress.

IMPACT

• It was quicker to note common student errors in-text using the QuickMarks, than repeatedly hand writing the same comments.

• Students were able to read & start acting on my feedback as soon as I did it, rather than waiting until the next class.

• I could quickly refer to previous drafts and the comments I had given to monitor uptake.

• I could browse work from students who were not in my class, via the Turnitin feedback suite, to see a broader range of essays and also see the feedback that colleagues were giving because, in this case, the point of submission was the same for the whole cohort. As this was my first experience teaching the programme, this was particularly useful.

REFLECTIONS

The speed of communication with students was the biggest benefit – as soon as my marking was done students could see it. This meant that students could formulate questions about my feedback before class, making the time in class much more productive.

In terms of quality of marking I think there might be a tendency to over-mark using the QuickMarks, because it only takes a second to add a one yet creates quite a lot for the student to do – reading an explanation and perhaps visiting a website. I’d like to explore the impact of this on uptake.

Finally, on a practical level I found this helped my organisation – all the scripts, scores and comments are in one place. It was also easier to submit scripts for moderation: I just gave the names of students to the Course Directors who could go into the system and see the scripts themselves.

FOLLOW UP

• I’m currently using it in a similar way on the International Foundation Programme (IFP).

• At present all students can do is upload their work then download my comments. I’d be interested in a function which allows students to respond to my comments – making corrections or asking questions. This would support the feedback cycle.

• To improve the reliability of the summative scores, I wonder whether we can learn from elements of comparative judgment programmes such as No More Marking.

LINKS

www.nomoremarking.com

http://www.reading.ac.uk/internal/ema/ema-news.aspx

https://www.reading.ac.uk/ISLI/study-in-the-uk/isli-pre-sessional-english.aspx