Using quickmarks and rubrics in online assessment – Catherine Foley

Catherine Foley is a lecturer in Primary Maths Education in the Institute of Education. She is Director of the Primary School Direct programme which trains people to be teachers whilst they are working in schools.

Image of Catherine Foley

OBJECTIVES

Catherine describes her experience of using the Feedback Studio to move from Word-based marking an assignment to full use of Grademark.

CONTEXT

Catherine Foley is a lecturer in Primary Maths Education in the Institute of Education. She is Director of the Primary School Direct programme which trains people to be teachers whilst they are working in schools. Her experience of electronic marking relates primarily to a 20 credit postgraduate module which is part of this programme, developing the reflective practice and critical thinking of the trainees. The module is assessed through one piece of written work which is assessed formatively and summatively and is taken by approximately 80 students each year.

IMPLEMENTATION

Up until the current academic year, although students would submit their work through Turnitin (for both formative and summative attempts), they would receive feedback in the form of underlined grading sheets and text-based comments which would be completed for each student and uploaded to be released to them via Grade Centre. As with other IoE programmes, all submission, grading and feedback for this assessment is now carried out electronically.

This year, we decided to use the full electronic feedback option for both assessments since the first formative experience would give students (and staff) the chance to get used to the system. We
developed our own rubric for the assessment. For the formative assessment, we decided not to use quickmarks but just to focus on becoming familiar with using the rubric. For the summative
assessment, both a rubric and quickmarks were used: the quickmark set is the same as that used for other initial teacher training programmes.

In my own marking, I found it helpful, when getting started, to open out the full rubric in a grid from the sidebar in the feedback studio. After a while, I was clear what the different statements meant and so could use the sliders more confidently.

IMPACT

  • Speed of marking. Although marking has not been any quicker so far overall, it is likely that this will speed up as the administrative problems are ironed out and we get to know the
    system. Not having to save individual files saves a lot of time which can be spent on quality feedback.
  • Ease of moderation. Because all the assessment and feedback is in the same place, it is much more straightforward and a module convenor is easily able to quality-assure the marking
    that is taking place.
  • Curriculum review opportunity. Developing our own rubric for the assessment encouraged us to review what we had been doing. It made use stop and examine our taken-for-granted practice.
  • Student ownership of feedback. We had a workshop on developing academic writing and it was interesting to see all the students with their laptops open, looking at very specific
    pieces of contextualised feedback received online for their first assignment.
  • Using rubric reports for bespoke study advice sessions. We used the function in Turnitin to generate a report on how well students had achieved as a cohort in relation to the different
    rubric themes. We sent the report to one of the study advisers who was then able to use this to pinpoint areas to focus upon in helping students work towards their next assignment.

REFLECTIONS

Many of the challenges we experienced were due to the fact that the assessment is marked by five different members of staff:

  • When we were using Word-based documents for feedback, we could shape and guide the feedback which tutors were giving more easily (for example with a writing frame). In the feedback studio, the text comment box presents markers with a blank space so it has been harder to ensure a common approach across markers. We therefore agreed a common structure for feedback in this box.
  • The marking team had differing levels of experience with electronic marking. Because the quickmark set had to be uploaded by each marker to their Blackboard account and not all markers were present on campus at the same time, this was a logistical challenge.
  • With the options for quickmarks, rubric statements and open text comments, it would be easy for markers to over-assess each piece of work. Our agreement was that, since students were getting extra feedback in terms of the first two kinds of feedback, the final text comments should be brief and simply recognise specific areas of success then pinpoint areas for
    development.
  • Limitations in functionality of the feedback studio. Some markers liked to be able to use Word to check the number of times a student has used a particular phrase or look at the
    consistency between citations and references: you can’t currently move around the document so easily (unless you download it). Some warning or confirmation messages from
    the system (for example when moving onto the next piece of work) would make it still more user-friendly. With several people involved in marking an assignment, it is easy for markers
    to accidentally change each other’s grades – it would be helpful if grades and comments could be ‘locked’ in some way. Are different levels of access possible, so that external examiners can see the feedback studio but without being able to change feedback?
  • There are still issues (mostly to do with administrative protocols) to iron out. The IoE is currently reviewing its moderation processes and determining the extent to which
    students know they have been included. Programme directors are working with their admin teams to determine exactly how
    academics will be informed when an ECF assignment has been submitted.

From boning a duck to reflecting on utopia in English Literature: Using blogs in teaching, learning and assessment

Lauren McCann, Centre for Quality Support and Development   l.j.mccann@reading.ac.uk

Chloe Houston, School of Literature and Languages   c.houston@reading.ac.uk

In the 2009 hit film ‘Julie et Julia’, real life American office worker Julie Powell (Amy Adams) spends a year cooking her way through culinary legend Julia Child’s ‘Mastering the Art of French Cooking’ and ‘blogs’ about it. Powell picks up a following and generates a dialogue with her readers who comment on her posts and offer her advice, from how to prepare a lobster to how to bone a duck. Blogs are, of course, more than just about French cooking. There are blogs about all sorts of things. They have never been bigger and have become an increasingly useful tool in education too. In this article, we’ll explore this tool and find out how it’s been used for summative assessment on the BA in English Literature.

What is a blog?

A blog – short for web log – is a personal online journal that can include various media and is intended for sharing with others, like an open web-based diary. Most blogs have some kind of commenting system so that people can share their thoughts on entries. Blogs encourage students to clearly express their ideas and engage in social learning.

In Blackboard Learn, instructors can create and manage blogs from within a course. Enrolled users can then view and create entries and comments in them. They can be used for various purposes and as a tool for both formative and summative assessment, providing an alternative to more traditional methods.

Case study: Using blogs in English Literature at the University of Reading

Dr Chloe Houston has used the blog tool this year in a new third year module, ‘Utopia: The Ideal Society in English and American Literature’. Chloe was interested in diversifying the assessment methods experienced by her students and in moving away from the conventional essay. Aware that after graduation, students could be expected to write in a variety of media for a range of audiences, she was keen to give them the opportunity to write in a different format and share their ideas with their peers.

In getting ready to use the tool, Chloe did a good deal of preparation which was key to her ultimate success, contacting TEL CQSD for advice and researching academic blogs. She set up a Blackboard blog to be used as 50% of the module’s assessment in which students were expected to post entries during the term. An inexperienced blogger, she made use of a post-graduate student with relevant experience to help prepare the students and provided support materials. Mid-term evaluation suggested students were enjoying working in this way and end of module evaluations confirmed this, with the additional benefit that Chloe found the assessments more varied and interesting to mark! When asked if she had any advice for other staff thinking about trying out blogging, she exclaimed, “Immerse yourself in the blogging culture and just do it!”

Student Josie Palmer was one of a number who reported positively on her experiences of using the blog tool: “With students having grown up around technology… I feel a blog is a positive step forward in the way work is assessed. It’s easy to access and manage, it’s interactive, as you can read other student’s work and comment on what they have written… This differs greatly from essays… [The blog] gives students the opportunity to upload work and receive feedback more frequently… We are given more of an opportunity to explore ideas in different ways, with a simple format, as opposed to putting all the work collected over a term into one final essay. I think that as a format of assessment the blog works brilliantly!”

In this short video, Chloe discusses her use of the blog in her module:

What next?

If this article has inspired you to find out more about using the blog tool in your own teaching, please see Blackboard’s Support for Staff tab and/or contact the TEL CQSD team for advice. You can also subscribe to the TEL team’s very own blog at http://blogs.reading.ac.uk/tel/

Bibliography

Salmon, G, 2013. E-tivities: The Key to Active Online Learning . 2nd ed. Abingdon: Routledge.

Downes, S, 2004. Educational Blogging . EDUCAUSE Review , [Online]. 39 (Number 5), 14-26. Available at: http://www.downes.ca/post/40939 [Accessed 01 March 2016].

Hammond , M , (2006). Blogging within Formal and Informal Learning Contexts: Where are the Opportunities and Constraints?’  In Networked Learning. University of Warwick , 2006.

Julie et Julia , 2009. [DVD] Nora Ephron, USA: Sony Pictures.

Blending face-to-face and online to deliver group seminars

Jeremy Lelean, Staff Engagement                                                                                                                                                           j.lelean@reading.ac.uk

Context

Soil Security Programme (SSP), School of Agriculture, Planning and Development

PhD students, external institutions and organisations

Description

 The Soil Security Programme is a PhD Student research network that includes a number of
other institutions and external bodies. Students are dispersed around the country and
sometimes abroad.
 The ability for the dispersed members of the network to join seminars held at Reading by PhD
students would help facilitate increased communications and information sharing.
 Two face-to-face seminar events have been held at which members have been able to join
remotely via Collaborate.
 Members were sent a ‘guest link’ and joining instructions and were able to watch the
presentations given in the physical room.
 The initial seminar had 11 participants, 9 in the room and 3 joined remotely.
 A USB speakerphone was attached to the laptop in the room to provide the audio and a
webcam was used to show what was happening. Presentations were delivered using
‘Application Share’ in Collaborate.
 Jeremy facilitated the session to ensure the remote participants were kept informed of what
was happening in the physical space.

Impact

Using Collaborate was a success and participants found the experience was very good. There
were some minor points raised but this did not detract from usability.
 Remote participants could easily join in sessions that they wouldn’t otherwise have been able to
attend.
 Recordings of the seminars were made available to members of the network.
 SSP plan to deliver an online conference using Collaborate to help build a community of early
career researchers and PhD students in the field of soil science.

Thoughts and reflections

 Remote participants weren’t able to see where the speakers were pointing to on the slides. Ask
speakers to use the inbuilt Pen and Laser Pointer tools when PowerPoint is used in Presenter
View to highlight slides.
 It was necessary to restart application share when moving between different PowerPoint
presentations.
 Remind participants in the physical space to remember that there are remote participants.
 Participants in the physical space can’t see the chat taking place in Collaborate.
 Chat was particularly useful for communication between the facilitators and remote users
without disturbing the seminar speakers.
 Ensure that remote users can hear those speaking in the room clearly. It may be necessary for
the facilitators to repeat questions or ask people to speak more loudly.

 

Engaging students in online careers events using Blackboard Collaborate

Daniel Kiernan & Graham Philpott, Henley Business School                                                                                              d.kiernan@henley.ac.uk   g.philpott@henley.ac.uk

Context

MBA students, Finance

Number of participants in sessions: 26

Session length: 20 minutes

Description

 Getting students to attend careers events during particular periods of term can be difficult. The
use of Collaborate was piloted to see if attendance could be increased by providing online
sessions when students typically don’t engage with face-to-face careers events.
 A short 20 minute presentation was given with PowerPoint slides and included separate online
poll questions.
 Students were encouraged to pose questions using the ‘Chat’ feature

Impact

 Student feedback was positive.
 The online event had higher attendance than would be expected for an equivalent face-to-face
session held in the same period. “We would typically really struggle to get 26 attendees to a
careers event during the summer term.”
 It was easy to organise and deliver the event.
 Not all of the students who pre-registered actually attended the webinar (15 attended, 17
didn’t). Most students stayed for the entire session.
 Dan was able to send those that weren’t able to attend a link to a recording after the session.

Thoughts and reflections

 Dan is keen to make future sessions more interactive, with more questions and responses. This
should help address attendance concerns. If you attend you get your question answered live!
 Possibly have an assistant to help moderate the chat and pose questions to the presenter.
 Think about the way in which you want to present your content and how this affects your ability
to manage and facilitate the session.
 The PowerPoint slides were displayed on Dan’s computer in Presenter View and delivered in
Collaborate using ‘Application Share’. PowerPoint presented in this way requires 2 screens and
also meant Dan wasn’t able to see the Chat while the slides were up.
 Check your camera angle and be mindful of it during the session.
 If you are recording the session, remember to exit the webinar properly, using the ‘Leave
Session’ button otherwise the recording continues.
 The recording captured the screen, audio/video and chat but didn’t capture the poll on screen
as this was viewed in a separate tool.

 Students attended the session using the ‘guest link’. This doesn’t record the email of the
students, so you’ll need to think about how students sign-up if you want to contact them (e.g.
via email) after the session.
 How should the recording of a session be made available after the session? Do you devalue the
benefit of attending the webinar if it’s made available to everyone? Should it only be sent to
attendees as an incentive to attend?
 Having a recording meant Dan was able to reflect on the content of presentation and consult
with his colleagues.

 

 

Closing the gap! Bringing together students studying at different campuses using Blackboard Collaborate

Kate Fletcher, Sue Slade, Kevin Flint, Raj Vaiyapuri, Wee Kiat Ong, School of Chemistry, Food and Pharmacy; Pharmacy

Context

MPharm Programme: Introduction to Professionalism and Practice

Undergraduate (UG) students, Part 1

Number of participants in sessions: 20 (9 in the UK and 11 in Malaysia)
Session length: 60 minutes

Description

 Part 1 students studying the MPharm course at both the Reading and Malaysia campuses were
brought together using Blackboard Collaborate to compare Pharmacy Practice in each country.
 Kate wanted to encourage crossover between campuses and for students to get to know each
other before the Malaysian students came over to study in the UK for Part 3.
 Students based at each campus logged in to Collaborate on individual computers with a
headset.
 Both groups of students were in the Clinical Skills Suite on each campus with laptops and
headsets.
 Staff supported students in the physical rooms to get them settled and set-up.
 The session was designed around set discussion activities and students separated out into
groups that included students from both campuses, using the ‘Breakout room’ feature.

Impact

 Collaborate provided an effective way for students studying at different campuses to learn
together and begin to build relationships.
 Close cooperation was needed between the UK and Malaysian staff to set up the session.
 Students quickly picked-up how to use the tool, were using the Chat tool without prompting
and easily able to undertake the tasks in the breakout rooms.
 The session was activity based and students were discussing with each other. This made best
use of the technology to facilitate communication.
 There were good levels of interaction between students using the audio and video. However,
the first time people use the system interaction can initially be awkward.
 Some cultural differences were perceived. Malaysian students were quieter in the
conversations and UK-based students tended to lead.

Thoughts and reflections

 Kate and Sue were thoroughly prepared for the session and had rehearsed how to use the
‘breakout rooms’ and written a session plan with timings.
 Don’t expect to get as much done as you would in a face-to-face session or allow more time for
activities in this environment.
 As the students were located in the same room together they were spread out to minimise the
transfer of noise between them when talking. Pharmacy had a large enough room to allow this.
Feedback from students indicated they could easily take part from home.
 Pharmacy needed to purchase suitable headsets that could be re-used by different students.
Allow sufficient time to arrange ordering from IT.
 Make sure Chrome is installed on the University computers students are going to use.
 There was a significant investment of time and a learning curve to set up the session, as this
was the first time they had attempted this. Future sessions should be easier to facilitate.
 It’s not yet possible to save what has been written on the whiteboards in the breakout rooms.

(Use the PC – Microsoft Clipping tool https://support.microsoft.com/engb/help/13776/windows-use-snipping-tool-to-capture-screenshots
or MAC keyboard shortcut to take a screenshot of the whiteboard.)

 

Using Blackboard Collaborate for small group tutorials with distance learning students

Adrian Aronsson-Storrier, School of Law                                                                                                                             a.m.storrier@reading.ac.uk

Context

LLM International Commercial Law (Distance)

Description

 Adrian held small group seminars with groups of around 5 students per online workshop.
Workshops were scheduled in all of the distance LLM modules, and ran weekly through the
Spring and Autumn terms. Collaborate was also used for individual dissertation supervision
sessions.
 These were Postgraduate Masters level distance learning students enrolled in a range of
optional LLM modules. Students attended from across the UK and the world.
 The Law School already offered online workshop sessions using a competing webinar product
(Adobe Connect). This software was complex for students to use, not supported centrally by
the University and was paid for from the School’s budget. We sought to investigate alternative
web conferencing solutions that would be simpler for our students whilst maintaining
equivalent functionality (slide sharing, chat, whiteboard etc).
 Blackboard Collaborate was chosen to replace Adobe Connect as it was simpler for students to
use (a more straightforward interface reduced initial student training time, the integration into
Blackboard made it simpler for students to log in and participate).
 Preparation was similar to distance workshops previously delivered with the earlier Adobe
Connect web conferencing tool. For some workshops slides were prepared, in others a series
of tutorial style questions were circulated to students in advance for discussion.
 After giving students an initial training session, delivering a class on Collaborate took no more
effort than delivering an equivalent session in an on campus module.

Impact

 Students quickly adapted to Collaborate. They made frequent use of the chat function and the
‘raise hand’ function, particularly in larger groups where many students wished to contribute to
a discussion.
 Student’s enrolling in the distance LLM are required to have access to their own computer,
headphones and internet connection.
 From a support perspective, the move to Collaborate required less ongoing staff and student
training than our previous web conferencing software – once set up on Blackboard it was simple
for students and staff to access Collaborate sessions for their weekly workshops.
 Blackboard Collaborate achieved everything we had previously delivered to students using
Adobe Connect. It had the advantage of being simpler for students to use, and the blackboard
integration made connecting to the sessions simpler.

Thoughts and Reflections

 Lecturers in the school of law tended to use Collaborate from their homes (distance workshops
are often scheduled outside core hours, to accommodate students in diverse time zones). This
required staff to have sufficient equipment (laptop, headphones or a headset).
 One challenge – which often impacts distance learning when working with students in less
economically developed nations – was issue of the student’s poor internet connection
impacting sessions. At times students (particularly in Africa and the Middle East) had poor
internet connections which prevented full video streaming. While the software does allow
students to participate by providing streaming audio only, this is less immersive for the student.
 Ensure that all participants are making use of headphones or a microphone headset. If students
rely on computer speakers there will often be some level of echo introduced into the web
conference, which can be distracting. Students without headphone should be encouraged to
mute their microphones when not speaking.
 Provide students with an introductory session on the software before beginning online
instruction. We used a general online induction day for students as a trial, allowing them to test
that the software worked and giving them time to learn the functionality before being required
to use it in class.