Remaking Marking Conference poster, displaying birds migrating in an arrow formation

On Tuesday 4th September 2018, 74 colleagues representing 34 UK Higher Education institutions attended the EMA Programme, ‘Remaking Marking Conference’ in the Meadow Suite. The conference enabled colleagues from across the sector to come together to share good practice and to discuss challenges and opportunities involved in the implementation of electronic assessment.

A series of guest speakers presented a summary of their institution’s approach to embedding EMA and a discussion of the effectiveness of this approach. Dr Rachel Maxwell, the Head of Learning & Teaching Development at University of Northampton, presented first and delivered a brief history of implementation at the University of Northampton followed by an overview of the pedagogical transformation that EMA delivered. Dr Maxwell included debate on a number of issues including the opportunity created by electronic management of assessment for colleagues across the sector to reflect on their assessment practices asking whether, for example, exams were more of a habit than geared towards the delivery of learning outcomes.

Dr Emma Mayhew, the Academic Director of the EMA programme at the University of Reading, then explored the normalisation of online assessment across the sector, drivers for change and the various barriers faced by institutions when designing and implementing EMA.  The presentation explored new online marking opportunities including Quickmarks, audio feedback and rubrics and concluded by indicating that, whilst some aspects of EMA represent a complex challenge for the sector, several providers have been able to address key concerns and begin to make full use of new functionality to enhance the assessment experience.

Stuart Hepplestone, representing Sheffield Hallam University, began his presentation by outlining the assessment journey and reported on a number of  technical solutions, bespoke customisations, and the processes undertaken by Sheffield Hallam to implement the University’s vision.  He then addressed the challenges and lessons learned in relation to institutional culture, stakeholder attitudes and technology since full implementation of online management of assessment at Sheffield Hallam in September 2017.

Dr Pete Lonsdale, Learning Technology Manager from Keele University, described the online platform he created for administering practical assessments in Nursing and Midwifery. This platform streamlined the assessment process, offered new methods of providing feedback, and highlighted areas of practical support that required refinement or clarification. The presentation included comment on the need to adapt working methods and practices to new technologies, and the potential need to revisit or revise traditional pedagogic principles (plus some Lego!).

The final speaker was Dr Simon Kent from Brunel University who discussed whether digital exam assessment constitutes ‘evolution’ or ‘revolution’. As an institution, Brunel is confident as it explores the deployment of digital examination using Wiseflow. A limited trial has been conducted at Brunel involving software that facilitated Multiple Choice Questions (MCQs) and short essay based responses. Several attendees were particularly interested in the information about digital exams that was included in several papers and it is clear that it is a developing field for several UK Higher Education institutions.

The conference included space for networking and for the sharing of good practice; after lunch, attendees answered an interactive questionnaire on Mentimeter where they were asked to rate their knowledge of learning technology and online marking processes. The questionnaire revealed that knowledge of online exams and system integration was comparatively low compared to online submission and online marking.

The conference concluded with an enthusiastic whole-room discussion where the speakers responded to a variety of questions that has been posted on Mentimeter. This final section of the day was devoted to discussion of good practice within a wholly collegiate atmosphere.

Attendees were asked to complete a questionnaire rating the organisation and impact of the conference: 100% of responses rated the event ‘good’ or ‘excellent’ and 97% of responses indicated that the conference was likely to enhance their own assessment practice. The aim of the conference was to overcome EMA barriers and make the most of new functionality to deliver a better assessment experience for all stakeholders, and the feedback, together with the atmosphere of the day, suggest that this aim was fully achieved.

Check out the full Twitter Feed for the event for more on #RemakingMarking.

Presentations

Windows of Opportunity: Overcoming EMA barriers and making the most of new functionality to deliver a better assessment experience [Click to open]

Bio: Emma Mayhew is Associate Professor of Politics and International Relations and Academic Director of the Electronic Management of Assessment Programme at the University of Reading, UK. Her research focuses on the use of technology enhanced learning to facilitate student engagement, learning, attainment and satisfaction. She focuses, in particular, on developments surrounding multi-media learning, screen capture, video feedback, online assessment and learning analytics.

Abstract: Electronic Management of Assessment, as a transformative major change process, is attracting considerable attention from HE providers, particularly given the increased focus on the quality of teaching and learning provision within the sector. This presentation outlines how a number of institutions have addressed key challenges, specifically surrounding the adoption of online submission, feedback and grading, within the sector in the last five years. It draws on a range of reports produced by JISC, as part of their major two year EMA project, regular HeLF (Heads of E-Learning Forum) EMA annual survey reports, institutional project reports and project blogs together with a series of interviews held with a number of providers over the last two years. It explores institutional responses to the complexities of change design, stakeholder management, policy and process alongside technical issues surrounding systems integration. The presentation moves on to explore new opportunities open to adopters, facilitated by the shift to online assessment, and how examples of innovative practice have impacted on both staff and student experiences of the assessment and feedback cycle. It focuses, in particular, on the use and impact of new marking tool functionality including Quickmarks, audio feedback and GradeMark Reporting but also new ways of working with assessment analytics. The presentation will conclude that, although some aspects of EMA represent a complex challenge for the sector, a number of providers have been able to address key concerns and start to make full use of new functionality to enhance the assessment experience.

New dog, new tricks: An e-assessments journey [Click to open]

Bio: Dr Pete Lonsdale is Learning Technology Manager at the School of Nursing and Midwifery at Keele University, working on developing innovative approaches to electronic assessment, student engagement, and building online content. Pete has worked as a learning technologist for over eight years, supporting the use of existing platforms such as VLEs, portfolio systems, and assessment tools. As a software developer he has also built several bespoke systems to support learning and teaching. His particular interests are in how technology can help learning and assessment escape traditional classroom environments and help provide flexible opportunities for students to engage with education.

Abstract: Technology offers us new tools to enhance and transform assessment in a range of areas, but the process of transitioning from traditional to electronic assessment can yield unexpected discoveries about the processes involved. I will describe the introduction of a bespoke online platform designed for administering practical assessments in Nursing and Midwifery. This platform streamlined the assessment process, offered us new affordances and new ways of providing feedback, whilst simultaneously highlighting areas of practice that needed to be refined and clarified. We also found that with new technology comes new ways of working, but not all of those are necessarily what you are looking for, and might not fit with pedagogical ideals.

"A brief history of EMA and an overview of pedagogical transformation at Northampton" [Click to open]

Bio: Dr Rachel Maxwell (@DrRachLTB) is Head of Learning and Teaching Development at the University of Northampton where she is currently leading a number of projects supporting the student experience, including improving the first year experience and the development of a framework of graduate attributes embedding employability and Changemaker skills across our curricula. Her work also focusses on developing assessment and feedback practices suitable for the 21st century, promoting academic integrity and supporting staff to introduce innovation into their own pedagogic practice. Rachel previously worked as a Learning Designer, supporting staff to redesign modules and programmes as part of an institutional move to blended learning, building on her work as a lecturer in both HE and FE.

Abstract: 

The University of Northampton introduced its SaGE project (Submitting and Grading Electronically) in 2012. This presentation charts the development and implementation of the project in the light of strategic imperatives and drivers to both policy and practice, exploring the challenges and barriers to adoption along the way. It also considers the value of adopting an evidence-based approach to delivering project objectives, with a strong level of support that focused on ensuring academic staff agency and building institutional capability. The success of this approach meant that the project was deemed completed after 3 years with EMA fully embedded in both policy and practice.

Alongside this project, and in an initiative unique to the sector, the University has completely revised its approach to learning and teaching and undertaken a thorough review of the Framework provisions on assessment. These changes have prompted further changes to marking practice and … as we look ahead to 2018 … the possibility to digitise the final piece of the assessment jigsaw – end-of-year paper-based exams. The presentation will conclude with a look at developments to assessment in general, and EMA in particular.

Her PhD is in human rights law, medical ethics and torture prevention, although she normally just tells her students she has a PhD in torture and not to mess!

Digital Assessment: Evolution or Revolution? [Click to open] Bio: I am a Reader and Director of Learning and Teaching in the Department of Computer Science.  I have a combination of academic and commercial skills from my experience as a university lecturer and as a software architect in the financial services industry.  I have been responsible for the first large-scale deployment in the UK of a Bring-Your-Own-Device (BYOD) approach to examinations in the Department of Computer Science at Brunel.  I am currently involved in rolling this out across the University, and considering the longer term implications of moving away from paper and towards digital exams. Abstract: The presentation offers a background story of how Brunel came to make the move towards Digital Exams, where we are now, and what we see for the future of Digital Exams at Brunel and in UK HE.
Assessment Journey: enhancing the educational experience and institutional effectiveness of assessment and feedback [Click to open] Bio: Stuart Hepplestone is Senior Lecturer in Digital Capability at Sheffield Hallam University. Stuart has supported the use of technology in learning and teaching for over 19 years. His main area of work and research interest is in promoting the appropriate use of technology in the process of assessment and feedback. Abstract: In this session, Stuart will report and reflect on how Sheffield Hallam University has implemented a consistent, seamless and improved assessment experience for students and staff supported by efficient processes and integrated systems. The University undertook a three-year ‘Assessment Journey Programme’ working with its stakeholders to identify good practices and the changes needed in order to shape the institutional vision for assessment and feedback and the technology required within it. Stuart will:
  • share the University’s learner-focussed principles for assessment along with the resulting lifecycle (after Ferrell & Gray, 2013) developed to describe each stage of the assessment process. The lifecycle formed the basis for the assessment framework that provides stakeholders with a single point of access for assessment guidance and resources, policy and regulatory information;
  • report on the technological solutions that have been exploited, the bespoke customisations that have been developed, and the processes that have been developed to achieve and implement the University’s vision;
  • tell the stories, challenges and lessons learned in relation to institutional culture, stakeholder attitudes and technology since full implementation of online management of assessment at the University in September 2017.
References: Ferrell, G. and Gray, L. (2013). Electronic management of assessment: Using technology to support the assessment lifecycle, from the electronic submission or assignments to marking and feedback.