Take-home exam

By edwardwhite Comment Closed

Stuart Lakin, Law
s.j.lakin@reading.ac.uk

Overview

In a Part Two Law module, Public Law (LW2PL2), we have moved away from the conventional exam to a take-home exam. We publish the exam paper on Blackboard at an arranged date and time. We give the students approximately 48 hours to complete and submit their answers electronically.

The impact has been entirely positive as compared to the old exam approach. Students prefer this format. The quality of their answers is markedly better. The results are better, and are consistently among the highest of all Part Two modules.

Objectives

  • To ensure that work produced in the exams is presented to a professional standard.
  • To allow students the opportunity to provide greater intellectual depth in their answers, and allowing the ability for independent research to form part of the assessment.
  • To have students demonstrate time management, in order to allow them to effectively complete their take-home exam while revising for their other examinations.

Context

We had three reasons for undertaking the activity:

First, we reasoned that LW2PL2 was better suited, pedagogically speaking, to the new format. The subject-matter is theoretical, and we assess by essay only (as opposed to by problem questions). We look for deep understanding of the issues rather than an ability mechanically to apply memorised rules. The take-home format encourages an independent research mindset.

Secondly, we thought it valuable to provide some variety in the way that Part Two students are assessed. The assessment across the Part Two modules had hitherto been by conventional exam only. Whatever the merits and demerits of the traditional exam, it can be refreshing for students to experience some other form of assessment.

Thirdly, we responded to the University call for alternative assessment. On pragmatic grounds, the take-home exam frees up room space and reduces complex timetabling requirements.

Implementation

We prepared the first cohort of students by giving them a mock take-home exam in lieu of their usual non-assessed essay. We asked them to prepare an answer to a question as if they were preparing for the exam itself. We have continued this practice ever since.

In addition, I prepared a detailed explanation of our rationales and expectations for the take-home exam, and provided this to the students. This document exists to inform students of the benefits and the opportunities provided by the format, and also ensures that they fully appreciate the assessment criteria of the format. I talk through this document with the students throughout the year.

Impact

In short, the activity has been highly successful. I believe that colleagues are considering this format for their own modules. By having students word process their exam answers, a lot of the recognised disadvantages of handwritten answers (handwriting often being slow and uncomfortable, and producing results that are messy and poorly legible, as well as the anxiety caused by these disadvantages) can be avoided. It is also easier for students to structure their essays.

By having the take-home exam scheduled during the University exam period, it is important that students manage their time effectively in completing the exam. Students are made aware that the assumption when marking is that they will have spent approximately two hours answering each question: this allows them more time than a conventional exam, but also allows time for students to make space for other commitments they might have, such as revision for other exams.

Above all, we have found that the format is a better way of encouraging scholarly engagement with the module content. We emphasise in our rationales/expectations document that the format has an element of independent research.

The level of success of the activity was unexpected. The first cohort of students to do the take-home exam were nervous and rather distrustful of the activity. Happily, they passed their positive experience down to the next year’s cohort, and that pattern has continued ever since.

Reflections

In my view, the take-home exam format treats students as independent thinkers in a way that the conventional exam does not. The emphasis is on the quality of argument and research rather than on memory skills and the ability to perform under pressure. Having said that, the new format does not entirely dispense with the latter types of skills – there is still a deadline, and students will still need to revise in advance.

There were admittedly risks involved in introducing this new format. LW2PL2 is an extremely important, compulsory module which counts towards the final degree. With hindsight, it may have been more prudent to experiment with this format in a Part One module. On the other hand, we put a great deal of thought into the format, and communicated well with the students. In these respects, we minimised the risks.

Follow up

The activity has remained largely the same as it began. We have experimented with changing the publication and submission times for the exam. We originally published the exam at midnight. This led to many students staying up all night to work on the paper. We now publish the exam at 9 am.

Final Year Group Based Research Projects

By edwardwhite Comment Closed

Professor Elizabeth Page and Dr Philippa Cranwell, Chemistry, Food and Pharmacy
e.m.page@reading.ac.uk
Year of activity: 2015-16

Overview

Group-based research projects have been introduced into the BSc Chemistry programme for final year students. Small teams of students investigate different aspects of a research problem, each working on a separate strand. The results are combined and overall conclusions drawn. The team-based approach more closely resembles the nature of research in the chemical industry. The approach can be translated to many other disciplines.

Objectives

  • To provide final year students with the opportunity for open-ended investigative laboratory research.
  • To work as a team to plan and design a suitable approach and experiments to explore the problem.
  • To carry out original research and collate and analyse results.
  • To draw conclusions and present the results both orally and as a dissertation.
  • To develop a variety of key transferable skills required for the workplace.

Context

All accredited Chemistry programmes must contain individual independent investigative work, historically in the form of a final-year research project. Since the rapid expansion of chemistry undergraduate numbers, many departments have moved from laboratory-based projects to literature reviews or short, open-ended practical work. Group projects provide an alternative approach where undergraduates carry out a worthwhile chemical investigation, with the potential of yielding useful results within the restricted time, and with the limited resources available.

Implementation

A Teaching and Learning Development Fund (TLDF) Grant in 2015 allowed us to appoint two undergraduate students to investigate some potential ideas for research projects over the summer of 2015. The students carried out initial trials into a series of research topics in the broad areas of inorganic, organic, physical and analytical chemistry. On the basis of these preliminary investigations a short briefing sheet was drawn up for each research question, to be used as a starting point for the teams.

Final year students on the BSc Chemistry and BSc Applied Chemistry (NUIST) programmes were invited to select areas of preference in chemistry for their final year project. Students were organised into teams of 3-5 students on the basis of project preferences and undertook two short (five week) projects, the first of which acted as a trial run to allow students to become familiar with an independent research environment. Each team was allocated an academic supervisor to whom they reported their results weekly. During the final week of each project team members discussed their results and prepared a presentation. Students were given feedback on the first presentation to help inform the second. The second project was written as a formal report, with each student writing up their individual investigations and the whole team contributing to the introduction and final discussions and conclusion.

Students were assessed on the basis of their individual laboratory notebook, their oral presentation and project report. They were asked to evaluate their peers’ contributions under a variety of categories to produce a factor which could be used to scale any group component marks.

Impact

In 2015-2016 a total of 12 team-based projects were carried out in 4 different research areas. As the topics were re-visited (i.e. the same topic used more than once), the second group of students were able to carry on the investigation from where the first group finished.

All projects were successful in producing results that the students were able to analyse and discuss. The value of the results to the research question varied significantly with the team and the nature of the project. Students were not penalised if they worked in a project area that did not easily yield positive results: they were advised that their grades depended upon their input into the project and their oral and written communication skills in presenting the project. In the majority of cases the teams worked well to plan and execute experiments that led to conclusive results.

Although the numbers were relatively small in 2015-2016, the team-based approach reduced academic supervision and training time, as one staff member could supervise a team of students. More results were obtained from the team-based approach than when students worked independently. The research questions had to be selected carefully and some preliminary work done, but despite this some of the projects yielded new results that are publishable. Students improved their team working skills significantly and have ample experiences to discuss at interviews.

Reflections

The success of each group project depended to a large extent on the individual supervisor and the group dynamics. Ownership of the project by the supervisor led to more successful outcomes and better group dynamics. It was observed that groups of 4 students seemed to work better than 3 or 5, as research problems often break down to comparing A against B, and therefore workload could be more easily divided. Interestingly, students requested one long project in future rather than two short ones because they felt that with a long project they could really make a meaningful impact with their work.

As the project reports were to be submitted shortly before the exam period, some students were anxious to complete their contributions in good time and found it difficult to work with their peers who had a more relaxed approach. Because of the high weighting (40 credits) on the project, we will require individual project reports in future. In addition, combined group reports were difficult to assess fairly, even with peer evaluation.

Links

The work was presented at the 2nd Enhancing Student Learning Through Innovative Scholarship Conference meeting in June 2016.

Flipping the classroom in Meteorology

By edwardwhite Comment Closed

Dr. Andrew Charlton-Perez, School of Mathematical and Physical Sciences
a.j.charlton-perez@reading.ac.uk
Year of activity: 2015/16

Overview

12484A flipped learning approach to teaching the part 3 and part 4 module, ‘The Global Circulation’ (MT38A/4YA) in Meteorology was developed and tested. This approach was very successful, encouraging students to apply complex ideas to real-world problems.

Objectives

  • Develop a new set of learning resources which could be used both in a flipped learning or traditional lecture based delivery.
  • Test and evaluate if teaching in a ‘flipped learning’ style improved student engagement and higher-level learning.
  • Implement authentic assessment that models the real-world process of enquiry and peer-feedback.

Context

I’ve been teaching this module for eight years adding some enquiry-based learning elements around four years ago. While it has always received good student feedback, a colleague who moderated my exam scripts last year made me consider if students were as actively engaged with the module as I had previously thought. Looking at student work it was clear that while students could remember and reproduce sophisticated concepts and mathematical derivations, their ability to apply this knowledge to unfamiliar situations was limited.

Implementation

Prior to the course I developed a significant new set of learning resources for the course:

  • 21 short videos (between 4 and 6 minutes long) targeted at difficult concepts.
  • 13 online quizzes of ten questions in Blackboard Learn.
  • 24 new learning activities linked to the research literature which students completed during class. 12 of these activities were supported by simple numerical models developed from scratch in open-source Python code.

The course was delivered to students in Spring 2016. The first time the class met as a group I explained the flipped classroom idea and we negotiated an approach to learning.

Following this first meeting, the class operated in three, three-week units. During the first two weeks of each unit, students studied notes and videos in their own time and then completed the on-line quizzes (with instant feedback) prior to attending the next class. In class, students had a choice of two learning activities. Students worked with each other in small teams and with me to complete the problems, writing notes in a rough lab book.

In the third week of each unit, students prepared a more formal write-up of one of the problems as summative assessment for that unit. In extracting information from their lab book, students needed to think about how to frame the problem by stating a hypothesis they wished to test and put the work they had done in the context of the current research literature.

In class, students exchanged their work and gave peer-feedback to each other, before completing the formal write-up with the chance to ask additional clarification questions.

Impact

The change to a flipped learning style had a transformative impact on the module; Student feedback highlighted the benefits of the investment in a number of ways including that they enjoyed the flipped learning approach:

  • ‘The structure of the module was the best out of my three years of university, flipped classroom should be done more’
  • ‘Always felt engaged with the lectures thanks to a different learning style’
  • ‘… I could run over the tricky concepts in more detail on a 1-to-1 basis with the lecturer. Often the same question applied to others and the class environment allowed group discussions which really enhanced the learning in a relaxed and productive way.’

And that their learning was improved:

  • ‘…brought my attention to active research areas at the front of study – it got me very interested in the exercises.’
  • ‘Class room discussions made me learn more than in a lecture style class’
  • ‘It isn’t an easy module, but it is very rewarding’

Reflections

Teaching in a flipped learning style also had a significant impact on my own enthusiasm for teaching the course because the improvement in student learning and engagement was tangible. The image shows an example of student work produced on a white board by a small group during one of the class sessions, applying ideas from the core course material to a recent research paper. It was extremely exciting to see students applying complex ideas in this way and succeeding in writing high-quality research reports on their work.

The flipped learning approach also challenged me to think more deeply about the material because I needed to produce engaging and manageable problems for students to work on.

It was also very rewarding to see how much students made use of the new teaching materials I developed. By monitoring the use of the videos on Blackboard I could see that typically the videos were viewed between 100 and 150 times by the ten students on the course, indicating how important these videos were for student learning. Based on student feedback, short and engaging videos encouraged repeated viewing. I also included music in the videos, often with an oblique reference to the content and students commented that they enjoyed this element of fun! At the end of the course, students requested the ability to download and keep copies of the videos (this functionality is currently not available in Blackboard).

An important part of producing the videos was to also provide transcripts to ensure they were accessible for all students and this took quite a bit of time in addition to video production (which was relatively straight-forward).

The on-line quizzes had a completion rate of 80% with average marks above 70% for most students. As with the videos, the high level of engagement with these materials suggests that they were of an appropriate length and level of challenge for students (pitched so that a student who had studied the notes and videos could answer most questions without further detailed application).

Using an enquiry-based learning assessed work activity to enhance assessment and feedback

By edwardwhite Comment Closed

Professor Paul Almond, School of Law
p.j.almond@reading.ac.uk

Overview

9367A project to encourage students enrolled on a Part Three module within the School of Law, Criminology (LW3CRY), to develop a more sophisticated understanding of the links between criminological theory and policy, through a redesign of the Assessed Work project contained within the module. Results have suggested that the project was successful in achieving its objectives, and there were additional, unexpected benefits.

Objectives

  • Rework the assessment for the module so that students are better able to gain an understanding of the links between criminological theory and policy.
  • Utilise the principles of enquiry-based learning.

Context

One of the established learning outcomes for LW3CRY requires students to: ‘Apply theoretical criminological concepts to practical issues within the field of crime, law and social control’. Students had in the past tended to struggle to make meaningful connections between these two things, and to take a very simplistic view of the theory-policy relationship.

Implementation

The assessment project for the module was reworked so that it utilised principles of enquiry-based learning and required students to do something with the substantive material that they engaged with during the course. Students were set the task of producing a report for a fictional client, the ‘Minster for Justice’, recommending how a budget of £100m should be spent (on policy choices drawn from a list of available options). This open-ended requirement forced students to define their own terms for answering the question, in that they had to construct and apply the theoretical framework that explained and justified their choices, and settle on a series of recommendations that they put forward. As there is no ‘right answer’ students engage with the process of choosing and justifying rather than reaching a specified ‘correct’ conclusion. The report produced at the end of the project had to clearly explain choices with reference to theory and evidence.

The problem given to the students as the basis of the project was ‘client-centred’, in that they were supposed to be working for the Minister. To this end, the launch document and project materials were formatted in the style of official Government documents and the launch was in the form of a video podcast from the Minister (played by an actor). Project updates were also in the form of video and audio podcasts on Blackboard Learn, and the Minister had his own email address from which to send updates and respond to enquiries. Finally, in order to provide some realism in the ‘client-facing’ research relationship, some details and features of the project were staged so as to be changed or updated as the project progressed.

Impact

The average mark for the assessed work project rose from 60.9% in the previous year to 62.8% when the Project was implemented. In addition, subsequent performance in the examination for the module also improved from 60.1% to 61.2%, demonstrating that the gains in terms of the learning outcomes had carried across from the initial assessment activity.

Reflections

In order to allow this assessment change, the module convenor created the materials and released them via Blackboard Learn, responded to enquiries and provided updates, and then assessed the assessed work reports. Although this involved quite a lot of initial work, the materials and design are reusable meaning that there is a diminishing workload attached to the Project as it is reused in subsequent years.

A couple of problems arose in relation to implementing the project. Firstly, some students were unclear as to what the requirements of a ‘report’ were, and how this should differ in style, structure, and approach from an essay. Despite reassurances that ‘report’ simply meant ‘focused on providing a take-home message about the recommended policies’, they found this terminology confusing. In subsequent versions of the project, more guidance has been provided on what this requires. Secondly, the scope of the project was quite broad (in that students could end up writing small amounts about a large number of policy items), meaning that they were not able to demonstrate the depth of understanding required. Tweaks in the costs of individual items have been introduced to combat this.

Overall, this was successful, and has been utilised in subsequent academic years. It is effectively ‘future-proofed’ in that minor changes to costs, policy choices, and details allow for the materials and project to be reused again. It also involves a very specific problem, reducing opportunities for plagiarism and ‘essay-buying’. An unexpected benefit was the way that this assessment could dovetail with the rest of the course; the use of Blackboard Learn to communicate and store materials increased through-traffic on the course page generally, and it also gave a good focus to subsequent revision classes (the Minister delivers generic feedback and gives suggestions for improvement). The feedback available for this project is easily adaptable in terms of explaining the specific criteria and requirements of the examination; the style of exam question set has been altered in order to achieve ‘constructive alignment’ and ensure that the skills learned in this project are of use in the subsequent assessment.

Improved Neural Network assessment by staged laboratory practicals

By edwardwhite Comment Closed

Professor Richard Mitchell, School of Systems Engineering
r.j.mitchell@reading.ac.uk

Overview

6470Adjustments were made to teaching, assessment, and feedback in a Part Two module within the School of Systems Engineering, Neural Networks (SE2NN11), successfully using three-staged laboratory practicals in order to encourage students to use neural networks on a ‘real world’ application. Making these changes saw an increase in the number of students successfully producing a neural network.

Objectives

  • Increase the number of students successfully producing a neural network.
  • Provide greater and prompter feedback to students.

Context

The major assessment for SE2NN11 requires students to write a program to implement a particular neural network and to then use that network on a ‘real world’ application. The students demonstrated their network by the end of the autumn term, where verbal feedback was given, and they then applied it to the real world problem of their choice in the next term. Previously, students had difficulty with the first stage, and so fewer moved on to the (more interesting) second stage, with only around 75% of students submitting a report.

Implementation

During the pilot year (2009), the tasks associated with writing the neural network were carefully divided into three, and three associated 90 minute lab sessions were organised, two weeks apart, for the work. The lecturer plus two laboratory demonstrators were available to provide help to the students at these sessions.

For each session, a Microsoft Word template file was provided, and the students copied and pasted relevant program output or small parts of the program (functions) into the appropriate parts of the template. A simple marking scheme was associated with each part, worth 30 marks: typically students could get 0, 1 or 2 for a piece of code plus 0 or 1 for comments; or a student could get 0 or 1 depending on whether the program output was correct. There was also space for comments to be written.

These files were then submitted to the lecturer who circled the relevant mark for each part and added relevant comments. Detailed feedback was thereby generated very easily and very quickly. The aim was to give feedback within a week of the session, allowing the student a further week to make any necessary corrections on one part before starting the next part of the program. In fact, the first week’s work was marked within two days.

Impact

Each year since this scheme has been introduced, around 95% of students have been able to produce a neural network, a significant increase in submissions.

Reflections

The impression obtained in the pilot was that a greater proportion of students had a working neural network compared with previous years, suggesting a great success of this scenario. As such the scenario has been used each year since with some changes to the templates (and to the program to help reduce plagiarism between years).

One problem is that the structure of the program is so tightly defined that there is little scope for variation in code – hence copying is difficult to detect. This is partly addressed by requiring the student to comment their code and to discuss the object-oriented aspects of their program in the final report. In addition students were expected to do experiments in their own time to investigate the effects of changing specific parameters in the program. The instructions for the final report were made clearer to try to ensure this happens.

One disadvantage of the approach is that it discourages independent thought more than is ideal. The much increased submission rate, however, is encouraging.

The important aspects of this scheme are the division of the project into suitable, easily marked sub-tasks, the extra support provided in the development of the program and the inherent feedback between sessions.

Follow up

Following the pilot year, it was realised that some functions were more complex than others, so the marking scheme was changed so that an appropriate number of marks for the code and comments was available for each function.

From 2015, with the move to online submission, students upload their document to Blackboard Learn, where the work is readily marked. Rather than circling the marks, the ‘insert text’ option is used to allow the marks to be entered. Comments on errors or suggestions for improvements are also easily added in an appropriate context.

The second part of the assessment requires the student to apply their neural network to a ‘real world’ problem of their choice, to see if the network can learn that problem. In effect the students are researching whether a neural network is appropriate. Given that, rather than asking the students to write a report on their work, they are now asked to present their research in the form of a four page conference paper. This tests them with a new skill, complementing the report writing skills they use elsewhere. This innovation has also proved successful.