Using an enquiry-based learning assessed work activity to enhance assessment and feedback

By edwardwhite Comment Closed

Professor Paul Almond, School of Law
p.j.almond@reading.ac.uk

Overview

9367A project to encourage students enrolled on a Part Three module within the School of Law, Criminology (LW3CRY), to develop a more sophisticated understanding of the links between criminological theory and policy, through a redesign of the Assessed Work project contained within the module. Results have suggested that the project was successful in achieving its objectives, and there were additional, unexpected benefits.

Objectives

  • Rework the assessment for the module so that students are better able to gain an understanding of the links between criminological theory and policy.
  • Utilise the principles of enquiry-based learning.

Context

One of the established learning outcomes for LW3CRY requires students to: ‘Apply theoretical criminological concepts to practical issues within the field of crime, law and social control’. Students had in the past tended to struggle to make meaningful connections between these two things, and to take a very simplistic view of the theory-policy relationship.

Implementation

The assessment project for the module was reworked so that it utilised principles of enquiry-based learning and required students to do something with the substantive material that they engaged with during the course. Students were set the task of producing a report for a fictional client, the ‘Minster for Justice’, recommending how a budget of £100m should be spent (on policy choices drawn from a list of available options). This open-ended requirement forced students to define their own terms for answering the question, in that they had to construct and apply the theoretical framework that explained and justified their choices, and settle on a series of recommendations that they put forward. As there is no ‘right answer’ students engage with the process of choosing and justifying rather than reaching a specified ‘correct’ conclusion. The report produced at the end of the project had to clearly explain choices with reference to theory and evidence.

The problem given to the students as the basis of the project was ‘client-centred’, in that they were supposed to be working for the Minister. To this end, the launch document and project materials were formatted in the style of official Government documents and the launch was in the form of a video podcast from the Minister (played by an actor). Project updates were also in the form of video and audio podcasts on Blackboard Learn, and the Minister had his own email address from which to send updates and respond to enquiries. Finally, in order to provide some realism in the ‘client-facing’ research relationship, some details and features of the project were staged so as to be changed or updated as the project progressed.

Impact

The average mark for the assessed work project rose from 60.9% in the previous year to 62.8% when the Project was implemented. In addition, subsequent performance in the examination for the module also improved from 60.1% to 61.2%, demonstrating that the gains in terms of the learning outcomes had carried across from the initial assessment activity.

Reflections

In order to allow this assessment change, the module convenor created the materials and released them via Blackboard Learn, responded to enquiries and provided updates, and then assessed the assessed work reports. Although this involved quite a lot of initial work, the materials and design are reusable meaning that there is a diminishing workload attached to the Project as it is reused in subsequent years.

A couple of problems arose in relation to implementing the project. Firstly, some students were unclear as to what the requirements of a ‘report’ were, and how this should differ in style, structure, and approach from an essay. Despite reassurances that ‘report’ simply meant ‘focused on providing a take-home message about the recommended policies’, they found this terminology confusing. In subsequent versions of the project, more guidance has been provided on what this requires. Secondly, the scope of the project was quite broad (in that students could end up writing small amounts about a large number of policy items), meaning that they were not able to demonstrate the depth of understanding required. Tweaks in the costs of individual items have been introduced to combat this.

Overall, this was successful, and has been utilised in subsequent academic years. It is effectively ‘future-proofed’ in that minor changes to costs, policy choices, and details allow for the materials and project to be reused again. It also involves a very specific problem, reducing opportunities for plagiarism and ‘essay-buying’. An unexpected benefit was the way that this assessment could dovetail with the rest of the course; the use of Blackboard Learn to communicate and store materials increased through-traffic on the course page generally, and it also gave a good focus to subsequent revision classes (the Minister delivers generic feedback and gives suggestions for improvement). The feedback available for this project is easily adaptable in terms of explaining the specific criteria and requirements of the examination; the style of exam question set has been altered in order to achieve ‘constructive alignment’ and ensure that the skills learned in this project are of use in the subsequent assessment.

Improved Neural Network assessment by staged laboratory practicals

By edwardwhite Comment Closed

Professor Richard Mitchell, School of Systems Engineering
r.j.mitchell@reading.ac.uk

Overview

6470Adjustments were made to teaching, assessment, and feedback in a Part Two module within the School of Systems Engineering, Neural Networks (SE2NN11), successfully using three-staged laboratory practicals in order to encourage students to use neural networks on a ‘real world’ application. Making these changes saw an increase in the number of students successfully producing a neural network.

Objectives

  • Increase the number of students successfully producing a neural network.
  • Provide greater and prompter feedback to students.

Context

The major assessment for SE2NN11 requires students to write a program to implement a particular neural network and to then use that network on a ‘real world’ application. The students demonstrated their network by the end of the autumn term, where verbal feedback was given, and they then applied it to the real world problem of their choice in the next term. Previously, students had difficulty with the first stage, and so fewer moved on to the (more interesting) second stage, with only around 75% of students submitting a report.

Implementation

During the pilot year (2009), the tasks associated with writing the neural network were carefully divided into three, and three associated 90 minute lab sessions were organised, two weeks apart, for the work. The lecturer plus two laboratory demonstrators were available to provide help to the students at these sessions.

For each session, a Microsoft Word template file was provided, and the students copied and pasted relevant program output or small parts of the program (functions) into the appropriate parts of the template. A simple marking scheme was associated with each part, worth 30 marks: typically students could get 0, 1 or 2 for a piece of code plus 0 or 1 for comments; or a student could get 0 or 1 depending on whether the program output was correct. There was also space for comments to be written.

These files were then submitted to the lecturer who circled the relevant mark for each part and added relevant comments. Detailed feedback was thereby generated very easily and very quickly. The aim was to give feedback within a week of the session, allowing the student a further week to make any necessary corrections on one part before starting the next part of the program. In fact, the first week’s work was marked within two days.

Impact

Each year since this scheme has been introduced, around 95% of students have been able to produce a neural network, a significant increase in submissions.

Reflections

The impression obtained in the pilot was that a greater proportion of students had a working neural network compared with previous years, suggesting a great success of this scenario. As such the scenario has been used each year since with some changes to the templates (and to the program to help reduce plagiarism between years).

One problem is that the structure of the program is so tightly defined that there is little scope for variation in code – hence copying is difficult to detect. This is partly addressed by requiring the student to comment their code and to discuss the object-oriented aspects of their program in the final report. In addition students were expected to do experiments in their own time to investigate the effects of changing specific parameters in the program. The instructions for the final report were made clearer to try to ensure this happens.

One disadvantage of the approach is that it discourages independent thought more than is ideal. The much increased submission rate, however, is encouraging.

The important aspects of this scheme are the division of the project into suitable, easily marked sub-tasks, the extra support provided in the development of the program and the inherent feedback between sessions.

Follow up

Following the pilot year, it was realised that some functions were more complex than others, so the marking scheme was changed so that an appropriate number of marks for the code and comments was available for each function.

From 2015, with the move to online submission, students upload their document to Blackboard Learn, where the work is readily marked. Rather than circling the marks, the ‘insert text’ option is used to allow the marks to be entered. Comments on errors or suggestions for improvements are also easily added in an appropriate context.

The second part of the assessment requires the student to apply their neural network to a ‘real world’ problem of their choice, to see if the network can learn that problem. In effect the students are researching whether a neural network is appropriate. Given that, rather than asking the students to write a report on their work, they are now asked to present their research in the form of a four page conference paper. This tests them with a new skill, complementing the report writing skills they use elsewhere. This innovation has also proved successful.

Use of a modified problem-based learning approach in aphasia therapy teaching

By edwardwhite Comment Closed

Dr Arpita Bose, School of Psychology and Clinical Language Sciences
a.bose@reading.ac.uk
Year of activity: 2011/12

Overview

9254A modified problem-based learning approach was developed and implemented in Communication Impairment 3 (PL2CI3/PLMCI3) within the School of Psychology and Clinical Language Sciences.  While the adoption of this approach was unpopular with students on the module, there was a notable improvement in the marks achieved in exams, and this suggests that subtle modification may provide a problem-based learning approach to which students respond well, and that provides for the achievement of improved grades.

Objectives

  • Implement a problem-based learning approach in aphasia therapy teaching.
  • Enable students to apply knowledge obtained through study to be applied to real world clinical cases.
  • Through use of problem-based learning, prepare students for the workplace by allowing them to experience and practise decision making skills and processes.

Context

Speech and Language Therapies programmes taught at the University of Reading aim to prepare evidence-based practitioners, able to apply their knowledge to clinical problems, and make effective decisions in their practice. The problem-based learning approach has been widely adopted within a wide range of academic contexts and professional disciplines, including for Speech and Language Therapy. Under the problem-based learning approach in Speech and Language Therapy, students are encouraged to solve problems that are set in the real world, enabling them to use specific knowledge obtained through self-directed learning with the support of their lecturer to make clinical decisions.

Implementation

The initial task was to create a raft of fictional case studies. The creation of good ‘problems’ is the essence of successful problem-based learning approaches, and so several weeks were spent modifying each case so that students would be able to understand the content area that needed to be taught. Additionally, it was necessary to find appropriate resources that could go with the case studies.

Several resources were generated in order to support the students in solving the case studies, and specific pointers were provided towards the thinking about the case studies (in class), literature (detailed reference list), web-based resources, and resources from the department and library. The module convenor was available for discussion to the assigned group during module-specific office hours.

At the beginning of the problem-based aspect of the module, classes were divided into groups of between five and six students. Each week, the groups chose one of the two cases within the week’s topic, and determined the therapy for a fictional client based on the questions for each case.

Each group was required to give a presentation on their assigned case study, answering questions in five sections. Following this there were two to three minutes available for the audience to critique the answers, and for other possible solutions to be discussed, with students basing their critiques upon their own reading. This allowed various methods to be discussed using different cases. Following this, each group updated their slides and submitted a report, with both of these being uploaded to Blackboard Learn, allowing all students on the module access to the slides and information relating to a particular case, which they could use for their own learning and have available for future clinical practice. In addition, students working on a case study received formative feedback from the seminar leader and their peers.

Having built up their ability to respond to theoretical clinical cases during the teaching of the module, in the module’s examination one of the two questions on the therapy section was modeled on solving a case based on available information, with students being required to attempt one of the two questions.

Impact

In the pilot year, in the examination the problem-based question was attempted by more of both the undergraduate and postgraduate cohorts, and the mean marks were higher for students attempting the problem-based question than those that did not. Additionally, individual students expressed interest in doing aphasia topic for their theses, and module evaluation revealed that students felt better prepared for aphasia therapy in their placements.

Reflections

While the results obtained by students in examinations and engagement suggested value in the implementation of the problem-based learning approach, this was not without its difficulties. The principal difficulty experienced was a poor reaction to the introduction of problem-based learning approach on the part of the students on the module. The introduction of problem-based learning approaches increased the workload upon students, who also had to fit the workload around their placements, and students were unappreciative of the benefits that this increased workload might bring. This may also have resulted from the fact that the undergraduate students on the module were in their third year of study, and so had difficulty adjusting to the expectations of the problem-based approach.

The second issue was that developing a problem-based learning approach necessarily increases the workload of the module convenor. It takes a considerable amount of time to write the cases and generate the resources for the students. As Dr Bose felt that developing teaching in this manner would help students learn the material better, she was willing to put the time and effort in, but this should be a consideration for others looking to adopt a problem-based learning approach.

As a result of these issues, changes were planned to and enacted upon the module in order to get students on board with the problem-based learning approach, and prepare them early on for the demands of the approach, with the workload expectations being somewhat adjusted in order to better respond to the existing workload of students. Additionally, as the delivery of a problem-based learning approach was workload-intensive, arrangements were made to provide co-teaching staff, allowing the workload to be made more manageable.

Follow up

Following the pilot year of using a modified problem-based learning approach in aphasia therapy teaching, problem-based learning has continued to form part of the delivery of this module. Following student feedback in the pilot year, the problem-based elements of the course had been stripped back somewhat in order to respond to this, and student feedback has improved: recent examination results and student feedback, however, suggest that minimal use of a problem-based learning approach is not sufficient if one wishes to see the benefits of such an approach, and that therefore the amount of problem-based learning that is required should now be increased.

Thanks to the effort put in during the first years of running the module using the modified problem-based learning approach, there now exist a number of suitable case studies for use in this approach, and thus the workload is not as intensive as it once was, and only minimal amounts of work are required to ensure that the case studies are current and relevant.

Improving student engagement with assessment and feedback through peer review

By edwardwhite Comment Closed

Professor Helen Parish, School of Humanities
h.l.parish@reading.ac.uk

Year of activity: 2014-15

Overview

9070

The project investigated recent research and practice in peer assessment and feedback in order to implement a peer assessment model for use within History, and develop a framework for the adoption of said model in cognate disciplines where evaluation of substantial text-based assignments is an important part of assessment.

Objectives

  • Present students with well-managed opportunities to engage in feedback and assessment and learn from it.
  • Present staff with access to tried and tested models for implementation that can be used and tailored across disciplines.

Context

The importance of increasing the impact of assessment in feedback and learning is recognised by the University’s teaching and learning enhancement priorities, and is evident in the ‘Engage in Assessment’ and ‘Engage in Feedback’ materials.  The requirement to pursue an agenda for feedback is also highlighted by the expectations of employers that graduates of the University of Reading will be able to assess and evaluate the work of others, by comments on feedback made by University of Reading students in the National Student Survey, and by discussions with potential students on Open Days.

Implementation

There were five stages to the project:

  1. A literature search on the topic and detailed engagement with recent scholarship, undertaken by the Principal Investigator.
  2. A ‘competitor analysis’, undertaken by a research assistant, looking at the extent that peer feedback is present on Humanities curricula at other institutions.
  3. Development of a model for the trial of peer assessment informed by the previous two stages.
  4. Implementation of this model as a ‘pilot project’ in the Department of History.
  5. Obtaining student feedback on the process and reflection by the Principal Investigator.

The feedback gained during the early stages of the project revealed that students were reluctant to allow their work to be reviewed by their peers, even when anonymised.   This necessitated the envisaged model to be altered, whereby the written work being ‘peer reviewed’ was either from previous cohorts within the Department or alternative sources.

Once the pilot project was developed, there were three stages:

  1. Development of an understanding of marking and assessment criteria. Students read the assessment criteria of their module, and were then tasked with rewriting these in their own words.
  2. Applying these criteria to written work. Students then read a sample essay (not taken from the group), and with reference to the marking criteria, were asked to give a mark to the essay, with a summary of reasons they had come to this judgment.  This was followed by a discussion of the written feedback provided.
  3. Focus group and project review.  It was intended that students would meet to talk about the project, and more general issues to do with assessment and feedback, in the presence of an experienced observer external to the department.

Impact

One of the principal benefits of the project was that students became more aware of the marking criteria by which their assignments were assessed, as although they found these clear, few students had actually taken the time to read these before. An additional benefit was that the activity helped develop students’ academic confidence, as they were impelled to adopt a critical attitude to writing within scholarship, and gained experience of promoting their point of view to their peers.

Reflections

Feedback from questionnaires suggested that students enjoyed the project; that they now had a better understanding of assessment and feedback; that the project had been helpful with the preparation of their own written work; and that they were now more confident in the assessment of their own work prior to submission.

The reluctance of students to submit their own work to review by their peers meant that there was a less direct link between the peer feedback provided and the specific assignment for each module.  By using work from previous cohorts or alternative sources, however, it was possible to get students to engage more willingly with the process of peer review.

The main disappointment was that it proved impossible to gather a large enough group of students to participate in the focus group stage of the project.  This may have been due to the proposed scheduling of the focus groups at a time when students had recently participated in a Departmental Periodic Review and submitted their final coursework of the academic year.  Nevertheless, valuable feedback on the pilot was provided through questionnaires and verbal communication.

It was interesting to observe that students held broad spectrum of ideas about what constituted good work, arising from a lack of understanding about the criteria against which work is marked. From this perspective, the project was valuable, as students were familiarised with the marking criteria and how these applied to written pieces Students were able to look ‘behind the scenes’ at the marking process, with student applying the marking criteria as individuals, but then needing to decide as a group upon a final mark for pieces they were reviewing.

Follow up

Following the pilot project, the use of peer review to engage students in assessment and feedback has been used by other members of staff within the Department of History, with similar success. Other than the specific pieces of work and criteria used for peer review purposes, there was nothing within this project that was specific to the Department of History or School of Humanities, and so this activity could easily be adapted for use in other Departments and Schools across the University.

The peer review approach has been successfully applied within the Department of History to student presentations in seminars. As student presentations are more ‘in the moment’ and designed with a peer audience in mind, students have not expressed the same reticence to have their peers review their work, and those presenting have appreciated receiving immediate feedback.

Incorporating research seminar series into teaching and learning

By edwardwhite Comment Closed

Dr Louise Johnson, School of Biological Sciences
l.j.johnson@reading.ac.uk

Overview

4258In the School of Biological Studies a module, Seminars in Biology (BI3S78) was created, utilising the existing research seminar series within the School to structure assessment. The module is easily adaptable for other subject areas, and has proved very popular with students, and has aided in their development of academic and personal skills.

Objectives

  • Better utilise the School’s existing research seminar series.
  • Develop a number of students’ skills, including report writing, writing for different audiences, experimental design, literature searching, and referencing.

Context

The School of Biological Sciences had long held research seminars, with speakers conducting innovative research in the biological sciences being invited to deliver a seminar on their research. Despite these research seminars being an available resource for students to engage with new research in their field of study, and advertisement of the seminars within the school, attendance was disappointing, with many students failing to utilise such a valuable resource. Within the School of Chemistry, Pharmacy, and Food a Part Four module, Current Topics in Chemical Research, exists, having students attend research seminars and creating assessed reports upon these. This provided the inspiration for the creation of a similar module in the School of Biological Sciences.

Implementation

The School holds around 20 research seminars a year on current research in life sciences. On each seminar attended, students complete a Seminar Report Form with a summary of the seminar topic, three things they learned from the seminar, a comment on the quantitative methods used, references (one scientific review article and two primary research papers) for someone who wished to research the topic, and their justification for these sources. This ensures not only that students attend the research seminars, but also actively engage in note-taking during the seminar. Commenting on the quantitative methods used by the seminar speaker encourages students to reflect upon their own use of quantitative methods. By providing references for someone else to research the topic, students develop their skills in referencing and literature searching. The reports are marked on a pass/fail basis, and students must have achieved passes on at least 10 reports. This forms 20% of the module grade.

At the beginning of the module, students take an online test on statistics and experimental design, in order to develop their understanding of these subjects, as well as to highlight common mistakes. The test constitutes 10% of the module grade.

The first written assignment is to write a summary paragraph of between 200 and 300 words on a seminar, suitable for publication in the scientific journal Nature. The marking criteria for this are based upon the publication guidelines of Nature, as well as the usual University marking criteria. This develops the ability to convey the importance and context of research findings. This assignment constitutes 15% of the module grade.

The second written assignment is a 1500 word critical analysis of research presented in a seminar. This assignment encourages students to engage with the ideas, evidence and techniques presented in a chosen seminar. Students consider the strengths and weaknesses of the research, and suggest ways in which the research might be improved. This assignment constitutes 40% of the module grade. Students are not permitted to write their critical analysis on the research seminar on which they wrote their summary paragraph.

The final assignment is to write a 500 word news article suitable for inclusion in a popular science publication, for an audience of non-specialists, on one of the seminars. As with the critical analysis, students are not permitted to write their article on either research seminar on which they completed their summary paragraph or critical analysis. This assignment provides students with experience of writing for a non-specialist audience, promoting clarity of expression and an ability to select what areas of research will be of appeal to a broad audience.

Impact

Student feedback, both formal and informal, has demonstrated that the module is very popular. Students have reported that the assignments helped them develop skills that they were able to use in other areas of their study, and students have referenced the seminars in their examinations, suggesting that they engage well with the module and the seminar topics. Students make special reference to how attending the seminar series have helped them refine their own interests within life sciences, and some students have actually found PhD places as a result of their attendance of research seminars.

Reflections

Students find the module challenging, because it draws upon different skills to other modules within the School. Having students operate outside their academic comfort zone, however, is a valuable learning exercise, and there has been no desire to alter the course to any great extent, as student feedback does not justify this.

The greatest challenge involved in running this module is the varied topics upon which seminars may be held, as this requires markers of the reports to be specialists within the area. With the popularity of the module resulting in large student numbers, this can create a large workload for markers.

There is great value in running this module, as it develops a number of skills in students, including report writing, statistics, referencing, literature searching, writing for different audiences, and experimental design.

Integrating Facebook into team-based learning

By edwardwhite Comment Closed

Dr Christopher Voisey, Henley Business School
c.j.voisey@reading.ac.uk
Year of activity: 2014/15

Overview

11331The core Part Two undergraduate module, International Business Management and Strategy (MM272), was redesigned on two pillars: teamwork, and social media (Facebook). Formal student evaluations were high and feedback from focus groups with students was very positive – students found the use of Facebook effective and enjoyable, and students felt fully engaged.

Objectives

  • Increase student engagement with team-based learning.
  • Integrate Facebook into module delivery.

Context

Team-based learning forms a core part of the Part Two module MM272; approximately 150 students are enrolled on this module. Yet inter-team discussions within the team-based learning context were limited to and by the classroom. Team-based learning for modules with larger student cohorts is an especially promising context in which Facebook may enhance learning outcomes. A recent project at the University of Reading had evaluated Blackboard Learn and email as being confusing and dated to students for the purposes of sharing, and that Facebook provided a more flexible and familiar platform. As a two-sided network in which posters and readers provide each other with network benefits through interactions, Facebook complements team-based learning by allowing for posting of key team arguments online, and for multiple rounds of comments and responses – dynamic interactions that strengthen learning.

Implementation

First, a closed course Facebook group was created, students were divided into teams, and six Facebook-enabled tasks were designed. These tasks were to vote for module coursework mark allocation across assessment areas, to submit case-based assignments by in-class posting onto dedicated Facebook events (with tagging of other teams for comment), to appeal multiple choice questions, to post analyses of current business topics, for the module convenor to provide assignment-feedback (but not marks), and to post a Q&A and documents for download.

Impact

Facebook enabled development of norms of ‘professional informality’; barriers were lowered and there was greater tacit understanding of the subject, with higher learning outcomes as evidenced from exams. Facebook was inclusive, and gave a ‘voice’ to students who might be more reticent in class discussion. Class time was not monopolised by one speaker, but all voices were provided an audience through postings and comments online.

Reflections

Facebook has afforded advantages understood from a social constructivist perspective on learning – learning emerges from social activity. Students observed each other, their postings, and this shaped their behaviours, leading to the development of norms in interacting, and increasing the level of scholarship in assignments, consistent with social learning theory. Socially, the boundary between personal and professional becomes blurred.

LS1TAL Techniques and Skills for Applied Linguistics: Improving the student experience

By edwardwhite Comment Closed

Professor Jane Setter, School of Literature and Languages
j.e.setter@reading.ac.uk
Year of activity: 2014-15

Overview

12759 (1)Students and staff worked together during 2014-15 to develop the Department of English Language and Applied Linguistics’ new compulsory Part One module Techniques and Skills for Applied Linguistics (LS1TAL), run for the first time in 2014-15, to make it more interesting, useful and relevant to 21st Century undergraduate students. The project sought, among other things, to address the University Board for Teaching and Learning Enhancement Priority to evolve our approaches to teaching and learning, with a specific focus on using Technology Enhanced Learning in class and in assessment.

Objectives

  • An improved module following input from students which addresses their needs more closely in the development of the study of our discipline and transition to Higher Education.
  • Better use of Technology Enhanced Learning elements in content, delivery and assessment.
  • Enhanced use of the formative assignment English Language and Applied Linguistics students are asked to prepare, and of the feedback given.
  • The development of good practice which can be shared across our modules and with others in the University and across the sector.

Context

The module convenor had produced a new compulsory module for 2014-15 which addressed many of the issues arising for students in the first year of study, such as general transition to Higher Education; how to do tertiary-level academic writing; how to use and get the best out of tools such as Turnitin; how to present yourself effectively online with a jobs-market orientation; how to do assessments using Technology Enhanced Learning approaches, such as blogging and short videos. Student involvement in the development of the module was desired to make it particularly relevant and useful to them.

Implementation

Students taking the module were recruited during the Autumn term 2014-15 to be leaders and participants in the development of the module; four came forward. PLanT funding was applied for and received. Subsequently, focus groups were held at four points during the year between students and staff to review module content and suggest ways to develop it. Finally, with the agreement of the students, a new weekly schedule for the module (which runs in the Autumn and Spring terms) was drawn up.

Specifically, students asked for more concise and focussed input on transitions to Higher Education and online presence, including a specific session on building a LinkedIn profile, a more hands-on approach to library skills training, input from students at Parts Two and Three, and sessions from Student Counselling and Wellbeing.

Impact

Students and staff involved felt that the outcomes had been very positive. Students valued having the chance to develop a new module, and staff enjoyed working with students, finding out what they already knew and what needed further development, and understanding their viewpoint on their needs as new entrants to Higher Education in the UK. The real test for the module, however, will be the revised module running in the 2015-16 academic year, during which data will be collected from the new cohort using module evaluation forms to compare with last year’s evaluations and see whether student satisfaction has improved.

We were delighted that students had no issues with the Technology Enhanced Learning elements in the module content or assessment. In fact, they thought they were very useful and well-integrated.

An unexpected outcome has been the Director of English Language and Applied Linguistics’ Postgraduate taught programmes’ interest in the module to see whether it could be adapted for MA students.

Reflections

The activity was made successful by the involvement of students in developing this module. Teaching and Learning Dean Dr David Carter had commented that it seemed like a very well-designed module; the decision about how to develop the module could have been made by an individual based solely on student feedback questionnaires, but it seemed much better to have the hands-on involvement of students, as the module is aimed at supporting students through Part One. Their involvement enabled students to have input into their degree which resulted in real change, to see first-hand the issues involved in module development, and also to appreciate the Department’s attention to teaching, learning, the development of transferable skills, and supporting students to get the best from their university study and beyond.

Teaching in a divided classroom: the impact of internationalisation and marketisation on business education

By edwardwhite Comment Closed

Dr John Latsis, Henley Business School
j.s.latsis@henley.ac.uk
Year(s) of activity: 2013-14

Overview

9337In the postgraduate module Managing People and Organisations (MMM048), provided by the School of Leadership, Organisations and Behaviour, assessment methods were altered in a manner that was mindful of the increased internationalisation and marketisation of UK Higher Education, in order to assist the transition of international students.

Objectives

  • To provide a method of assessment that fit the needs of international students less acculturated to UK Higher Education
  • To design this method of assessment so that it does not disadvantage more culturally expert students.
  • To have the first assessment of the module prepare students for essay-writing for further assessments.

Context

Results from previous years of MMM048 revealed that students on the module struggled with the first assessed essay, but showed significant improvement for their second assessed essay. In particular, the results suggested a difficulty for international students, who constituted over 80% of module students, to acculturate themselves to the expectations of UK Higher Education. Specifically, linguistic competence in written coursework, understanding of the requirements of critical engagement and argumentation in essay writing, and the needs of some students for individual follow-up meetings to discuss module content, were issues that needed to be addressed.

Implementation

Potential solutions, such as engaging in targeted small group tutoring, or simplifying the content of the module, were unacceptable. Engaging in targeted small group tutoring would negatively impact the workload of teaching staff, to the detriment of other duties, and would result in those groups that received said teaching having an unfair advantage. Simplifying the intellectual content of the module was undesirable, as the content of the module consistently received good feedback, and to do so would give a false impression of what was expected of students in their postgraduate study. Additionally, as students in the bottom quartile of the mark distribution generally showed evidence of improvement over the course of the module, this suggested that the content was not itself too difficult.

What was developed was an extended essay plan as a form of assessment, a hybrid solution that maintained the essay-writing element of the first part of the module, but allowed students to gain a hands-on insight into the expectations of UK postgraduate Higher Education. Students were provided with an essay plan handbook, explaining the expectations of how an essay would be written, providing a ready-made generic structure, with subheadings, approximate word counts for each section, and the usual guidelines with which students are provided. The essay plan is shorter than the full essay which previously formed the first assessment of the module (1000 words rather than 2500 words), and is worth less (15% rather than 30%). Additionally, the requirement for students to write in continuous English prose, which students might initially find difficult, is softened, as students are allowed to develop their ideas in bullet points in order to save space. The development of the extended essay plan format was carried out in consultation with the In-Sessional English Support team, in order to assure that the template was worded as clearly as possible.

Impact

The net effect of the change was significant. The failure rate for the first assessment dropped to 0%, and there was a reduced failure rate in the module as a whole. Student satisfaction surveys for the module achieved higher scores than previously, and students reported in casual conversation that they would continue to use the template and accompanying handbook to help them write their essays for other modules, as they found it a very useful tool to organise their thoughts and keep their arguments on track.

Reflections

The essay plan format is beneficial for the following reasons:

  1. It replaced the need for coaching to be provided in the context of a large class size with a variety of individual needs.
  2. It makes explicit the cultural clues that students with experience of the UK Higher Education system understand through verbal communication, but that have proved difficult to communicate verbally to students without this experience.
  3. It provides an explicit performance standard with instructions and mark-breakdowns that makes assessment clear and maintains standards of fairness across all levels of ability.
  4. As a result of the format including multiples questions, one of which is more difficult than the others, there is still the possibility for the most able students to demonstrate their ability by effectively addressing a difficult topic.
  5. It draws the markers’ attention away from linguistic ability and puts the emphasis on clarity of argument and quality of ideas, re-uniting a divided classroom.

While the new approach does reduce the flexibility that students have to express themselves within the constraints of the template, and benefits non-native English users more than it does native English users, the format allows students to be assessed on their understanding of the module content, and their ability to reflect critically upon it and construct a coherent argument.

Follow up

The essay plan assessment format has continued to be utilised within MMM048. There have been some minor changes to the wording of the essay plan and associated guidance as a result of input from the University Study Advice team. While comparisons between cohorts are difficult to perform, it is encouraging that marks on MMM048 improved last academic year.

Working in partnership… by Dawn Willoughby

By edwardwhite Comment Closed

With ever-increasing speed it seems we have reached that time of year again when teaching is finished, exam results are calculated and our thoughts start to turn from the current academic session to planning for the next one. This offers some time to reflect on successes and think about the areas of teaching where we would like to make changes. The “stand-out” feature of my module portfolio in 2014-15 has been the increased level of working in partnership with colleagues to teach and support our undergraduate students. Six of the seven modules I delivered this year were co-taught, an approach which has brought some challenges and plenty of rewards.

 

… with PhD students

In the Henley Business School, I am fortunate to work each year with a small team of PhD students who are responsible for supporting the delivery of lecture material in Statistics through weekly tutorials for undergraduates. This partnership provides an important opportunity for PhD students to strengthen their transferable skill set and become more effective facilitators; I have seen them gain a clearer understanding of pedagogy and an increased level of confidence. Similarly, there are benefits here for our undergraduates: they receive more individualised support for their taught course and they can also gain an appreciation of how their learning relates to the research undertaken in the School. And for myself, my motivation is improved by the enthusiasm of the PhD students and their “get-involved” attitude towards the programme delivery.

 

… with industry-based professionals

For my module in the School of Systems Engineering, there can be no doubt that engaging with IT professionals enhances the employability skills of our undergraduate students. In this case, the programme involves a group-based web development project for which employees of a local web design company act as the client. Developing this working relationship over the past few years has provided an extra dimension to my teaching. In recent discussions it has become clear that the company also values their involvement: “It offers an opportunity to gain skills and experience that simply would not be available to most of our staff members in their usual roles within the business. It has also afforded us an opportunity to give something back to the community by sharing our expertise – something we feel strongly about.”

 

Engaging in team-based teaching presents challenges especially in maintaining seamless delivery and providing consistent information to students. Sometimes there can also be logistical and communication problems associated with bringing together a diverse set of people with different working practices and other research-based or commercial objectives. However, all of these difficulties can be overcome when an inclusive environment is created in which open discussion is encouraged. From my experience, I have also learnt the importance of devising and sharing formal documentation to ensure consistency in assessment and provision of feedback to students.

 

When I first started working at the University, I expected teaching in higher education would be a rather solitary experience providing very little opportunity for interaction with colleagues. Ten years later, I can testify that this is certainly not the case as I work on a variety collaborative teaching and learning projects with colleagues from several Schools and Departments. It may be true that team-based teaching requires some additional commitment and effort but in my opinion this approach can bring benefits to University staff and students, and to external partners. And so, as planning gets underway for the next cohort of students, I would encourage consideration of team-based teaching as an option for module delivery.

Tailored formula sheets – the ‘cheat sheet’ idea by Dr Karen Ayres

By edwardwhite Comment Closed

About 10 years ago I was intrigued when a colleague described taking exams at an Australian university, where he was given a blank sheet of paper on which he could write anything he wanted to take into the exam. He referred to this as a ‘cheat sheet’. I was familiar with both open and closed book exams, and their pitfalls, and also with providing a formula sheet to students in an exam. But this idea was something completely different, being an individualised exam accompaniment. I was even more intrigued to hear him say that the benefit of this sheet was that he had been forced to properly revise the material in the module so that he could work out what to include – there was no point wasting precious space writing down things he would be able to remember, but it was very important to write down things which he was afraid he would forget.

 

This memory stayed with me, and I always wanted to try out the idea myself, but couldn’t see how it could be implemented that easily in a centrally-administered exam. But then this year, as a way of creating a faux semester system for our finalists, we replaced the summer term exam with a department-administered class test in January for two modules. One of these was my final year statistics module on Multivariate Data Analysis, and so I seized my chance!

 

The idea of being able to take into the test an A4 sheet on which they could write anything they wanted to was well received by the students when I first announced it – it automatically relieved some of the pressure they felt about having to memorise formulae, or all steps in a proof. In terms of the effect on how I wrote the exam, this was no different really than writing an open book exam – there needed to be more emphasis on questions which applied the methods, or were open ended in what they were asking for, rather than requiring the statement of a formula or reproduction of a basic proof. However, this didn’t actually require much adjustment to my style of writing questions. Implementing the idea was also fairly straightforward. The module finished in the final week of the Autumn term, and the students handed in their ‘cheat sheet’ on the first day of the Spring term for me to photocopy onto coloured paper (to prevent any additional sheets being smuggled in!), and distribute at the start of the test. The (named) sheets were thrown away at the end of the test, to ensure they were not attached in any way to the anonymised answer booklets.

 

I’m pleased to say that the ‘cheat sheet’ idea was an unmitigated success. Although it is impossible to quantify its effect because of cohort effects, and the fact that the test was now in January rather than in the summer, feedback from the class about the idea has been positive and the marks were a little higher than in the past. The most gratifying thing for me though is hearing many students say that they didn’t actually need to look at their sheet in the exam, because they had spent so much time writing and rewriting the sheet to make sure it included everything they wanted it to, that they ended up learning and remembering all of the material anyway! So I seem to have educated my students by stealth!  But all joking aside, I think that the ‘cheat sheet’ idea has benefits across the board and particularly when viewed from a diversity perspective. Different students will struggle with different things for different reasons. Letting them help themselves by constructing their own formula/information sheet which is tailored to their strengths and weaknesses acknowledges that diversity, and is one way to put students even more in control of their learning and also their attainment. I definitely encourage others to try this idea!