Redesigning postgraduate curricula on commercial law through student engaging, research-informed and multidisciplinary pathway programmes

By edwardwhite Comment Closed

Professor Stavroula Karapapa, School of Law                          s.karapapa@reading.ac.uk

Overview

In 2015/2016, we substantially redeveloped our postgraduate provision in commercial law through the introduction of a pioneering, student-engaging, research-informed and multidisciplinary set of postgraduate pathway programmes. Contrary to the programmes previously in place, the new curriculum is unique in its pathway design allowing students to develop a breadth of commercial law expertise whilst also specialising in their area of interest (for a full list of programmes see here). The project on which this entry reflects has resulted in an innovative curriculum that shaped the identity of Centre for Commercial Law and Financial Regulation (CCLFR) as a centre of excellence on cutting-edge themes of commercial law.

Objectives

  • To redevelop our postgraduate curriculum in commercial law through the introduction of cutting-edge themes of study based on the principles of research-informed and multidisciplinary teaching.
  • To empower student learning, improve student experience, and foster the development of a learning community.
  • To hear the student voice towards the design of the curriculum and to proactively and directly engage ‘students as partners’ in the development and evaluation of the core module for the new programmes.

Context

As often happens in Higher Education, the postgraduate programmes in Commercial Law previously in place were the result of the work of independent colleagues at various points in time, starting in 2011. Modular options reflected this dynamic, and they were also impacted by continuous staffing changes over the years. The pathway programmes are the result of collective effort within the School of Law, effective consultation with students and evaluation of their feedback, and constructive collaboration with colleagues from various Schools and services across the University (including marketing, careers, conversions etc.).

Implementation

Following a review of our PGT provision, we redeveloped our commercial law curriculum on the basis of three pillars:

  1. student feedback (module evaluation forms and ‘graduation’ forms collected since 2011) concerning suggestions for improvement, informal comments from students enrolled in 2015/2016 on ideas for new modules/programmes and engagement of ‘students as partners’ in the development of the core module for the new pathway programmes;
  2. extensive market study carried out by marketing and the (then) PGT Director regarding areas worth expanding on;
  3. expansion of our module offerings through the valuable contribution of numerous colleagues in the School of Law and consultation with various Schools across the University that agreed to open up relevant modules, effectively enhancing multidisciplinarity in our programmes.

Instead of offering numerous programmes with no clear link to each other, we introduced a set of pathway programmes (including 5 new PGT programmes and a redesign of the existing ones) whereby all programmes are centred around one core legal field, International Commercial Law, and students have the option to follow a pathway on a specialist area designed around our research strengths as a School and as a University, essentially building on research-informed teaching. Part of this redesigning process was the revision of the compulsory module for all pathways, LWMTAI-Advanced Issues in Commercial Law, which was based on the engagement of students as partners, drawing on a UoR small-scale research project that was initiated in June 2016, an entry of which is available here and here.

Impact

The collective effort of numerous colleagues in the School of Law and the support from various Schools and services across the University resulted in the development of a pioneering set of pathway programmes, centred around the values of research-informed teaching and multidisciplinarity and developed on the basis of student feedback. The project enhanced student engagement, taking on board student views on the learning design. The redrafting of a core module (LWMTAI) had direct impact on student learning, enabling students to proactively review their own learning process and to develop an increased sense of leadership and motivation. There was also positive correlation between the introduction of new pathways (especially Information Technology and Commerce; Energy Law and Natural Resources) and PGT recruitment. Anecdotal evidence indicates that the conversion rate of existing PGT students to our PGR programme has also increased. Importantly, the redevelopment of our programmes created a distinctive identity for CCLFR as a centre of excellence on cutting-edge themes of commercial law.

Reflections

The success of the redevelopment of our PGT curriculum was based on three pillars:

  1. Collective effort: The redevelopment of the programmes required the engagement of various colleagues from the School of Law who met on numerous occasions to reflect on the programmes and introduced new modules on cutting-edge themes to meet the needs of the new pathway design. This effort exceeded business as usual. An example of such collective effort is the redesign of the core module of the pathway programmes which followed the ‘student as partners’ approach and was implemented with the collaboration of various members of staff from the School of Law.
  2. Student engagement: Unlike what usually happens in higher education with ex post student feedback, the pathway design used that feedback constructively towards designing new programmes, taking into consideration student comments in evaluation forms and also engaging students in the programme design process. Importantly, it was students themselves that proactively informed the curriculum of the core module for all pathway programmes, with their voice having being heard even before the completion of the taught component.
  3. Cutting-edge themes and research-informed teaching: At the heart of student feedback was the desire to increase the number of modular offerings from other Schools and Departments, effectively to enhance the multidisciplinary approach that was already in place. Introducing more modules from other Schools to our curriculum on the basis of their relevance and appropriateness to our pathways has become a learning process to us as educators in that it has resulted in dynamic synergies and an innovative curriculum as end-result of the exercise.

Links

Details on our new pathway programmes are available here: http://www.reading.ac.uk/law/pg-taught/law-pgt-courses.aspx

How pre-sessional English has develop the use of Turnitin, submission, marking and feedback to support students’ essay and exam writing.

By edwardwhite Comment Closed

Jonathan Smith is the School Director for Technology Enhanced Learning in ISLI (International Study and Language Institute). He is also a PSE (Pre-sessional English) Course Director and teacher of English.

The Pre-sessional English programme accepts around 600 to 800 students each year. Their students develop English skills in academic writing, reading, speaking and listening.

In the area of academic writing Jonathan Smith and his team have been exploring the use of Turnitin (Tii) GradeMark to facilitate electronic marking and feedback via:

. E-submission of written essays.

. E-marking and e-feedback via GradeMark using QuickMarks and text comments.

. Student engagement with feedback in subsequent production of written work.

About five or six years ago, before the use of GradeMark was adopted in the university, a group of pre-sessional staff attended a conference in Southampton in which colleagues of other universities presented how they were using GradeMark. It seemed a tool that could not only save time producing feedback but produce feedback of a more consistent quality. A couple of years later PSE started exploring its use with our cohorts of English academic writing students.

Listen to Jonathan’s experience on how he got involved with electronic submission, marking and feedback via Tii in this podcast.

Jonathan Smith, provides all PSE teachers with a one-hour workshop on how to use Turnitin and Grademark. Part of the training involves the use of the PSE ‘QuickMarks’ for e-feedback. These QuickMarks focus on common student errors with explanations and links to relevant sources – and can be used to provide in-text feedback. ‘QuickMarks’ are based not only on common grammar and lexical errors but also on the complexity of the language structures used and coherence and cohesion in the texts. Students are also assessed on content, use of references and other areas of relevance to academic essay writing.

After the training session, tutors set up submission points for formative work, in this manner students grow accustomed to submit work, access feedback, see and compare their own progress.

Students receive feedback almost immediately and they can work on the feedback either to bring it to the next class or towards their next assignments.

 

 

 

 

 

 

 

From the teachers’ perspective it was noticed that it was quicker to note common student errors in-text using QuickMarks. It was possible to see colleagues’ feedback comments which facilitated new tutors becoming familiar with marking and feedback across the cohorts.


 

 

 

 

 

 

One of the big advantages is that Turnitin is a one stop shop for both checking similarity and producing and receiving feedback. Students upload their essays, they can see their similarity reports and have the opportunity to take action and re-submit. There are a few technical issues around doing that, but the pre-sessional programme is committed to students seeing their similarity reports and using them to get a better idea of the quality and acceptability of their work.

Visit the EMA programme site to find out more case studies and updates http://www.reading.ac.uk/internal/ema/ema-news.aspx

THE BENEFITS OF NEW MARKS AVAILABILITY ON RISIS FOR PERSONAL TUTORING: By Dr Madeline Davies, EMA Academic, and Kat Lee (External)

By edwardwhite Comment Closed

The EMA Programme has delivered a new function within RISIS that allows colleagues to see their students’ sub modular marks on the Tutor Card. We have all had access to the Tutor Card for some time and it has provided an invaluable snapshot of a student’s degree history, particularly useful for writing references and for monitoring attendance. However, in terms of sub modular marks, it has always functioned retrospectively: prior to the start of the new academic year, our students’ updated assessment records from the previous session are available on the Card but they have never been available during the academic session.

The sub modular mark screens accessible via the Tutor Card mean that we will no longer have to wait until the end of the academic year to have access to our students’ assessment information and this creates a range of benefits for personal tutors in particular. Easy access to the sub modular marks will provide an early indication of any problems that our students may be having and this will allow us to address these issues in a timely manner.

The information becoming available is significantly more extensive than a list of marks alone: a series of codes is used to flag up, for example, a result involving academic misconduct or extenuating circumstances requests (scroll down the page to translate the codes via the key), and a hover function under ‘Notes’ provides submission details so that personal tutors can tell when a ‘late’ penalty has been applied or when there has been another change to a mark (see image). Any one of these situations would require personal tutor intervention but, until now, this information has not been available to us unless our tutees have chosen to disclose it in personal tutor meetings.

The new screens are, then, particularly significant for our work as personal tutors: the wealth of information made available gives tutors the means to identify and support students who are struggling before they find themselves in crisis. Proactive and early intervention is always more effective than reactive response, and the additional access to information during the year that has been made available by EMA allows us to ensure that no student falls behind without us realising it.

The new screens also connect with the University’s inclusivity agenda in that students coming to us from non-traditional educational backgrounds can need extra support in their first months with us. The screens will alert us to situations where Study Advice, or Counselling and Wellbeing, need to be consulted.

In addition, students who may be of concern in academic engagement and/or Fitness to Study processes, can be checked at every assessment point, and this will allow Senior Tutors and SDTLs the opportunity to assess a student’s ability to cope with the pressure of assessment deadlines. This in turn facilitates early intervention in problematic cases and provides an easily available record of performance in cases requiring escalation.

The role of the personal tutor primarily involves offering tutees academic advice in response to their marks, feedback and more general concerns. The addition to the Tutor Card of sub modular marks and notes during the course of the year underpins this work and creates the opportunity for meaningful discussions with our tutees. New access to this information allows us to respond to student issues ‘in real time’, thus allowing personal tutors to act as effective academic advisors, and to engage in crucial developmental dialogue with the students in our care.

To view a screencast that shows you how to navigate the sub modular mark screens on the tutor card, click https://www.screencast.com/t/sKCH4czjJ

To view a screencast that shows you how to navigate the Module Convenor Screens that are now also live, click http://www.screencast.com/t/MjCxE6UxfM

For further information on the EMA Programme, please click http://www.reading.ac.uk/ema/

Exploring different types of video cameras for use in practical classes and outreach By Dr Philippa Cranwell, Mrs Susan Mayes and Dr Jenny Eyley

By edwardwhite Comment Closed

A successful TLDF application in April provided us with funds to explore the use of different lapel-mounted cameras to look into student-student and student-staff interactions within a practical laboratory environment. This work is still ongoing, but we have learnt some interesting lessons about buying lapel-mounted cameras along the way, and have also used them successfully in outreach initiatives.

Cameras trialled

In total, four types of camera were trialled that cost between £49.95 and £120 (RRP; correct as of August 2017). With all the cameras we purchased additional memory cards, although some were supplied with small memory cards.

The first three were of a similar design; a camera, shaped like a USB stick, with a clip on the back to allow it to be mounted on a pocket. The cameras trialled were: the Veho VCC-003-MUVI-BLK MUVI Micro Digital Camcorder (RRP £39.95); the Conbrov® Spy Cameras DV12 720P (RRP £59.99); and the Conbrov® WF92 1080P (RRP £69.99). All arrived quickly and were very easy to set-up, although none had a screen so it was not possible to see the recording without putting the images onto a computer. We quickly realised that mounting these cameras on a lab-coat pocket was not satisfactory because they were quite weighty and fell forwards, resulting in a great deal of footage of the floor. A body harness was available for the Veho camera (RRP £39.95), which would have addressed this problem, but it was decided not to continue with this style of camera due to the lack of screen resulting in no real-time feedback of recording quality.

L to R: Veho VCC-003-MUVI-BLK MUVI Micro Digital Camcorder; Conbrov® Spy Cameras DV12 720P; Conbrov® WF92 1080P

The camera that was most suitable for our needs was the Apeman Underwater Action Camera Wi-Fi 1080P 14MP Full HD Action Cam Sports Camera 2.0 (RRP £119.90). This camera came with 2 batteries, each recording up to 90 minutes of footage. We purchased micro SD cards separately; cards over 32MB are not supported by this camera. In addition to the camera we purchased a Togetherone Essential Accessories Bundle Kit (RRP £59.99) that provided a large number of additional items to mount the camera as required. Some of the most useful items in the pack included a “selfie-stick” that was used by school children on an outreach visit, a body harness and a head-mounted harness. The camera itself arrived in a plastic container, which is waterproof and protects the camera, but when recording dialogue it is less useful as the sound is muffled. However, there are alternative holders so the camera can be mounted on the body or head in an open case allowing clear dialogue to be captured.

The Apeman Underwater Action Camera Wi-Fi 1080P 14MP Full HD Action Cam Sports Camera 2.0 and the Togetherone Essential Accessories Bundle Kit

Use in outreach

The cameras were successfully used by secondary school students who took part in a trip to Thames Water sewage treatment works. This trip was organised by the chemistry outreach team as part of the Chemistry for All project, in order to show students how chemistry is used in all parts of their daily life. The number of students able to have this experience was limited by the space on the observation platforms, therefore the students used the cameras to film their experience and produce a video diary of the day. The videos will be edited and shared with other students on return to school, widening the reach of the activity beyond the students who attended. The teacher who was in attendance with the students commented that “having the Apeman cameras during the tour meant they were more excited and enjoyed it more”

 

        

Photographs taken by the students at the Thames Water sewage treatment works

Outlook

The Apeman cameras have been a useful addition to the Department, particularly for outreach purposes. We will continue to use the cameras for outreach, and also to undertake some observations of students undertaking practical work for the TLDF-funded project and another internationalisation project in conjunction with ISLI.

 

 

Launching the FLAIR CPD scheme at the University of Reading Malaysia – By Dr Eileen Hyder

By edwardwhite Comment Closed

One of the highlights of 2017 for me was launching the FLAIR CPD scheme at the University of Reading Malaysia. A substantial part of my role involves talking to colleagues about their work to help them to develop ideas for their FLAIR CPD application. These conversations give me wonderful snapshots into the fantastic work happening across our institution. This is such a privilege and is probably what I love most about my work. I knew I would find it fascinating to talk to colleagues at UoRM and to learn more about the work they are doing in such a different context. However, the conversations I had there were not just fascinating but a real eye-opener for me.

One aspect of an application for Associate Fellowship or Fellowship is to write 600 words on designing and planning learning. Because the sessions/modules delivered in Malaysia have often been designed at Reading, this raised questions about whether colleagues at UoRM would be able to demonstrate this type of activity. However, the discussions that took place in the workshops threw out many examples that quickly showed us that any concerns we had were misplaced.

One example that sticks in mind came from a colleague in Psychology. He explained to us that some Psychology students at Reading will have studied the subject at school and he added that, even those who haven’t, will more than likely be aware of some key figures and concepts included in the university curriculum. However, because Psychology does not feature on the school curriculum in Malaysia and because awareness of figures like Freud or concepts like psychoanalysis cannot be taken for granted, he needs to reflect carefully on what has been designed at Reading UK to ensure it can be delivered effectively at UoRM.

Another colleague explained to us that modules at UoR UK are sometimes designed around the research interests of staff. In a case like this, the module might be taught by a team of as many as eight colleagues, with each person delivering a session built around their area of expertise. However, the same module will be delivered by only one tutor at UoRM. While I have had experience of delivering sessions designed by someone else, I have never been in a position like this. I knew I would be conscious of the limits of my expertise compared to the experts at Reading UK and be anxious about whether I would be able to provide an equally high quality learning experience for my students. I felt huge respect for the way colleagues at UoRM take responsibility for designing sessions that do this.

Through these conversations and others we quickly came to realise that we had been naive in thinking it might be difficult for colleagues at UoRM to write about designing/planning learning. We realised that far from being passive deliverers of material designed at Reading UK, they work very hard to translate and customise learning for the UoRM context. This means exercising professional judgement and skills to make learning relevant and accessible to their students.

One of the things I love about my role is how it enriches my own understanding of teaching and learning. Working with colleagues at UoRM certainly broadened my understanding of what counts as designing/planning learning. The Curriculum Framework is leading to exciting discussions about how our curricula are designed. My experiences at UoRM have led me to think that we should involve as wide a range of colleagues as possible in these discussions. Just because someone might not have had autonomy in the original design of a module does not mean that they have no agency. The Curriculum Framework is an important catalyst for discussions around curriculum design and around the global relevance of our programmes/modules. Involving colleagues who take something designed in one context and deliver it in another could add richness and value to these discussions.

Forecasting, Feedback and Self-reflection by Dr Peter Inness

By edwardwhite Comment Closed

Overview:

Each year a group of part 2 students from Meteorology make their way across campus to the Minghella Building to film weather forecasts in the professional “green screen” studio. As well as improving their forecasting ability this module also helps students to improve their presentation skills – a key employability attribute in many careers.

Objectives:

During the module students will;

  • make short video weather forecasts in a professional studio
  • receive feedback on performance in order to improve on the quality of the work
  • give peer feedback to fellow students in order to develop this useful life skill
  • reflect on their performance and consider how they can use the feedback to improve future performances.

Context:

Presentation skills are a crucial aspect of many jobs, whether it be in front of a camera or face to face with an audience. Lecturers in Meteorology may not always be the best people to coach these skills so we draw on experience in a School where performance and presentation is at the heart of everything they do.
Students spend 4 sessions in the TV studio, working up to the filming of a “live” TV weather forecast. After each rehearsal, students receive detailed feedback on their performance from staff and also from their fellow students. Crucially they are also asked to reflect on their own performance and how they might improve it. This self- reflection aspect is something we would like to encourage across the Meteorology department as it is a skill which perhaps doesn’t come naturally to a scientific discipline in the same way as it does in a performance related discipline such as film and theatre.

Impact:

Students are very appreciative of the high level of feedback on performance in this module, as evidenced in module evaluation questionnaires. The feedback also has a massive impact on improving the students’ performances across the module, resulting in some near professional standard performances by the end.

It is obvious that the encouragement to reflect and take on board feedback is a major driver of improved student performance in this module.

Reflections:

Working in an environment in which feedback and self-reflection are built into the activities has made me as a module convenor in a science department realise that this is something we can use more effectively across many of our other modules, not just those which involve presentation.

Self-reflection and peer feedback have a clear impact on performance in this module and we need to find ways to incorporate more of these activities into the rest of our taught modules.

I am now actively looking at ways that we can make reflection an integral part of how our students approach their learning.

 

Supporting Diversity through Targeted Skills Development: Helping Students to Speak a New Language by Alison Fenner SFHEA (Institution Wide Language Programme, ISLI)

By edwardwhite Comment Closed

Context

As the student population becomes increasingly international, the IWLP language class cohorts are becoming ever more diverse. It has become evident to tutors in IWLP (as throughout the University) that the linguistic, educational and cultural aspects of a student’s background can play an important role in their language acquisition, often helping some aspects while hindering others. In language learning, they may experience varying success in the development of the four language skills of listening, reading, speaking and writing, performing well in some skills while experiencing difficulty in others.

The Language Learning Advisor scheme and the development of a PLanT project

With this in mind, in the Autumn Term of 2016 I successfully applied for PLanT (Partnerships in Learning & Teaching) funding to provide targeted support sessions in oral work and pronunciation for those students who found these areas more challenging. The aim of the project was to improve their performance, motivation and, crucially, confidence. PLanT funding is awarded by CQSD and RUSU for projects involving both staff and students, and I invited three Language Learning Advisors (two undergraduates from the Department of Modern Languages and European Studies and one multi-lingual undergraduate from Psychology and Clinical Language Sciences) to deliver the sessions. Since these sessions had a particular focus, they were delivered on a small-group basis rather than the one-to-one basis more usual for Language Learning Advisors. They were delivered to students studying German at beginner level.

The three Language Learning Advisors were part of the peer-to-peer Language Learning Advisors scheme, which I have run since 2012. In the scheme, I train students who are successful language learners (usually languages undergraduates in the DMLES or students from the higher stages of IWLP) to advise their peers in DMLES and IWLP on the acquisition of effective language learning strategies, including the development of particular language skills and independent learning. The Advisors help students to develop effective self-evaluation, to reflect on their learning styles and to set achievable long-term and short-term goals in their language learning. Students also benefit from the support and encouragement offered by their Advisors in the continued dialogue of follow-up sessions in which progress is monitored.

Before the PLanT-funded sessions began, I and the Advisors discussed the needs and strategies involved. I monitored the progress of the sessions, and at the end of the academic year the Advisors submitted records of activities completed and materials used, and reflections on their experience. Two Advisors worked with me on preparing a presentation for the LTRF (Learning and Teaching Research Forum) of the International Study and Language Institute in June; the third had already left the University by then but helpfully recorded her contribution on video. The presentation met with a positive response and was a valuable experience for the Advisors, enabling us to inform a wider audience about the PLanT project and about the Language Learning Advisor scheme in general. It also gave the Advisors the opportunity to present at a staff forum.

Project outcomes

This project was a very positive experience. I was able to harness the enthusiasm and creativity of the three Advisors to develop a new student-based initiative which, in at least one case, confirmed an Advisor’s choice of teaching as a career path. The students receiving the support benefited through increased fluency, improved pronunciation and greater confidence; this was clear from their feedback comments, which included: ‘The small-group oral session is helping me a lot, [X] is very kind and patient’, ‘The [tutor] is very friendly. There is an obvious improvement in my pronunciation.’

I intend to continue to run these small-group skills-based sessions in future years, since I believe that they address a clearly-perceived and increasing need. The experience gained this year, together with the Advisors’ reflections and information about materials and activities employed, will be of great value in achieving this end.

Involving students in the appraisal of rubrics for performance-based assessment in Foreign Languages By Dott. Rita Balestrini

By edwardwhite Comment Closed

Context

In 2016, in the Department of Modern Languages and European Studies (DMLES), it was decided that the marking schemes used to assess writing and speaking skills needed to be revised and standardised in order to ensure transparency and consistency of evaluation across different languages and levels. A number of colleagues teaching language modules had a preliminary meeting to discuss what changes had to be made, what criteria to include in the new rubrics and whether the new marking schemes would apply to all levels. While addressing these questions, I developed a project with the support of the Teaching and Learning Development Fund. The project, now in its final stage, aims to enhance the process of assessing writing and speaking skills across the languages taught in the department. It intends to make assessment more transparent, understandable and useful for students; foster their active participation in the process; and increase their uptake of feedback.

The first stage of the project involved:

  • a literature review on the use of standard-based assessment, assessment rubrics and exemplars in higher education;
  • the organization of three focus groups, one for each year of study;
  • the development of a questionnaire, in collaboration with three students, based on the initial findings from the focus groups;
  • the collection of exemplars of written and oral work to be piloted for one Beginners language module.

I had a few opportunities to disseminate some key ideas emerged from the literature review – School of Literature and Languages’ assessment and feedback away day, CQSD showcase and autumn meeting of the Language Teaching Community of Practice. Having only touched upon the focus groups at the CQSD showcase, I will describe here how they were organised, run and analysed and will summarise some of the insights gained.

Organising and running the focus groups

Focus groups are a method of qualitative research that has become increasingly popular and is often used to inform policies and improve the provision of services. However, the data generated by a focus group are not generalisable to a population group as a whole (Barbour, 2007; Howitt, 2016).

After attending the People Development session on ‘Conducting Focus groups’, I realised that the logistics of their organization, the transcription of the discussion and the analysis of the data they generate require a considerable amount of time and detailed planning . Nonetheless, I decided to use them to gain insights into students’ perspectives on the assessment process and into their understanding of marking criteria.

The recruitment of participants was not a quick task. It involved sending several emails to students studying at least one language in the department and visiting classrooms to advertise the project. In the end, I managed to recruit twenty-two volunteers: eight for Part I, six for Part II and eight for Part III. I obtained their consent to record the discussions and use the data generated by the analysis. As a ‘thank you’ for participating, students received a £10 Amazon voucher.

Each focus group lasted one hour, the discussions were entirely recorded and were based on the same topic guide and stimulus material. To open discussion, I used visual stimuli and asked the following question:

  • In your opinion, what is the aim of assessment?

In all three groups, this triggered some initial interaction directly with me. I then started picking up on differences between participants’ perspectives, asking for clarification and using their insights. Slowly, a relaxed and non-threatening atmosphere developed and led to more spontaneous and natural group conversation, which followed different dynamics in each group. I then began to draw on some core questions I had prepared to elicit students’ perspectives. During each session, I took notes on turn-taking and some relevant contextual clues.

I ended all the three focus group sessions by asking participants to carry out a task in groups of 3 or 4. I gave each group a copy of the marking criteria currently used in the department and one empty grid reproducing the structure of the marking schemes. I asked them the following question:

  • If you were given the chance to generate your own marking criteria, what aspects of writing/speaking /translating would you add or eliminate?

I then invited them to discuss their views and use the empty grid to write down the main ideas shared by the members of their group. The most desired criteria were effort, commitment, and participation.

Transcribing and analysing the focus groups’ discussions

Focus groups, as a qualitative method, are not tied to any specific analytical framework, but qualitative researchers warn us not to take the discourse data at face value (Barbour, 2007:21). Bearing this in mind, I transcribed the recorded discussions and chose discourse analysis as an analytical framework to identify the discursive patterns emerging from students’ spoken interactions.

The focus of the analysis was more on ‘words’ and ‘ideas’ rather than on the process of interaction. I read and listened to the discussions many times and, as I identified recurrent themes, I started coding some excerpts. I then moved back and forth between the coding frame and the transcripts, adding or removing themes, renaming them, reallocating excerpts to different ‘themes’.

Spoken discourse lends itself to multiple levels of analysis, but since my focus was on students’ perspectives on the assessment process and their understanding of marking criteria, I concentrated on those themes that seemed to offer more insights into these specific aspects. Relating one theme to the other helped me to shed new light on some familiar issues and to reflect on them in a new way.

Some insights into students’ perspectives

As language learners, students gain personal experience of the complexity of language and language learning, but the analysis suggests that they draw on the theme of complexity to articulate their unease with the atomistic approach to evaluation of rubrics and, at times, also to contest the descriptors of the standard for a first level class. This made me reflect about whether the achievement of almost native-like abilities is actually the standard against which we want to base our evaluation. Larsen-Freeman’s (2015) and Kramsch’s (2008) approach to language development as a ‘complex system’ helped me to shed light on the idea of ‘complexity’ and ‘non-linear relations’ in the context of language learning which emerged from the analysis.

The second theme I identified is the ambiguity and vagueness of the standards for each criterion. Students draw on this theme not so much to communicate their lack of understanding of the marking scheme, but to question the reliability of a process of evaluation that matches performances to numerical values by using opaque descriptors.

The third theme that runs through the discussions is the tension between the promise of objectivity of the marking schemes and the fact that their use inevitably implies an element of subjectivity. There is also a tension between the desire for an objective counting of errors and the feeling that ‘errors’ need to be ‘weighted’ in relation to a specific learning context and an individual learning path. On one hand, there is the unpredictable and infinite variety of complex performances that cannot easily be broken down into parts in order to be evaluated objectively, on the other hand, there is the expectation that the sum of the parts, when adequately mapped to clear marking schemes, results in an objective mark.

Rubrics in general seem to be part of a double discourse. They are described as unreliable, discouraging and disheartening as an instructional tool. The feedback they provide is seen as having no effect on language development as does the complex and personalised feedback that teachers provide. Effective and engaging feedback is always associated with the expert knowledge of a teacher, not with rubrics. However, the need for rubrics as a tool of evaluation is not questioned in itself.

The idea of using exemplars to pin down standards and make the process of evaluation more objective emerges from the Part III focus group discussion. Students considered pros and cons of using exemplars drawing on the same rationales that can be found debated in scholarly articles. Listening to, and reading systematically through, students’ discourses was quite revealing and brought to light some questionable views on language and language assessment that most marking schemes measuring achievement in foreign languages contribute to promote.

Conclusion

The insights into students’ perspectives gained from the analysis of the focus groups suggest that rubrics can easily create false expectations in students and foster an assessment ‘culture’ based on an idea of learning as steady increase in skills. We need to ask ourselves how we could design marking schemes that communicate a more realistic view of language development. Could we create marking schemes that students do not find disheartening or ineffective in understanding how to progress? Rather than just evaluation tools, rubrics should be learning tools that describe different levels of performance and avoid evaluative language.

However, the issues of ‘transparency’ and ‘reliability’ cannot be solved by designing clearer, more detailed or student-friendly rubrics. These issues can only be addressed by sharing our expert knowledge of ‘criteria’ and ‘standards’ with students, which can be achieved through dialogue, practice, observation and imitation. Engaging students in marking exercises and involving them in the construction of marking schemes – for example by asking them how they would measure commonly desired criteria like effort and commitment – offers us a way forward.

References:

Barbour, R. 2007. Doing focus groups. London: Sage.

Howitt, D. 2016. Qualitative Research Methods in Psychology. Harlow: Pearson.

Kramsch, C. 2008. Ecological perspectives on foreign language education. Language Teaching 41 (3): 389-408.

Larsen-Freeman, D. 2015. Saying what we mean: Making a case for ‘language acquisition’ to become ‘language development’. Language Teaching 48 (4): 491-505.

Potter, M. and M. Wetherell. 1987. Discourse and social psychology. Beyond attitudes and behaviours. London: Sage.

 

Links to related posts

‘How did I do?’ Finding new ways to describe the standards of foreign language performance. A follow-up project on the redesign of two marking schemes (DLC)

Working in partnership with our lecturers to redesign language marking schemes 

Sharing the ‘secrets’: Involving students in the use (and design?) of marking schemes

AN ‘APPY CHRISTMAS IN AGRICULTURE: SHARING OUR TEACHING AND LEARNING EXPERIENCES By Dr Alice Mauchline & Prof Julian Park

By edwardwhite Comment Closed

During December 2016, we had the chance to share our teaching and learning experiences here at the University of Reading with thousands of other educators around the world by providing a case study for a seasonal online course called ‘The 12 apps of Christmas’.

The free, open, short, online Continuing Professional Development (CPD) course was run for the third time by the Dublin Institute of Technology. The programme released ‘an app a day’ for the first 12 weekdays in December and over 3,000 participants logged in to get quick outlines of different ways in which they could integrate mobile learning into their teaching and learning practices. The aim was to raise awareness of the benefits of mobile apps and technologies, to provide upskilling for educators and to help expand their personal learning networks. The course was a collaborative effort with case studies from Ireland, UK and the USA and now that it has finished, the site has been left online as an open resource for all to use. It is available here: https://the12appsofchristmas2016.wordpress.com/

The case study was produced in collaboration with colleagues at the Universities of Sheffield and Chester as a dissemination activity for the Enhancing Fieldwork Learning (EFL) project.  The EFL team have been working together to research and share innovations in field teaching and learning with a particular focus on the use of mobile technologies.

The app we focused on for the case study was ‘Geospike’; this app allows instant location recording using the internal GPS of a mobile device, to which photos, videos and field notes can be attached. This functionality means the app can be used as a georeferenced field notebook. The pedagogic case study we wrote described how we used the app to log field sampling sites in Iceland with Final year undergraduates from the University of Reading and the University of Akureyri, Iceland on a joint Microbiology field-based module led by Prof Rob Jackson (School of Biological Sciences).

Photos from the Iceland fieldtrip showing students using the iPads to log their sampling locations in GeoSpike (we gratefully acknowledge the Annual Fund for their support in purchasing a set of iPads to support field learning at Reading)

The experience of sharing our pedagogic innovations through the 12 apps of Christmas provided us with the opportunity to interact with educators, students, librarians and learning technologists across the globe. The cohort included people with a multitude of different subject backgrounds and experiences which led to very interesting conversations through Twitter and exchanges of comments on the website.

Frances Boylan @boylanfm A map of #12appsDIT followers (https://twitter.com/boylanfm/status/808324692109119488)

Several other apps with similar functionality to Geospike were discussed along with many suggestions of alternative, innovative uses of this kind of app in teaching and learning activities. Our favourite feedback was on Twitter from @LeithaD “#12appsDIT Really love the case study for GeoSpike. A nifty app is one thing, but a well-constructed learning activity is even better!”

Learn more:

Enhancing Fieldwork Learning https://enhancingfieldwork.org.uk/

12 Apps of Christmas https://the12appsofchristmas2016.wordpress.com/

Take part:

DIT aren’t running the 12 Apps of Christmas in 2017 but there are a couple of others to try this year:

Reflections on university transition from a new staff member By Dr Alana James

By edwardwhite Comment Closed

I started university this year, or at least it feels like I have upon starting my new job as a Lecturer in the School of Psychology and Clinical Language Sciences (PCLS). Every face around me is unfamiliar, the campus seems an unnerving maze, and simple processes have become logic puzzles. Oh the joy I felt at using a printer successfully (let’s not mention the attempts at scanning a document). There are many enjoyable aspects – meeting lovely new colleagues and joining in the School’s coffee mornings for example – but the transition is more disorientating than I expected. At the end of my first week I was grateful for some downtime at home, and found myself reflecting upon how my experience compares with the transition to university for new students.

New students face the same challenges I am but may also be living independently, away from their support network, for the first time. Many go home each day to a new place and have to figure out new washing machines and cookers never mind printers, as well as try to get along with housemates. For those commuting there are other challenges, including being at the mercy of traffic or public transport, and trying to forge friendships between classes. I have worked in universities before, and am able to draw upon previous experience; many new students arrive without having spent much, if any, time in a higher education environment. We know that factors such as being the first in your family to go to university or having a disability can make the transition even harder.

My own disorientation in these first days at the University of Reading has reminded me how all-encompassing the transition to university can be. As an academic my focus is often upon ensuring my new students have the academic skills needed to be an independent learner, but it’s important to be mindful that this is just one aspect of the overall transition experience. It’s easy to forget that the initial onset of new faces, places, and challenges can be mentally and physically wearing as well as exciting. When I meet my new students at the start of the next academic year I will try to recall how I felt when I joined the UoR.

One of the influencing factors in my decision to join the UoR was its commitment to student support, particularly mentoring. Harnessing our students’ potential to support each other through mentoring can ease mentees’ transition into university, whilst developing the mentors’ own skills and experience. I have previously run a scheme where psychology students mentored A-level pupils, giving them an insight into what university life is really like, and found that the mentors also benefited in terms of developing transferable skills and ideas about careers. Some recent research with my collaborator found that specialist mentoring, between qualified staff and mentees, is an effective form of support for students with mental health conditions and autism. I will certainly be encouraging my future students at the UoR to make the most of the STaR mentoring scheme and the mentoring connected to the Study Smart online course, first as mentees and later as mentors.

As for me, I am very much looking forward to the meetings with my staff mentor.