Constructing research methods and statistics teaching

By edwardwhite Comment Closed

Dr Lotte Meteyard, School of Psychology and Clinical Language Sciences
l.meteyard@reading.ac.uk

Overview

Statistics teaching to Speech and Language Therapists within the Department of Clinical Language Sciences was redesigned in response to module evaluations. Whereas students had previously reported anxiety about statistics and struggled to appreciate the relevance of statistics to their practice, the introduction of formative learning activities which integrated statistics teaching with other module content produced a reduction in anxiety about statistics, a benefit to students’ grades, and an increase in student module satisfaction with their statistics training.

Objectives

  • Increase the opportunities for students to consolidate and revisit knowledge of key concepts.
  • Make explicit links within and across the teaching content to clinical practice.
  • Provide learning activities, outcomes and objectives that are clear to students.

Context

The Research Proposal (PL3RPR) module is compulsory for all Part Three undergraduate and taught postgraduate students within the Department of Clinical Language Sciences. The module provides research methods and statistics teaching, and during the module students plan a research module and complete an ethics application, with these being used for their dissertations. Feedback, however, revealed that students found the statistics lectures confusing and poorly related to other module content. Having teaching provided by a number of staff members contributed to the module having a fragmentary nature.

Implementation

In order to increase the opportunities for students to consolidate and revisit knowledge of key concepts, technology, multiple practice and collaboration were focused on in order to create frequent, meaningful activities for students to complete. Lecture handouts were provided separate from the lecture slides in order to encourage engagement during lectures, and practical activities were used to teach basic quantitative concepts and research design. During activities, analyses of data was completed as a class, and formative exercises were set each week, involving a short reading and answering focused questions on that reading. These assessments were revisited at the start of each lecture in order to feedback and discuss answers to questions. In labs, written instructions were replaced with short videos demonstrating how to complete particular procedures. Worksheets required students to write out results and answer questions about the interpretation of data. The answers to these worksheets were made available on Blackboard Learn after the end of each lab class. For each week of statistics teaching an online multiple choice questionnaire was provided, offering students optional online practice in preparation for the statistics class test. Students were encouraged to have the statistical analysis software PASW or SPSS installed on their home devices to allow them to practice away from lectures.

To make explicit links within and across the teaching content and to clinical practice, the content of the module was restructured so that students were introduced to a particular concept, with this concept then being revisited in later activities. In order to build explicit links with clinical practice students were asked to identify why research skills are important for clinical work, to complete formative assessments that involved reading chapters on healthcare research or journal articles from speech therapy research. Key readings were taken from ‘real world’ sources, such as the magazine of the Royal College of Speech and Language Therapy or the NHS. Reflection was encouraged through formative assignments which were discussed in the following week’s lecture. These required student to identify why healthcare research is critical to practice, critique a randomised controlled trial, and identify research designs and statistical tests in clinically relevant journal articles.

To provide learning activities, outcomes and assessment tasks that are aligned and clear to students, the learning outcomes of the module were rewritten and linked directly to the summative assessments. Three summative assessments were themselves changed so that students prepared a research proposal poster, an ethics application and a statistics class test. The research proposal poster was introduced to give practice at a professional skill and reduce the duplication of content between making the proposal and the ethics application. By making the ethics application a summative assessment, students would be assessed on something directly relevant to the completion of their project, while only minimal staff input would be required for the document to be submitted to the School ethics committee. The weighting of each piece of assessment was changed so that they contributed more evenly to the overall module mark. Learning activities were designed to support students in accessing and evaluating literature, generating research designs and statistical analyses. By providing research proposal and ethics application examples, and templates for their own on Blackboard Learn alongside detailed guidelines for completing coursework, student were encouraged to seek out supporting material independently.

Impact

Students on the module completed a statistical knowledge multiple choice questionnaire during the first week of the module and again after completing their statistics class test. They also completed the Statistical Anxiety Scale before beginning the module, and again at the end of the module. Results demonstrated that statistical knowledge increased, with students’ median score going from 11/20 before the course (with a range between 5-14) to 15/20 after the course (with a range between 8 and 19). There was also a reduction in anxiety about statistics. Results also demonstrated that there was a significant positive correlation between the number of formative multiple choice questionnaires a student completed and their final score on the statistics class test. Median marks in the class test and research proposal both improved from the previous year, with no students failing the statistics class test. Student module satisfaction also increased.

Reflections

Situating the statistics and research methods teaching in practical activities and in the context of students’ professional learning was one of the most powerful changes made to the module: students responded positively to practical activities used to demonstrate statistical concepts. While full participation could not be guaranteed, enough students completed tasks to allow discussion and review of these at the beginning of each lecture.

By using technology for students to practice skills away from the classroom, students were able to increase their knowledge of statistics after the course. It was particularly gratifying to see the correlation between the number of multiple choice questionnaires completed on Blackboard Learn and the attainment of students during the statistics class test.

Having resources external to the classes available, the module convenor could be assured that students could have sufficient time and experience with concepts and software.

Improving student engagement with assessment and feedback through peer review

By edwardwhite Comment Closed

Professor Helen Parish, School of Humanities
h.l.parish@reading.ac.uk

Year of activity: 2014-15

Overview

9070

The project investigated recent research and practice in peer assessment and feedback in order to implement a peer assessment model for use within History, and develop a framework for the adoption of said model in cognate disciplines where evaluation of substantial text-based assignments is an important part of assessment.

Objectives

  • Present students with well-managed opportunities to engage in feedback and assessment and learn from it.
  • Present staff with access to tried and tested models for implementation that can be used and tailored across disciplines.

Context

The importance of increasing the impact of assessment in feedback and learning is recognised by the University’s teaching and learning enhancement priorities, and is evident in the ‘Engage in Assessment’ and ‘Engage in Feedback’ materials.  The requirement to pursue an agenda for feedback is also highlighted by the expectations of employers that graduates of the University of Reading will be able to assess and evaluate the work of others, by comments on feedback made by University of Reading students in the National Student Survey, and by discussions with potential students on Open Days.

Implementation

There were five stages to the project:

  1. A literature search on the topic and detailed engagement with recent scholarship, undertaken by the Principal Investigator.
  2. A ‘competitor analysis’, undertaken by a research assistant, looking at the extent that peer feedback is present on Humanities curricula at other institutions.
  3. Development of a model for the trial of peer assessment informed by the previous two stages.
  4. Implementation of this model as a ‘pilot project’ in the Department of History.
  5. Obtaining student feedback on the process and reflection by the Principal Investigator.

The feedback gained during the early stages of the project revealed that students were reluctant to allow their work to be reviewed by their peers, even when anonymised.   This necessitated the envisaged model to be altered, whereby the written work being ‘peer reviewed’ was either from previous cohorts within the Department or alternative sources.

Once the pilot project was developed, there were three stages:

  1. Development of an understanding of marking and assessment criteria. Students read the assessment criteria of their module, and were then tasked with rewriting these in their own words.
  2. Applying these criteria to written work. Students then read a sample essay (not taken from the group), and with reference to the marking criteria, were asked to give a mark to the essay, with a summary of reasons they had come to this judgment.  This was followed by a discussion of the written feedback provided.
  3. Focus group and project review.  It was intended that students would meet to talk about the project, and more general issues to do with assessment and feedback, in the presence of an experienced observer external to the department.

Impact

One of the principal benefits of the project was that students became more aware of the marking criteria by which their assignments were assessed, as although they found these clear, few students had actually taken the time to read these before. An additional benefit was that the activity helped develop students’ academic confidence, as they were impelled to adopt a critical attitude to writing within scholarship, and gained experience of promoting their point of view to their peers.

Reflections

Feedback from questionnaires suggested that students enjoyed the project; that they now had a better understanding of assessment and feedback; that the project had been helpful with the preparation of their own written work; and that they were now more confident in the assessment of their own work prior to submission.

The reluctance of students to submit their own work to review by their peers meant that there was a less direct link between the peer feedback provided and the specific assignment for each module.  By using work from previous cohorts or alternative sources, however, it was possible to get students to engage more willingly with the process of peer review.

The main disappointment was that it proved impossible to gather a large enough group of students to participate in the focus group stage of the project.  This may have been due to the proposed scheduling of the focus groups at a time when students had recently participated in a Departmental Periodic Review and submitted their final coursework of the academic year.  Nevertheless, valuable feedback on the pilot was provided through questionnaires and verbal communication.

It was interesting to observe that students held broad spectrum of ideas about what constituted good work, arising from a lack of understanding about the criteria against which work is marked. From this perspective, the project was valuable, as students were familiarised with the marking criteria and how these applied to written pieces Students were able to look ‘behind the scenes’ at the marking process, with student applying the marking criteria as individuals, but then needing to decide as a group upon a final mark for pieces they were reviewing.

Follow up

Following the pilot project, the use of peer review to engage students in assessment and feedback has been used by other members of staff within the Department of History, with similar success. Other than the specific pieces of work and criteria used for peer review purposes, there was nothing within this project that was specific to the Department of History or School of Humanities, and so this activity could easily be adapted for use in other Departments and Schools across the University.

The peer review approach has been successfully applied within the Department of History to student presentations in seminars. As student presentations are more ‘in the moment’ and designed with a peer audience in mind, students have not expressed the same reticence to have their peers review their work, and those presenting have appreciated receiving immediate feedback.

Exploring modern languages linguistics

By edwardwhite Comment Closed

Dr Federico Faloppa and Dr Chiara Ciarlo, School of Literature and Languages
f.faloppa@reading.ac.uk

Overview

12756The project successfully developed an introductory module in general linguistics, with a focus on foreign language specific issues for Part Two students who choose to do a single or joint honours language degree.  Providing a module for the teaching of linguistics to Modern Languages and European Studies students has had many benefits for the students, who report that the module has helped with their study of foreign languages.

Objectives

  • To determine which linguistics topics Part Two students would have an interest in, and benefit most from, studying in-depth.
  • To design a linguistics course for Modern Languages and European Studies students.
  • Provide students with theoretical knowledge which they can transfer to the study and understanding of other languages.
  • Provide students with skills that will improve their employability.

Context

Previously, the Department of Modern Languages and European Studies had offered only individual language tutorials or modules in the history of languages, and language in society, but little aimed at providing a general theoretical linguistic background of the languages that are taught.

Feedback on the language modules from previous years highlighted that a number of Modern Languages and European Studies students desired linguistics training. Although students were able to take courses offered by the Department of English Language and Applied Linguistics, these were only of partial help to language students, as these courses mainly focus on the English language, and are not designed to have a supporting role in the study of foreign languages. As a result, it was necessary to design and pilot a linguistics course for Modern Languages and European Studies students, including core linguistic principles and more language-specific issues, with an eye to recurrent errors in the students’ language production on which they would be able to reflect.

Implementation

The first six months of the project were devoted to the gathering and analysis of resources in order to carry out research activity on aspects of the teaching of linguistics in modern foreign languages degrees.

Research activity was conducted to compare the level of linguistics provision in modern foreign languages degrees in the UK, and to establish what areas of linguistics are given more prominence in modern foreign languages curricula. The results of this research contributed to the creation of a network of experts in the field of the linguistics of modern foreign languages, who were later invited to present their views on the topic in a workshop held at the University of Reading.

For the one-day workshop, experts in the field of the linguistics of modern foreign languages were invited to present their latest research. This event was addressed to all staff and students within Modern Languages and European Studies, English Language and Applied Linguistics, the International Study and Language Institute, and the Institute of Education, in order to generate a shared discussion on the integration between language study and the study of language.

These activities fed into the creation of a taster session for phonetics, phonology and syntax, to which Part One students were invited to attend. At the end of the event, students were asked to give their feedback on the relevance, usefulness, or difficulty of what was explained during the taster sessions. This feedback was valuable for helping finalise the pilot module description.

The pilot module description was then approved, and the new module was taught during the 2013-14 academic year.

Impact

The project was successful, as it achieved its principle aim of creating a module to teach linguistics for Modern Languages and European Studies students, with the course structure and content having been established through a consultative process in order to ensure that students are provided with a module that meets their expectations of a linguistics course, and is able to provide students with a theoretical understanding of linguistics that should support their learning of modern languages, and with skills that will more generally enhance their employability.

Teaching linguistics to Modern Languages and European Studies students has been of great benefit to the students. Teaching staff within the Department have noted that students taking linguistics modules have more confidence and accuracy in their pronunciation when speaking foreign languages, and generally make fewer errors.

Reflections

Beyond its use to refine the module that would be taught, the taster session was beneficial as it highlighted the benefits that students receive from taster sessions with regard to their making module choices: as a result, the School explored the possibility of providing taster sessions for students to guide them in choosing their modules. Additionally, the provision of such taster sessions is valuable as it provides information on student expectations for module convenors, who can plan and design their modules so that they better meet these expectations.

The success of the project lead to the establishment of a Language and Linguistics Workgroup in order to investigate the implementation and coordination of linguistics teaching within the School.

Students have found the formal learning of linguistics very useful for their study of Modern Languages. With a better understanding of linguistic theory, students are better able to appreciate the errors they make within their own study. Students appreciate the challenge of learning linguistics, but some aspects, for example phonetics and syntax, are very technical, and students seemed to find these the most difficult. To help students meet the challenge, different approaches to teaching these topics have been utilised within the the module, such as creating visual representations of syntax, or using information technology in the teaching of phonetics.

Follow up

The pilot year of teaching the module was greatly successful, and as a result the opportunity to learn linguistics was opened up to all students within the School of Modern Languages and European Studies. Whereas the pilot module was a Part Two module, the module has now been redesigned to allow its provision at Part One. By having the module provided at Part One, students are now able to obtain a strong foundation in the linguistic theory that underpins their study of a modern language before they go on to more in-depth study in Parts Two and Three. Additionally, while teaching linguistics at Part One requires some aspects of the module to be simplified, it is able to contribute to a pathway in Modern Languages and European Studies at the University of Reading, building up students’ linguistic knowledge over the course of their undergraduate study.

The project has also opened up the possibility for interdisciplinary cooperation: bilingual students from Modern Languages and European Studies collaborated on a project with Neurolinguistics students from the School of Psychology and Clinical Language Sciences.

LS1TAL Techniques and Skills for Applied Linguistics: Improving the student experience

By edwardwhite Comment Closed

Professor Jane Setter, School of Literature and Languages
j.e.setter@reading.ac.uk
Year of activity: 2014-15

Overview

12759 (1)Students and staff worked together during 2014-15 to develop the Department of English Language and Applied Linguistics’ new compulsory Part One module Techniques and Skills for Applied Linguistics (LS1TAL), run for the first time in 2014-15, to make it more interesting, useful and relevant to 21st Century undergraduate students. The project sought, among other things, to address the University Board for Teaching and Learning Enhancement Priority to evolve our approaches to teaching and learning, with a specific focus on using Technology Enhanced Learning in class and in assessment.

Objectives

  • An improved module following input from students which addresses their needs more closely in the development of the study of our discipline and transition to Higher Education.
  • Better use of Technology Enhanced Learning elements in content, delivery and assessment.
  • Enhanced use of the formative assignment English Language and Applied Linguistics students are asked to prepare, and of the feedback given.
  • The development of good practice which can be shared across our modules and with others in the University and across the sector.

Context

The module convenor had produced a new compulsory module for 2014-15 which addressed many of the issues arising for students in the first year of study, such as general transition to Higher Education; how to do tertiary-level academic writing; how to use and get the best out of tools such as Turnitin; how to present yourself effectively online with a jobs-market orientation; how to do assessments using Technology Enhanced Learning approaches, such as blogging and short videos. Student involvement in the development of the module was desired to make it particularly relevant and useful to them.

Implementation

Students taking the module were recruited during the Autumn term 2014-15 to be leaders and participants in the development of the module; four came forward. PLanT funding was applied for and received. Subsequently, focus groups were held at four points during the year between students and staff to review module content and suggest ways to develop it. Finally, with the agreement of the students, a new weekly schedule for the module (which runs in the Autumn and Spring terms) was drawn up.

Specifically, students asked for more concise and focussed input on transitions to Higher Education and online presence, including a specific session on building a LinkedIn profile, a more hands-on approach to library skills training, input from students at Parts Two and Three, and sessions from Student Counselling and Wellbeing.

Impact

Students and staff involved felt that the outcomes had been very positive. Students valued having the chance to develop a new module, and staff enjoyed working with students, finding out what they already knew and what needed further development, and understanding their viewpoint on their needs as new entrants to Higher Education in the UK. The real test for the module, however, will be the revised module running in the 2015-16 academic year, during which data will be collected from the new cohort using module evaluation forms to compare with last year’s evaluations and see whether student satisfaction has improved.

We were delighted that students had no issues with the Technology Enhanced Learning elements in the module content or assessment. In fact, they thought they were very useful and well-integrated.

An unexpected outcome has been the Director of English Language and Applied Linguistics’ Postgraduate taught programmes’ interest in the module to see whether it could be adapted for MA students.

Reflections

The activity was made successful by the involvement of students in developing this module. Teaching and Learning Dean Dr David Carter had commented that it seemed like a very well-designed module; the decision about how to develop the module could have been made by an individual based solely on student feedback questionnaires, but it seemed much better to have the hands-on involvement of students, as the module is aimed at supporting students through Part One. Their involvement enabled students to have input into their degree which resulted in real change, to see first-hand the issues involved in module development, and also to appreciate the Department’s attention to teaching, learning, the development of transferable skills, and supporting students to get the best from their university study and beyond.