Clinical skills development: using controlled condition assessment to develop behavioural competence aligned to Miller’s pyramid

By edwardwhite Comment Closed

Kat Hall,  School of Chemistry, Food and Pharmacy, k.a.hall@reading.ac.uk

Overview

The Centre for Inter-Professional Postgraduate Education and Training (CIPPET) provide PGT training for healthcare professionals through a flexible Masters programme built around blended learning modules alongside workplace-based learning and assessment.  This project aimed to evolve the department’s approach to delivering one of our clinical skills workshops which sits within a larger 60 credit module.  The impact was shown via positive student and staff feedback, as well as interest to develop a standalone module for continuing further learning in advanced clinical skills.

Objectives

The aim of this project was to use controlled condition assessment approaches to develop behavioural competence at the higher levels of Miller’s pyramid of clinical competence 1.

Miller’s Pyramid of Clinical Competence

The objectives included:

  1. engage students in enquiry by promoting competence at higher levels of Miller’s pyramid
  2. develop highly employable graduates by identifying appropriate skills to teach
  3. evolve the workshop design by using innovative methods
  4. recruit expert clinical practitioners to support academic staff

Context

Health Education England are promoting a national strategy to increase the clinical skills training provided to pharmacists, therefore this project aimed to evolve the department’s approach to delivering this workshop.  The current module design contained a workshop on clinical skills, but it was loosely designed as a large group exercise which was delivered slightly differently for each cohort.  This prevented students from fully embedding their learning through opportunities to practise skills in alongside controlled formative assessment.

Implementation

Equipment purchase: As part of this project matched funding was received from the School to support the purchase of simulation equipment which meant a range a clinical skills teaching tools could be utilised in the workshops.  This step was undertaking collaboratively with the physician associate programme to share learning and support meeting objective 2 across the School.

Workshop design: the workshops were redesigned by the module convenor, Sue Slade, to focus on specific aspects of clinical skills that small groups could focus on with a facilitator.  The facilitators were supported to embed the clinical skills equipment within the activities therefore promoting students in active learning activities.  The equipment allowed students the opportunity to simulate the skills test to identify if they could demonstrate competence at the Knows How and Shows How level of Miller’s Pyramid of Clinical Competence.  Where possible the workshop stations were facilitated by practising clinical practitioners.  This step was focused on meeting objectives 1, 2, 3 and 4.

Workbook design: a workbook was produced that students could use to identify core clinical skills they required in their scope of practice and thus needed to practise in the workshop and further in their workplace-based learning.  This scaffolding supported their transition to the Does level of Miller’s Pyramid of Clinical Competence.  This step was focused on meeting objectives 1 and 3.

Impact

All four objectives were met and have since been mapped to the principles of Curriculum Framework to provide evidence of their impact.

Mastery of the discipline / discipline based / contextual: this project has supported the academic team to redesign the workshop around the evolving baseline core knowledge and skills required of students.  Doing this collaboratively between programme teams ensures it is fit for purpose.

Personal effectiveness and self-awareness / diverse and inclusive: the positive staff and student feedback received reflects that the workshop provides a better environment for student learning, enabling them to reflect on their experiences and take their learning back to their workplace more easily.

Learning cycle: the student feedback has shown that they want more of this type of training and so the team have designed a new stand-alone module to facilitate extending the impact of increasingly advanced clinical skills training to a wider student cohort.

Reflections

What went well? The purchase of the equipment and redesigning the workshop was a relatively simple task for an engaged team, and low effort for the potential return in improved experience.  By having one lead for the workshop, whilst another wrote the workbook and purchased the equipment, this ensured that staff across the team could contribute as change champions.  Recruitment for an advanced nurse practitioner to support the team more broadly was completed quickly and provided support and guidance across the year.

What did not go as well?  Whilst the purchase of the equipment and workshop redesign was relatively simple, encouraging clinical practitioners to engage with the workshop proved much harder.  We were unable to recruit consistent clinical support which made it harder to fully embed the project aims in a routine approach to teaching the workshop.  We considered using the expertise of the physician associate programme team but, as anticipated, timetabling made it impossible to coordinate the staffing needs.

Reflections: The success of the project lay in having the School engaged in supporting the objectives and the programme team invested in improving the workshop.  Focusing this project on a small part of the module meant it remained achievable to complete one cycle of change to deliver initial positive outcomes whilst planning for the following cycles of change needed to fully embed the objectives into routine practice.

Follow up

In planning the next series of workshops, we plan to draw more widely on the University alumni from the physician associate programme to continue the collaborative approach and attract clinical practitioners more willing to support us who are less constrained by timetables and clinical activities.

Based on student and staff feedback there is clearly a desire for more teaching and learning of this approach and being able to launch a new standalone module in 2020 is a successful output of this project.

Links and References

Miller, G.E. (1990). The assessment of clinical skills/competence/performance. Acad Med, 65(9):S63-7.

International Law Mooting

By edwardwhite Comment Closed

Professor James Green, Law
j.a.green@reading.ac.uk
Year of activity: 2007-08

Overview

Since 2007, the Law School has run a Part Three module entitled ‘International Law Mooting’. This is a highly innovative module, where a team of four students participate in the prestigious Telders International Law Moot Court Competition. The competition involves the team presenting written – and then, crucially, oral – arguments on a fictional dispute in international law.

Objectives

  • Memorials are jointly written and a single mark is given to all students: this builds teamwork, and prepares students for the submission of written memorials in real cases.
  • The oral performance is assessed, meaning that advocacy and presentation skills are developed.
  • Students are also assessed on individual reflective portfolios, which reward reflective learning and emphasize skill development.

Context

The team competes externally, for the University of Reading, against other universities. This gives the University of Reading a profile nationally and internationally, and provides students with a wonderful experience. The work required to compete in the competition is significant, and so – after entering for the first time in 2006 as an extra-curricular activity – it was decided that student effort here had to be rewarded with appropriate degree credit, hence the creation of the module. It develops a wide range of practical legal skills that are simply not part of other, traditional, law modules.

Implementation

Of the various issues that arose with regard to implementing the mooting module, the most pertinent for possible implementation elsewhere is the manner in which this module was to be assessed to give best effect to its learning objectives. A key learning objective was to develop communication and advocacy skills – but there is a danger of placing emphasis entirely on the student’s performance in the single external moot. Pressure is high, and ‘stage fright’ very possible. It is also difficult to ensure quality review of the marking of oral presentations/mooting. It was therefore decided that this issue could be addressed by complementing the marks awarded for the oral performance by also awarding a percentage of the marks for a reflective assessment. This ensured that students gained the credit that they were due for their skill development across the module as a whole, and not just based on the moot final alone.

Impact

The module has been hugely successful over the years. Students consistently give extremely positive feedback on the unique module design, and team-orientated nature of the module. It is also almost always the case that students gain extremely high marks in the module, with a significant number of firsts having been awarded. Indeed, no student has achieved an overall module mark below the 2:1 classification in 9 years of running the module.

Reflections

We have, of course, reflected on the module over the years. One change we made was to increase the percentage of the overall grade for the oral performance, and to slightly reduce the amount for the portfolio. This was in response to student feedback – we had the balance a little too heavily on rewarding the reflection, and students felt they should get rather more credit for the moot itself. We feel, after reflection and a few tweaks to the module design, that the assessment methods now best suit the learning outcomes. By and large, though, the module is a resounding success and continues to run in a form that is not too dissimilar from what was originally envisaged in 2007.

Follow up

Nothing beyond what is stated in the ‘Reflections’ box, above.

Links

The Module Description Form for International Law Mooting: http://www.reading.ac.uk/modules/document.aspx?modP=LW3ILM&modYR=1617

The website for the external Telders competition: http://teldersmoot.com/

Assessing the use of Technology Enhanced Learning in Higher Education: the case of trading simulation software at the ICMA Centre

By edwardwhite Comment Closed

Dr Ioannis Oikonomou, ICMA Centre
i.oikonomou@icmacentre.ac.uk
Year(s) of activity: 2013-14

Overview

8948This project reviewed the effectiveness of the ICMA Centre’s use of trading simulation software, a unique combination of problem-based learning and role-playing which uses modern technology.  While it was found that students enjoyed having access to trading simulation software for their learning, a number of areas in which improvements could be made were identified, and recommendations were made to effect these.

Objectives

  • To assess the effectiveness of the ICMA Centre’s use of trading simulations software.
  • Highlight areas for improvement and make suggestions about the possible restructuring of the content of the offered trading simulation modules and ways of further enhancing their academic and practical usefulness for students.

Context

The ICMA Centre has three dealing rooms, which are used for conducting small group seminars, workshops and trading simulation sessions for modules at both undergraduate and postgraduate levels, as well as being a valuable tool for outreach purposes.

Although the ICMA Centre has been subject to periodic and contextual review, there has been no formal investigation that specifically targets the teaching and learning issues and transferable skills of the trading simulation software.

Implementation

To assess the effectiveness of the use of these facilities, historic feedback was analysed. The ICMA Centre had regularly undergone periodic and contextual reviews as according to University of Reading policy, with these reviews evaluating all aspects of the ICMA Centre’s programmes. This was therefore a valuable resource for understanding the strengths and weaknesses of the use of trading simulation software for teaching and learning within the wider context of the ICMA Centre’s delivery of programmes.

Also analysed were evaluation forms connected to trading sessions at both the undergraduate and postgraduate level for three academic years. The great benefit of these data were that it allowed quantitative analysis of trading simulation software, as students gave numerical scores to indicate their satisfaction. Qualitative data were also available, with students providing free text comments, which give specific details about what had worked well and what might need improving.

Interviews were conducted with module convenors and teaching assistants. This allowed greater detailed information to be generated on the strengths and weaknesses of trading simulation sessions, and offered the chance to discuss module convenors’ and teaching assistants’ perspective on trading simulation sessions. Additionally, interviews with staff were valuable for capturing some of the informal opinions and attitudes of students, which may not have expressed in formal evaluations.

The guidance offered by these analyses was used to formulate an online questionnaire in order to generate quantifiable data.  Finally, two student focus groups, one of undergraduate students and one of postgraduate students, were interviewed in order to expand upon the findings of the questionnaire. Effort was made to accurately represent the diversity of student backgrounds on ICMA programmes in the focus groups.

Reflections

Historic evaluation forms, interviews with module convenors and teaching assistants, the online questionnaire, and the focus groups had comparable findings.  Overall, students very much enjoyed the use of trading simulation software, and generally found it to be user-friendly, reasonable and realistic.  The realism and ‘hands-on’ nature of the platform are particularly beneficial characteristics, as adult learners tend to focus on tasks, especially when they believe they may encounter these in their lives.  The trading simulations were highlighted as being effective tools for the development of employable skills, and helped students to internalise complex financial concepts.

The principal negative aspects of users experiences of trading simulation software that were raised at multiple points during the study, were that students wanted more time using the trading simulation software, and better connection between lectures and use of the trading simulation software.  This was most keenly felt by undergraduate students, who receive significantly fewer trading hours than postgraduates, and who felt that the sessions could be better embedded into their teaching and learning portfolios. As a result of these findings, a number of recommendations were made for improving the delivery of teaching and learning with the use of trading simulation sessions.

Follow up

Progress has been made on fulfilling the recommendations of the report: Trading Simulation II has been moved from the Financial Modelling module to the more suitable Debt Markets and Instruments; module convenors have instructions to be mindful of the link between their lectures and trading simulation sessions, and for how performance in trading simulations sessions is to be benchmarked; alterations and additions have been made to the simulation software’s scenarios so that it can be utilised for different learning outcomes; availability of trading simulation sessions has been increased, and trading hours for students have been increased; students are given firm guidance and information on the interpretation of and access to their feedback; and an experienced trader has been employed as a sessional lecturer for the undergraduate training sessions.

With these alterations having been made, feedback on trading simulation sessions has improved, and students demonstrate deep and broad levels of learning on concepts they are able to explore through the use of trading simulation sessions.