Engaging students in assessment design

By laurenmccann Comment Closed

Dr Maria Kambouri-Danos, Institute of Education

m.kambouridanos@reading.ac.uk

Year of activity 2016/17

Overview

This entry aims to share the experience of re-designing and evaluating assessment in collaboration with students. It explains the need for developing the new assessment design and then discusses the process of implementing and evaluating its appropriateness. It finally reflects on the impact of MCQ tests, when assessing students in higher education (HE), and the importance of engaging students as partners in the development of new assessment tools.

Objectives

  • To re-design assessment and remove a high-stakes assessment element.
  • To proactively engage ‘students as partners’ in the development and evaluation of the new assessment tool.
  • To identify the appropriateness of the new design and its impact on both students and staff.

Context

Child Development (ED3FCD) is the core module for the BA in Children’s Development and Learning (BACDL), meaning that a pass grade must be achieved on the first submission to gain a BA Honours degree classification (failing leads to an ordinary degree). The assessment needed to be redesigned as it put the total weight of students’ mark on one essay. As the programme director, I wanted to engage the students in the re-design process and evaluate the impact of the new design on both students and staff.

Implementation

After attending a session on ‘Effective Feedback: Ensuring Assessment and Feedback works for both Students and Staff Across a Programme’ I decided to explore more the idea of using Multiple Choice Tests (MCQ). To do so, I attended a session on ‘Team Based Learning (TBL)’ and another on ‘MCQ: More than just a Test of Information Recall’, to gather targeted knowledge about designing effective MCQ questions.

I realised that MCQ tests can help access students’ understanding and knowledge and also stimulate students’ active and self-managed learning. Guided by the idea of ‘assessment for learning’, I proposed the use of an MCQ test during a steering group meeting (employees and alumni) and a Board of Studies (BoS) meeting, which 2nd year Foundation Degree as well as BACDL student representatives attended. The idea was resisted initially, as MCQ tests are not traditionally used in HE education departments. However, after exploring different options and highlighting the advantages of MCQ tests, the agreement was unanimous. At the last BoS meeting (2016), students and staff finalised the proposal for the new design, proposing to use the MCQ test for 20% of the overall mark, keeping the essay for the remaining 80%.

At the beginning of 2017, I invited all BACDL students to anonymously post their thoughts and concerns about the new design (and the MCQ test) on Padlet. Based on these comments, I then worked closely with the programme’s student representatives and had regular meetings to discuss, plan and finalise the assessment design. We decided how to calculate the final mark (as the test was completed individually and then in a group) as well as the total number of questions, the duration of the test, etc.  A pilot study was then conducted during which a sample MCQ test was shared with all the students, asking them to practise and then provide feedback. This helped to decide the style of the questions used for the final test, an example of which is given below:

There are now more than one million learners in UK schools who speak English as an additional language (EAL). This represents a considerable proportion of the school population, well above 15 per cent. To help EAL children develop their English, teachers should do all the following, except…

a. use more pictures and photographs to help children make sense of new information.

b. use drama and role play to make learning memorable and encourage empathy.

c. maintain and develop the child’s first language alongside improving their English.

d. get children to work individually because getting them into groups will confuse them and make them feel bad for not understanding.

e. provide opportunities to talk before writing and use drills to help children memorise new language.

Impact

Students were highly engaged in the process of developing the new design, and the staff-student collaboration encouraged the development of bonds within the group. The students were excited with the opportunity to actively develop their own course and the experience empowered them to take ownership of their own learning. All of them agreed that they felt important and as a student representative said, “their voices were heard”.

The new design encouraged students to take the time to gauge what they already know and identify their strengths and weaknesses. Students themselves noted that the MCQ test helped them to develop their learning as it was an additional study opportunity. One of them commented that “…writing notes was a good preparation for the exam. The examination was a good learning experience.” Staff also agreed that the test enabled students to (re)evaluate their own performance and enhance their learning. One of the team members noted that the “…test was highly appropriate for the module as it offered an opportunity for students to demonstrate their proficiency against all of the learning outcomes”.

Reflections

The new assessment design was implemented successfully because listening to the students’ voice and responding to their feedback was an essential part of the designing process. Providing opportunities to both students and staff to offer their views and opinions and clearly recognising and responding to their needs were essential, as these measures empowered them and helped them to take ownership of their learning.

The BACDL experience suggests that MCQ tests can be adapted and used for different subject areas as well as to measure a great variety of educational objectives. Their flexibility means that they can be used for different levels of study or learning outcomes, from simple recall of knowledge to more complex levels, such as the student’s ability to analyse phenomena or apply principles to new situations.

However, good MCQ tests take time to develop. It is hoped that next year the process of developing the test will be less time-consuming as we already have a bank of questions that we could use. This will enable randomisation of questions which will also help to avoid misconduct. We are also investigating options that would allow for the test to be administered online, meaning that feedback could be offered immediately, reducing even further the time/effort required to mark the test.

Follow up

MCQ tests are not a panacea; just like any other type of assessment tool, MCQ tests have advantages and limitations. This project has confirmed that MCQ tests are adaptable and can be used for different subject areas as well as to measure a great variety of educational objectives. The evaluation of the assessment design will continue next year and further feedback will be collected by the cohort and next year’s student representatives.

Independent research and research dissemination in undergraduate teaching

By laurenmccann Comment Closed

Dr. Ute Woelfel, Literature and Languages
u.wolfel@reading.ac.uk
Year of activity: 2016/17

Overview

In order to improve students’ engagement, support their abilities as independent learners, and increase their feeling of ownership for their academic work, elements of independent research and research dissemination through the creation of research posters were included in a Part 2 module.

Objectives

  • Boost independent learning.
  • Nurture research interests.
  • Increase feeling of ownership.
  • Develop employability skills.

Context

In 2016/17 I introduced a new Part Two module on German National Cinema (GM2CG: 20 credits/ 30 contact hours). The module is intended to give students a general overview of German cinema from the end of World War I to German unification and at the same time allow sustained independent work on themes of interest. In order to increase the engagement with the themes, the independent work is research-oriented demanding from students to reflect their own expectations and aims, their goals for the module and indeed the course, and develop their own interest and approach.

Implementation

The students were asked in the beginning to pick a period or topic from a list and prepare a presentation. The presentation was not part of the summative assessment but served as a foundation for further research. After the presentation, individual discussions with each student were used to decide which aspect of the theme/topic the student would like to pursue further. After each term, essay surgeries were offered in which students were given the opportunity to discuss the research done so far and decide a concrete research question for their essay (2,500 words/ 30%). The students were then asked to turn the findings of their essays into research posters for dissemination to non-specialist audiences (10%). In order to make sure that students also gain a general understanding of German cinema, a final exam (60%) is scheduled in the summer term.

Impact

The inclusion of independent research elements was very successful in that students did engage more than they normally do when given set topics and essay titles. The majority of students found secondary sources, even additional primary sources, and often identified research topics they would like to pursue in the future. Both the essay and the exam marks were above average. The poster challenged students to re-think their academic findings and present them in a new, visually organised, format for interested general audiences; as we used the posters to showcase the students’ work at the University’s Languages Festival, the Visit Days and a Reading Scholars outreach event, a sense of the importance of their work emerged as well as pride in what they had achieved grew. The students understood the relevance of the poster for the development of professional skills.

Reflections

The module worked well and highlighted most of all the potential our students have and can develop in the right learning environment as well as their willingness to work hard when they are committed. Their engagement with independent research signalled a wish to get active and explore options beyond the set class texts rather than being spoon-fed; there is a clear need for feeling involved, responsible and in charge of work. I was particularly surprised about how much effort students were prepared to put into the presentations despite the fact that they did not count towards the module mark; as they were used as foundation for assessment, students clearly understood their benefit.

The research elements made the module learning and teaching intensive as a good number of office hours and slots during the enhancement weeks were used for individual discussions of research and essay topics; as I want the students to put their research posters to good use as well, additional feedback slots were offered in which I discussed not just marks but ways of improving the posters; students showed great willingness to work even further on their posters just to see them exhibited, despite the fact that any further input would not change the mark.

Connecting with the Curriculum Framework: Using focus groups to diversify assessment (Part 1)

By laurenmccann Comment Closed

Dr Madeleine Davies, School of Literature and Languages

Overview

The Department of English Literature (DEL) is organising student focus groups as part of our TLDF-funded ‘Diversifying Assessments’ project led by Dr Chloe Houston and Dr Madeleine Davies. This initiative is in dialogue with Curriculum Framework emphases engaging students in Programme Development and involving them as stakeholders. This entry outlines the preparatory steps taken to set up our focus groups, the feedback from the first meeting, and our initial responses to it.

Objectives

  • To involve students in developing a more varied suite of assessment methods in DEL.
  • To hear student views on existing assessment patterns and methods.
  • To gather student responses to electronic methods of assessment (including learning journals, blogs, vlogs and wikis).

Context

We wanted to use Curriculum Framework emphases on Programme Review and Development to address assessment practices in DEL. We had pre-identified areas where our current systems might usefully be reviewed and we decided to use student focus groups to provide valuable qualitative data about our practices so that we could make sure that any changes were informed by student consultation.

Implementation

I attended a People Development session ‘Conducting Focus Groups’ to gather targeted knowledge about setting up focus groups and about analytical models of feedback evaluation. I also attended a CQSD event, ‘Effective Feedback: Ensuring Assessment and Feedback works for both Students and Staff Across a Programme’, to gain new ideas about feedback practice.

I applied for and won TLDF mini-project funding to support the Diversifying Assessments project. The TLDF funding enabled us to regard student focus groups as a year long consultative process, supporting a review of assessment models and feedback practices in DEL.

In Spring Term 2017, I emailed our undergraduate students and attracted 11 students for the first focus group meeting. We aim to include as diverse a range of participants as possible in the three planned focus group meetings in 2016-17. We also aim to draw contributors from all parts of the undergraduate programme.

To prepare the first focus group:

  • I led a DEL staff development session on the Diversifying Assessment project at the School of Literature and Languages’ assessment and feedback away day; this helped me to identify key questions and topics with colleagues.
  • I conducted a quantitative audit of our assessment patterns and I presented this material to the staff session to illustrate the nature of the issues we aim to address. This tabulated demonstration of the situation enabled colleagues to see that the need for assessment and feedback review was undeniable.

At the first focus group meeting, topics and questions were introduced by the two project leaders and our graduate intern, Michael Lyons, took minutes. We were careful not to approach the group with clear answers already in mind: we used visual aids to open conversation (see figures 1 and 2) and to provide the broad base of key debates. We also used open-ended questions to encourage detail and elaboration.

Group discussion revealed a range of issues and opinions that we would not have been able to anticipate had we not held the focus group:

  • Students said that a module’s assessment pattern was the key determinant in their selection of modules.
  • Some students reported that they seek to avoid exams where possible at Part Two.
  • Discussing why they avoid exams, students said that the material they learn for exams does not ‘stick’ in the same way as material prepared for assessed essays and learning journals so they feel that exams are less helpful in terms of learning. Some stated that they do not believe that exams offer a fair assessment of their work.
  • Students wholly supported the use of learning journals because they spread the workload and because they facilitate learning. One issue the students emphasised, however, was that material supporting learning journals had to be thorough and clear.
  • Presentations were not rated as highly as a learning or assessment tool, though a connection with employability was recognised.
  • Assessed essays were a popular method of assessment: students said they were proud of the work they produced for summative essays and that only ‘bunched deadlines’ caused them problems (see below). This response was particularly marked at Part Two.
  • Following further discussion it emerged that our students had fewer complaints about the assessment models we used, or about the amount of assessment in the programme, than they did about the assessment feedback. This is represented below:

To open conversation, students placed a note on the scale. The question was, ‘Do we assess too much, about right, not enough?’ (‘About right’ was the clear winner).

Students placed a note on the scale: the question was, ‘Do we give you too much feedback, about right, or too little?’ (The responses favoured the scale between ‘about right’ and ‘too little’.)

The results of this exercise, together with our subsequent conversation, helped us to understand the importance of feedback to the Diversifying Assessment project; however, subsequent to the focus group meeting, the DEL Exams Board received an excellent report from our External Examiners who stated that our feedback practices are ‘exemplary’. We will disseminate this information to our students who, with no experience of feedback practices other than at the University of Reading, may not realise that DEL’s feedback is regarded as an example of best practice by colleagues from other institutions. We are also considering issuing our students with updates when assessed marking is underway so that they know when to expect their marks, and to demonstrate to them that we are always meeting the 15-day turnaround. The external examiners’ feedback will not, however, prevent us from continuing to reflect on our feedback processes in an effort to enhance them further.

Following the focus group meeting, we decided to test the feedback we had gathered by sending a whole cohort online survey: for this survey, we changed the ‘feedback’question slightly to encourage a more detailed and nuanced response. The results, which confirmed the focus group findings, are represented below (with thanks to Michael Lyons for producing these graphics for the project):

A total of 95 DEL students took part in the survey. 87% said they valued the opportunity to be assessed with diverse methods.

Assessed essays were the most popular method of assessment, followed by the learning journal. However, only a small proportion of students have been assessed with a learning journal, meaning it is likely that a high percentage of those who have been assessed this way stated it to be their preferred method of assessment.

On a scale from 0-10 (with 0 being too little, 5 about right, and 10 too much), the students gave an average score of 5.1 for the level of assessment on their programmes with 5 being both the mode and the median scores.

34% found the level of detail covered most useful in feedback, 23% the feedback on writing style, 16% the clarity of the feedback, and 13% its promptness. 7% cited other issues (e.g. ‘sensitivity’) and 7% did not respond to this question.

66% said they always submit formative essays, 18% do so regularly, 8% half of the time, 4% sometimes, and 4% never do.

40% said they always attend essay supervisions (tutorials) for their formative essays, 14% do so regularly, 10% half of the time, 22% sometimes, and 14% never do.

Impact

The focus group conversation suggested that the area on which we need to focus in DEL, in terms of diversification of assessment models, is Part Two assessment provision because Part One and Part Three already have more diversified assessments. However, students articulated important concerns about the ‘bunching’ of deadlines across the programme; it may be that we need to consider the timing of essay deadlines as much as we need to consider the assessment models themselves. This is a conversation that will be carried forward into the new academic year.

Impact 1: Working with the programme requirement (two different types of assessment per module), we plan to move more modules away from the 2000 word assessed essay and exam model that 80% of our Part Two modules have been using. We are now working towards an assessment landscape where, in the 2017-18 academic session, only 50% of Part Two modules will use this assessment pattern. The others will be using a variety of assessment models potentially including learning journals and assessed essays: assessed presentations and assessed essays: vlogs and exams: wikis, presentations and assessed essays: blogs and 5000 word module reports.

Impact 2: We will be solving the ‘bunched’ deadlines problem by producing an assessments spread-sheet that will plot each assessment point on each module to allow us to retain an overview of students’ workflow and to spread deadlines more evenly.

Impact 3: The next phase of the project will focus on the type, quality and delivery of feedback. Prior to the Focus Group, we had not realised how crucial this issue is, though the External Examiners’ 2017 report for DEL suggests that communication may be the more crucial factor in this regard. Nevertheless, we will disseminate the results of the online survey to colleagues and encourage more detail and more advice on writing style in feedback.

Anticipated impact 4: We are expecting enhanced attainment as a result of these changes because the new assessment methods, and the more even spread of assessment points, will allow students to present work that more accurately reflects their ability. Further, enhanced feedback will provide students with the learning tools to improve the quality of their work.

Reflections

Initially, I had some reservations about whether student focus groups could give us the reliable data we needed to underpin assessment changes in DEL. However, the combination of quantitative data (via the statistical audit I undertook and the online survey) and qualitative data (gathered via the focus groups and again by the online survey) has produced a dependable foundation. In addition, ensuring the inclusion of a diverse range of students in a focus group, drawn from all levels of the degree and from as many communities as possible within the cohort, is essential for the credibility of the subsequent analysis of responses. Thorough reporting is also essential as is the need to listen to what is being said: we had not fully appreciated how important the ‘bunched deadlines’, ‘exams’, and ‘feedback’ issues were to our students. Focus groups cannot succeed unless those convening them respond proactively to feedback.

Follow up

There will be two further DEL student focus group meetings, one in the Autumn Term 2017 (to provide feedback on our plans and to encourage reflection in the area of feedback) and one in the Spring Term 2018 (for a final consultation prior to implementation of new assessment strategies). It is worth adding that, though we have not yet advertised the Autumn Term focus group meeting, 6 students have already emailed me requesting a place on it. There is clearly an appetite to become involved in our assessment review and student contribution to this process has already revealed its value in terms of teaching and learning development.

Game-based learning using social media

By laurenmccann Comment Closed

Dr Stanimira Milcheva, Henley Business School
stani.milcheva@henley.reading.ac.uk
Year of activity: 2015/16

Overview

We designed a simple game (called the REFinGame) which was aligned with the course material and launched it on Facebook. This approach, which could easily be applied to other discipline areas, was successfully used to enhance student learning and engagement with modules related to real estate finance.

Objectives

  • Allow students to develop transferable skills.
  • Allow students to apply course material in a real-world scenario.
  • Provide immediate and personalised feedback.
  • Improve interactions among students and between students and the lecturer.
  • Help make the module more interactive and enjoyable for students.

Context

Real Estate Finance and Debt Markets (REMF41), is a master’s module within Henley Business School. During the module students gain an awareness of the financing process for real estate from both a borrower’s and a lender’s point of view. The game was designed so that students could apply course material and learn to assess the risks associated with financing decisions.

Implementation

First, together with Professor Charles Ward, the REFinGame was designed before the beginning of the module. The design had to take into account the course material and make simplifying assumptions so that the game could be modelled to best represent reality. The idea was that students would play the game over the course of the module outside the classroom. The game is about making financing decisions. Students are split into property developers (investors) and lenders (banks). The developers make decisions on how many properties to develop depending on how much money they have and how much finding they need from the bank. Moreover, they decide on the type of the properties, the location and other characteristics. The banks decide how much funding to provide to each developer. The game is played on Facebook on a weekly basis as information is introduced on the Facebook Wall each week. Students advertise properties on the Wall, and a decision is made by the game coordinator on the transaction price of the buildings, based on the total supply by developers and the macroeconomic situation in that period. The main idea is that students learn to assess the risks associated with financing decisions as they can lose the virtual money they have available by making the wrong decisions. The game is won by the student who accumulates the greatest amount of money.

A closed Facebook group was created for the module, a logo was created for the game, and students were briefed how to play the game. The developers and lenders had to negotiate loan conditions using Facebook messages. They then advertised the properties they developed by putting pictures and information on the Wall. The purchase prices are then communicated to the developers by private message. Information about the economy and the markets us distributed as a post on the Wall. Students have to fill in a spreadsheet each week and send this to the game instructor. The game instructor then provides feedback to each student. At the end of the game, students shared their experience of the game by giving a presentation in which they presented their strategy and performance throughout the game and compared it to their peers. These presentations are assessed.

Impact

A significant relationship was found between the students who performed well in the game and their overall module mark. Less tangible outcomes are that the game can help students develop skills such as problem solving, creativity, and strategic behaviour, and also increases the interaction among students and between the students and the lecturer. In particular we found that playing a game on Facebook helped to better integrate students who might be more reticent in class discussions. The lecturer can develop a better idea of each student’s performance leading to students receiving tailored and regular feedback and being able to improve throughout the game. This is one of the main advantages that students identified, along with the playfulness of the game, and the ease with which the game is played on Facebook. The major issues students faced were the perception that course material is not directly applied in the game. This demonstrates that it is important to manage student expectations as well as have a structured approach when it comes to game design. Ultimately, our goal is to create guidelines for using self-designed simple games incorporating Facebook, and improve student learning.

Reflection

The novelty of our approach is that we did not design a video game or a digital game using special software, but instead designed a simple game to be played online using Facebook as a platform. We wanted to show how with limited resources and time an instructor can construct a game and engage students with it, as Facebook is free and widely used by students. We have observed that the main challenge in the design of the game is to ensure that it aligns with the course material and to manage student expectations. For this purpose the instructor should very clearly explain how the game can benefit the students and how they will be assessed. Also, it is crucial to communicate how the course material can be used within the game to make decisions. For this purpose, the game designer needs to make sure that the students see the direct link between the course material and the learning outcomes of the game.

Virtual teaching collections in Archaeology and Classics: turning artefacts into 3D models

By laurenmccann Comment Closed

Dr Robert Hosfield, School of Archaeology, Geography and Environmental Science

r.hosfield@reading.ac.uk

Year of activity: 2015/16

Sample image

Lykethos

Overview

The project tested different methods for producing and disseminating 3D models of existing artefacts in the teaching collections of Classics and Archaeology. 3D scanning was labour intensive and struggled to accurately represent some of the raw materials. By contrast photogrammetry was more cost and time effective and produced better quality results (see attached figure). Sketchfab was an effective, user-friendly platform for disseminating the models (https://sketchfab.com/uremuseum), and student feedback was positive.

Objectives

  1. Produce and evaluate 3D laser scans of 10 lithic artefacts and 5 ceramic artefacts from the teaching collections of Classics and Archaeology, with analysis of 3D model resolution, cost, and time requirements, and dissemination options;
  2. Document student evaluations of the new resources.

Context

Archaeology and Classics have wide ranging teaching collections of objects, both genuine and replica, from the human past (e.g. Greek and Roman ceramics). While students have access to this material in practical classes and seminars, out-of-class access is more difficult, due to (i) the intensive use of the teaching spaces holding the collections, and (ii) the fragility of selected specimens. The project explored methods that could enable students to engage with this material evidence through digital models.

Implementation

The project was primarily undertaken by four Reading students, both postgraduate and undergraduate: Rosie-May Howard (Bsc Archaeology, Part 2), Matthew Abel (BA Museum Studies & Archaeology, Part 1), Daniel O’Brien (BA Ancient History & Archaeology, Part 3), and James Lloyd (Classics, PGR). Supervision and support was provided by Prof. Amy Smith (Classics), Dr Rob Hosfield (Archaeology) and Dr Stuart Black (Archaeology). The four students undertook the following tasks:

(i) Testing the URE Museum’s NextengineTM HD 3D scanner and associated processing software ScanStudioTM to produce 3D laser scan models of selected artefacts (ceramics from the Ure Museum and stone tools from the Archaeology teaching collections).

(ii) Testing 3D printing of the laser scan models using the URE museum’s CubeProTM 3D printer.

(iii) Testing the digital representation of the same range of artefacts through photogrammetry, using memento by Autodesk.

(iv) Trialing the use of Sketchfab as a remote site for posting, storing and accessing the 3D models.

(v) Assessing student responses to the models through a Surveymonkey questionnaire.

Impact

(i) The 3D laser scan models provided volumetric data (unlike the photogrammetry models), but struggled with the regular shapes and repeating patterns which were characteristic of many of the ceramics. The laser scanning process was also time-intensive.

(ii) The laser scanner struggled to represent some of the stone artefacts, with the resulting models characterised by poorly defined edges and ‘holes’, due to the material properties of the flint raw material.

(iii) Photogrammetry was used successfully to create 3D models of ceramics from the Ure museum collection.

(iv) Sketchfab was a flexible interface for ‘touching up’ and annotating the models, and was more user-friendly than other options (e.g. scanstudio).

The quality of the 3D printing was mixed, leading to a decision during the project to focus on digital models that could be accessed on-line.

(v) Students responded positively to the virtual models, and would like to see more in future!

Sample survey questions and responses:

Q: What (if any) other objects/material types would you like to see as 3D models?

A: It would be interesting to see 3D models of smaller, more dainty objects as these can often be difficult to look at on such a small scale.

Q: Do you have any other comments?

A: This is a great project that should keep going! P.S. A scale will be helpful for accurately describing the objects. There’s a Part 2 Archaeology module called Artefacts in Archaeology and the scans could be used as an at-home resource by students.

Reflections

The project was successful in clearly highlighting the relative strengths and weaknesses of the 3D laser scan and photogrammetry methods for creating digital models of artefacts. In terms of cost and time it was clear that photogrammetry was a more effective method, while the experiments with 3D printing emphasised on-line hosts such as Sketchfab as the most effective way of disseminating the models.

More specifically, exploring the photogrammetry option highlighted the potential of the Agisoft PhotoScan software as an effective method for Museums or HEIs wishing to capture large collections for teaching and/or archiving purposes.

Student responses emphasised the importance of providing a wide range of models if these sorts of teaching resources are to be further developed.

Follow up

Archaeology has purchased copies of the Agisoft PhotoScan software and is currently looking to develop a photogrammetry-based digital database of its teaching collections.

At the Ure Museum 3D scans are being made available via Sketchfab and more thorough use of photogrammetry is being considered; virtual models of the vases scanned for CL1GH are being used in seminars this term.

Links

https://sketchfab.com/uremuseum

Using wikis for assessed group work in new history modules

By laurenmccann Comment Closed

Shirin Irvine – TEL Adviser, CQSD

Image of Shirin Irvine

Overview

For the academic year 2015/16, the Department of History offered a brand-new Part 1 programme as part of the History Project. This resulted in the development of three new core modules.

Dr Mara Oliva transformed common practice by using technology to carry out full electronic assessment for her module. This project included multiple aspects of digital pedagogy, using Blackboard to perform engaging assessment.  This was achieved through innovative and effective use of Blackboard Groups in combination with Blackboard Wikis and Turnitin Assignments, in addition to the Grade Centre for administering students’ marks.

What is a wiki?

A wiki is a collaborative tool that allows students to work as a group on one project and write shared content in the form of a website. They can create a series of web pages that can include images, web links and videos, collectively responding to a theme.

Dr Mara Oliva – Lecturer in Modern American History (20th century)

Image of Mara Olive

Mara explains how she used the wiki tool within Blackboard as a new tool for summative assessment.

The Culture Wiki

Journeys through History 2 aims to introduce students to major historical ideas, concepts, beliefs and knowledge systems, and to show how these are exemplified in material culture, with reference to artefacts, buildings, paintings and other works of art, literature and media.

We wanted the assessment tools we chose to reflect the cultural and visual elements of the module. Therefore we decided to use a group wiki of 2,000 words (50% of the module mark), which we called the Culture Wiki, and an individual 2,000-word essay on one of the historical concepts.

The Culture Wiki allowed students to create and contribute to several web pages of course-related material. They were expected to display their research, analytical and communication skills by building a website meant for public consumption. In small groups, students created their wikis based on a theme discussed during lectures. Lecturers provided themes in the module handbook and on Blackboard.

Our aims for using this form of assessment were to teach students the importance of teamwork and how to write in a concise and accessible way in order to develop an understanding of public history, which offers many employability opportunities to history graduates.

Impact – great results! 

Overall, the exercise was very successful! According to the feedback, both students and staff enjoyed working on the Culture Wiki. Students said it gave them a chance to look at history from a different angle and realise how many flexible and transferable skills they can gain through studying history.

We then decided to take this a step further and extend full electronic assessment to the individual assay, using Turnitin Assignments. This was received very enthusiastically by the students, who appreciated the immediacy and flexible, 24/7 access technology can offer.

The project, however, would have never taken off without the invaluable support of the TEL team, in particular Shirin Irvine, Lauren McCann and Maria Papaefthimiou. With their help we arranged training and guidance for the department staff on creating and assessing wikis, using Turnitin for e-assessment, and using the Grade Centre.

To support students, we provided a separate handbook with “how to build a wiki” guidelines, which was uploaded on Blackboard. I then dedicated part of the first lecture to introducing the exercise and answering the questions. Overall, students did not need much support and were very quick at learning – their questions were mainly content related.

We are very pleased with the outcome of the project, so we have decided to continue for the foreseeable future!

Embedding Employability Through Collaborative Curriculum Design

By edwardwhite Comment Closed

Embedding Employability Through Collaborative Curriculum Design

Name/School/ Email address

Amanda Millmore / School of Law / a.millmore@reading.ac.uk

Overview

This is a practical case study focusing upon the process of carrying out a collaborative partnership project with students to embed employability attributes into a trailblazing new module option for 2019/20 LW3CFS: Children, Families and the State.  This module is unique in that it is the first to embed employability attributes and skills within the module design. This project built upon previous work within the School of Law, which identified (by working with multiple stakeholders - students, staff and employers) 11 key employability attributes of a Reading Law graduate.

Not only do we now have a module with employability attributes built-in, but the student partners have gained a range of employability skills themselves by virtue of their involvement in the process. The student partners co-designed the module assessments, ran the student focus groups and presented the project at a number of national teaching and learning conferences this year. PLanT project funding was awarded and used to provide refreshments for focus groups and to enable students to travel to conferences to disseminate the project.

Objectives

I identified 3 key challenges that the project aimed to address:

  • Employability - how to equip students with the skills and attributes to succeed in employment.
  • Curriculum Design - how to embed those graduate employability attributes into a module.
  • Student Engagement and Collaboration - how to work effectively with students in partnership.

Context

In Law the professional pathways to careers are changing, with new routes opening up for vocational post-graduate and non-graduate training. These changes are raising questions for university law schools as to how much they should be focusing upon more practical and vocational skills.

My colleague Dr. Annika Newnham and I wanted to develop a new final year module, covering a discrete area of family law, closely allied to the kind of work that students may encounter in their early years of legal practice, with assessments mapped to legal employability skills. The brief was to design assessments for this new module which were mapped to legal employability skills and I looked to see how I could incorporate the student voice within the design process, deciding to engage them in the project as collaborative partners.

Implementation

Evaluation

Student views of their involvement in focus groups and as part of the core partnership group were sought throughout the project. All felt that this was a positive experience and welcomed the partnership and mapping of employability attributes.

Evaluation of the effectiveness of embedding employability into the module will be considered during the course of the running of the module. In addition to explicitly highlighting the attributes within the course materials and teaching, I intend to get the students to self-evaluate their awareness of and confidence in displaying the attributes at the start and again at the end of the module. I am also considering ways to utilise the assessed evaluative report to encourage reflection upon employability attributes. If the students will permit, I would also be interested to maintain contact with the students post-graduation to follow-up whether these skills have assisted them in their further study and careers.

Impact

Employability: The student partners have all developed employability skills from their involvement, in particular improved confidence, communication skills and leadership skills. These skills have been highlighted most through the opportunities that they have had to disseminate the project at national conferences.The wider student body has increased awareness of employability attributes.

Curriculum Design: The new module LW3CFS Children, Families and the State has student-designed assessments with employability attributes clearly mapped to them. Students involved have gained a greater understanding of the process of module design. The students acknowledged that this was a way for their opinions to be listened to, and for them to influence their own university experience, “University can be a very impersonal experience - it is always good to feel that your voice is being heard and that you can make an active impact on uni life and module development” (focus group participant). The module is oversubscribed in 2019/20 and is operating a waiting list. The high level of student interest (approximately 20% of the cohort have selected the module, which is significant given the rather niche subject area) is indicative of the support by students for the nature and timing of the assessments and an implicit endorsement of the staff-student partnership process.

Student Engagement & Collaboration: Students feel that they have been listened to, and been treated as true equitable partners in the process which embodies the University of Reading’s “Principles of Partnership” (2019). This has created greater feelings of community and power-sharing within the School of Law. The equitable nature of the power-sharing between staff and students was fundamental to the success of the project. This experience has been transformative for me as an academic, seeing how positively these students relished the challenge of collaboration, and became true partners in co-designing assessments. It has inspired me to look to other areas of my teaching practice to consider how I can partner with students to improve the student experience and student support in addition to classic teaching and learning activities.Students are interested in extending this trailblazing process to other modules, and colleagues and I are looking at expanding it to programme level.

Student Feedback: The following quotes are reflections from the student partners on the project:

"With all the discussions, I gained knowledge about the employability skills (communication, team work, problem solving, planning and organising, self-management, learning, research and analysis and the list goes on) and will take active actions to try to improve those skills in the future. I think I gained a lot of experience in involving in this project that I can put into practice into future projects or career as well."

"I am really looking forward and excited to learn about this module that I helped create. I think the School should definitely use this approach more often on other modules as a lot of the time when students are not satisfied/happy about how a module (or lecturer) we do not have much chance to voice out our opinions and make changes, so it is a good way to avoid that situation fundamentally. As students are likely to go into law practice after graduating, it is important to not only have essay or written examinations (that do not reflect real life law practice) as assessments. It’s really different to be good in examinations and to be good in practice."

Reflections

When I presented this project at the Advance HE conference in July 2019 I emphasised my 4 step plan for successful staff-student partnerships:

The partnership can relate to a discrete area of a project (in our case this was in relation to assessment design), and this fits well with Bovill’s (2017) ladder of participation. Once the boundaries of the project are clear, then it is vital to take a step back and relinquish control.

By keeping the student-staff partnership limited to a discrete area of module design (assessments) the boundaries were clear, and students could be given greater control. The key message is that equality of arms is vital, all viewpoints need to be welcomed and considered with no obvious staff-student hierarchy.

The limitations of the project were that it was focusing upon the modular level, rather than anything broader, so its impact is limited to that module, although the goodwill that it has generated amongst our students extends far beyond this single module.

A staff-student partnership needs to be approached with an equality of arms, so that all viewpoints are welcomed and considered, with no obvious hierarchy. As my student partner when presenting at the Advance HE conference said “For me personally as a student, you’re very much stuck in this kind of limbo where you’re not quite respected as an adult, but you’re not a child either...I’m an adult but not as respected as I would like to be in a professional environment. I wasn’t treated like that, I was treated as a complete equal and had the chance to run with my ideas, which was really important to me.”

Follow up

The module is due to run for the first time in 2019/20 for Final Year students in the School of Law.

My current plans for follow-up relate to the following areas:

  1. Further evaluation of the effectiveness of embedding employability attributes into a module (see evaluation section above).
  2. Consideration of better ways to highlight the employability attributes, for example by badging them (opening up possibilities for inter-disciplinary collaborations with creative colleagues and students.
  3. The success of this staff-student partnership has highlighted how this process could be scaled up to programme level within the School of Law. This is particularly in the light of reviews of the LLB programme within the context of the University of Reading’s Curriculum Framework review process and with an eye to the forthcoming changes to the professional vocational training at postgraduate level for lawyers. One of the challenges will be how we can widen and diversify the range of students in future curriculum design partnerships.

TEF

TQ1-5, SO1-3.

Links and references

ADVANCE HE 2016. Framework for embedding employability in higher education. Available from: https://www.heacademy.ac.uk/knowledge-hub/framework-embedding-employability-higher-education.

ADVANCE HE 2016. Framework for student engagement through partnership. Available from: https://www.heacademy.ac.uk/sites/default/files/downloads/student-enagagement-through-partnership-new.pdf.

BOVILL, C. 2017. A Framework to Explore Roles Within Student-Staff Partnerships in Higher Education: Which Students Are Partners, When, and in What Ways?  International Journal for Students As Partners,  1 (1). https://doi.org/10.15173/ijsap.v1i1.3062, 1.

HEALEY, M., FLINT, A & HARRINGTON, K. 2014. Students as Partners in Learning & Teaching in Higher Education [Online]. York: Higher Education Academy. [Viewed on 1 July 2019] Available from: https://www.heacademy.ac.uk/knowledge-hub/engagement-through-partnership-students-partners-learning-and-teaching-higher.

Refreshing Professional Practice

By edwardwhite Comment Closed

Refreshing our professional practice module

Name/School/ Email address

James Lloyd / ACD (Typography) /  j.c.lloyd@reading.ac.uk

Overview 

In 2017/18 we reviewed and revamped our flagship professional practice scheme known as ‘real jobs’. The impacts include: a significant increase in the number of students regularly attending feedback sessions and engaging with the process; a large number of highly presentable concluding reflective reports from students; a reduced impact on staff resource due to better management of information and processes; and greater level of clarity among the student body about the existence and benefits of the scheme.

Objectives

  • Boost student engagement
  • Embed contemporary workflows
  • Streamline assessment and feedback
  • Centralise information and information management
  • Initiate a briefing session for all students, for the first time
  • Generate more competent (and compelling) outcomes. Our new blog at typography.network/real-jobs collects students reflective reports on these projects
  • Increase student exposure to industry professionals
  • Give feedback as jobs are completed, rather than right at the end of the course
  • Provide students with personal, engaging stories to tell on job applications and at interviews

Context

This work is part of TY3PRP on our BA Graphic Communication course. The genesis of these changes came from a sense that the current system was stretched due to increasing student numbers and lack of staff time. We needed to centralise and standardise more procedures in order to free up most staff to function better in their primary role (as design supervisors) and leave issues relating to project management, industry practice and print production to new staff members with a more dedicated set of roles (specifically in professional print production and professional design management). We also sought to address the fact that Real Jobs had never been fully integrated into the modular system on which the University now runs, making it an outlier in many areas (including assessment, timetabling and briefing) and thus causing confusion to students.

Implementation

Surfacing the issues

The project was initiated following detailed discussion with Rob Banham, our DDTL, based on his experience of running the Real Jobs scheme for around a decade. We also took advantage of the training and techniques offered on the University’s Academic Practice Programme (which Geoff Wyeth and I participated in for 18 months) to assess, flesh out and test Rob’s analysis of the issues. It became clear the scheme was characterised by lack of clarity (with no real assessment criteria or workflow) and lack of student engagement (with the keenest students always doing well, but the majority shying away from the scheme). It also felt excessively manual in its admin – reliant on the generosity of staff time, rather than robust processes.

Planning for professionalisation

Over Summer 2017 I planned a new process, taking in feedback and concerns from a wide range of staff and students, trying to address a wide variety of issues and encode solutions into:

  • A Filemaker database of jobs, clients and students (for staff admin use only)
  • A Trello online project management board (used by supervisors and students). The board acts as a contract, a step-by-step process and a live project management tool, mirroring many aspects of life as a professional designer
  • A blackboard organisation – so the scheme has a VLE for the first time
  • A new annual briefing session for all Part 1 students, late in the year
  • A revised format for weekly Real Job meetings
  • A new rubric, mapped to new assessment criteria

Launch

We launched formally in Autumn term 2017, with most of the tools in-place.

In order to get all students up to speed, we ran the induction briefing session for all three year groups. The induction was crucial to the success of these changes. By bringing in staff, graduates and potential clients, we carry all students through a model for the new process over a two-hour session. They experience a dry run of the whole thing, and hear reflections from students and clients who’ve already been through it. The goal is to bring active awareness to the scheme and its process, so that when they attack projects for real, the barriers feel reduced.

Impact

The outcome is an entirely new process, much more tightly focused on student understanding, user needs and a more engaging and defined set of tasks – while still allowing students to explore projects in their own way.

Attendance at Real job meetings has gone from an average of around 9 students per week to something more like 30 or 40. A increased calibre of discussion has also been noted by staff.

Final reports have been transformed into more professional, thoughtful, meaningful and marketable blog posts.

Assessment is simpler. More time consuming, but more thorough, and clearer.

Higher throughput of jobs.

Students who DON’T wish to pursue a career as a professional designer now have a parallel route, through Experiential Learning Assignments that let them write about design rather than practice it.

Reflections

The changes work because there has been a top-to-tail review with solutions carefully targeted across a range of goals. Most of these solutions are working as expected, though there is room for improvement

As a Department we still lack the resources to truly ensure that all students and all jobs stay on track. We have more visibility and better insights, but process steps can be skipped without immediate remedy.

The assessment process is more involved than planned, and is not yet happening as jobs are completed, but still at the end of each year.

It’s hard to measure the impact on employability.

Only a few students take advantage of the offer of a thorough pre-press check on their work.

Follow Up 

We have continued to refine assessment criteria and rubric style, in an effort to simplify things for staff and students.

ELAs (which let students get credit for non-design aspects of their studies) are being rolled out slowly.

A module review is needed to assess whether students are entirely satisfied with the way the scheme runs. Anecdotal evidence suggest some students still find the prospect of these projects daunting, and they find ways to avoid engagement.

TEF

LE2, SO1, LE3

Links

http://typography.network/real-jobs-scheme/

http://typography.network/about-real-jobs/

https://www.bb.reading.ac.uk/webapps/blackboard/execute/announcement?method=search&context=course&course_id=_130257_1&handle=cp_announcements&mode=cpview (login required)

 

Increasing Engagement Through Use of Collections

By edwardwhite Comment Closed

Using collections in teaching & learning

Name/School/Email address
Rob Banham / ACD / r.e.banham@reading.ac.uk

Overview 
The work undertaken was to engage UG students in design history and printing history through its lettering, printing, and graphic design collections. The impact was improved attendance and greater engagement of students with history modules, increased use of collections by staff in other modules, development of new modules, and UROPs.

Objectives
• Increase attendance and engagement in the UG module history of graphic communication
• Improve students understanding by revisiting topics covered in lectures
• Encourage students to develop skills in analysis and debate
• To enable students to learn through task-based learning activities, rather than traditional seminars

Context
TGC’s approach to teaching design has always involved history, theory, and practice. Over time, changing practices and new technologies in design and printing meant that the history of the subject has become less connected with current design practice. UG students were therefore less inclined to positively engage with modules with a historical focus. The activity was undertaken to make history teaching in TY1HG (now TY1HG1) more dynamic and engaging, using interactive sessions with TGC’s collections, and task-based learning, in place of traditional seminars.

Implementation
• New module outcomes and seminar content for TY1HG1 (History of Graphic Communication 1), with task-based learning, relating to material from TGCs collection
• Additions to TGC’s collections

Impact
The revised module content achieved its immediate aims of improving student attendance and engagement in seminars, with students frequently commenting positively in module evaluation about the inclusion of collections-based seminars. Students were also more engaged with associated lectures, and with the coursework submissions and examination in the TY1HG1 module. There were also further, unexpected outcomes. The popularity of the revised module led to student requests for additional opportunities to study the history of the subject and new modules were introduced in Part 1 (TY1PRI: History of Printing and Printmaking, TY1HG2: History of Graphic Communication 2) and Part 3 (TY3ES: Ephemera Studies). There was also an uplift in the number of students using collections in general, and TGC’s collections in particular, as the basis for dissertations in Part 3. Staff also began to employ collections-based sessions in practical design modules across all three years.

Reflections
The activity was successful for a number of reasons. Students have the confidence to participate in seminars because the task-based learning activities mean that they can discuss questions in pairs or groups before sharing their answers. This is particularly appreciated by students with learning styles not suited to traditional seminars, or from WP backgrounds who tend to be intimidated by them. Students also find the hands-on nature of these seminars much more engaging than a straightforward discussion of sources and, perhaps most importantly, find the seminars a useful way of consolidating their learning from associated lectures. The activity had wider impact because the good practice and positive student feedback were shared through staff and student evaluation and review of teaching. The activity could have been better implemented by involving students in developing the revised module description, by de-colonizing the module curriculum (which we will be working on over the next 12 months), and providing students with more training regarding how to handle collections materials.

Follow Up
The TY1HG1 module has continued to evolve, with the introduction of new seminar topics, classes looking at University Special Collections and (as outlined above) new modules and increase use of collections in practical project teaching.

TEF
LE1, TQ1