This post has been uploaded to the T&L Exchange, and can now be found at:
Improved Neural Network assessment by staged laboratory practicals
Professor Richard Mitchell, School of Systems Engineering
r.j.mitchell@reading.ac.uk
Overview
Adjustments were made to teaching, assessment, and feedback in a Part Two module within the School of Systems Engineering, Neural Networks (SE2NN11), successfully using three-staged laboratory practicals in order to encourage students to use neural networks on a ‘real world’ application. Making these changes saw an increase in the number of students successfully producing a neural network.
Objectives
- Increase the number of students successfully producing a neural network.
- Provide greater and prompter feedback to students.
Context
The major assessment for SE2NN11 requires students to write a program to implement a particular neural network and to then use that network on a ‘real world’ application. The students demonstrated their network by the end of the autumn term, where verbal feedback was given, and they then applied it to the real world problem of their choice in the next term. Previously, students had difficulty with the first stage, and so fewer moved on to the (more interesting) second stage, with only around 75% of students submitting a report.
Implementation
During the pilot year (2009), the tasks associated with writing the neural network were carefully divided into three, and three associated 90 minute lab sessions were organised, two weeks apart, for the work. The lecturer plus two laboratory demonstrators were available to provide help to the students at these sessions.
For each session, a Microsoft Word template file was provided, and the students copied and pasted relevant program output or small parts of the program (functions) into the appropriate parts of the template. A simple marking scheme was associated with each part, worth 30 marks: typically students could get 0, 1 or 2 for a piece of code plus 0 or 1 for comments; or a student could get 0 or 1 depending on whether the program output was correct. There was also space for comments to be written.
These files were then submitted to the lecturer who circled the relevant mark for each part and added relevant comments. Detailed feedback was thereby generated very easily and very quickly. The aim was to give feedback within a week of the session, allowing the student a further week to make any necessary corrections on one part before starting the next part of the program. In fact, the first week’s work was marked within two days.
Impact
Each year since this scheme has been introduced, around 95% of students have been able to produce a neural network, a significant increase in submissions.
Reflections
The impression obtained in the pilot was that a greater proportion of students had a working neural network compared with previous years, suggesting a great success of this scenario. As such the scenario has been used each year since with some changes to the templates (and to the program to help reduce plagiarism between years).
One problem is that the structure of the program is so tightly defined that there is little scope for variation in code – hence copying is difficult to detect. This is partly addressed by requiring the student to comment their code and to discuss the object-oriented aspects of their program in the final report. In addition students were expected to do experiments in their own time to investigate the effects of changing specific parameters in the program. The instructions for the final report were made clearer to try to ensure this happens.
One disadvantage of the approach is that it discourages independent thought more than is ideal. The much increased submission rate, however, is encouraging.
The important aspects of this scheme are the division of the project into suitable, easily marked sub-tasks, the extra support provided in the development of the program and the inherent feedback between sessions.
Follow up
Following the pilot year, it was realised that some functions were more complex than others, so the marking scheme was changed so that an appropriate number of marks for the code and comments was available for each function.
From 2015, with the move to online submission, students upload their document to Blackboard Learn, where the work is readily marked. Rather than circling the marks, the ‘insert text’ option is used to allow the marks to be entered. Comments on errors or suggestions for improvements are also easily added in an appropriate context.
The second part of the assessment requires the student to apply their neural network to a ‘real world’ problem of their choice, to see if the network can learn that problem. In effect the students are researching whether a neural network is appropriate. Given that, rather than asking the students to write a report on their work, they are now asked to present their research in the form of a four page conference paper. This tests them with a new skill, complementing the report writing skills they use elsewhere. This innovation has also proved successful.
The online studio: using Technology Enhanced Learning to support independent learning
Dr Jeanne-Louise Moys, School of Arts and Communication Design
j.l.moys@reading.ac.uk
Year of activity: 2014–15
Overview
The project explored what kinds of online resources BA Graphic Communication students engage with and need and, through an iterative design process (combining prototyping and user testing), developed a new online resource interface to support learning. As a result, staff and students within the Department of Typography & Graphic Communication were able to gain a better understanding of students’ preferences and needs, with regards online resources.
Objectives
- Identify what kind of resources students prefer and what kind of interaction they should support.
- Develop a prototype that responds to these needs.
- Test the prototype and refine it further.
Context
Typography students engage in a great deal of independent learning. This includes a significant amount of online research that students conduct in relation to their studies. The Department wanted to find ways to support this and extend it through Technology Enhanced Learning.
Implementation
First, tutors and students took part in collaborative brainstorming workshops, in which the main issues which the project sought to address were discussed. These workshops fed into the creation of a short questionnaire, which the student project leaders used to elicit feedback from their peers about their preferences and common methods of working and communication both among themselves and with staff. The findings of the questionnaire, which was completed by 25 students, reinforced the need for a new online resource interface and allowed the creation of a focused design brief to guide the development of the prototype.
As a result of the questionnaire feedback, the prototype prioritised ease of navigation, as respondents had indicated that they wanted the resource to allow them to browse well-structured categories, and also to make searches for specific resources.
Following the development of the prototype, a series of semi-structured interviews with staff and students was conducted, to gain feedback on the resource, with this feedback being used to refine the prototype. For example, feedback indicated that students wanted to be able to search for resources thematically rather than necessarily be limited to a structure that reflected the structure of particular modules taught within the Department.
Impact
The project has enabled staff within the Department of Typography & Graphic Communication to improve their understanding of students’ preferences and needs and the ways in which they engage with online learning resources. This forms a useful foundation that can inform the ways in which we continue to support teaching and learning. The prototype that has been developed is an excellent starting point, and has received positive feedback from both students and staff.
Reflections
Given the positive feedback of students and staff, there is a strong desire within the Department to continue to explore ways of implementing the resource to benefit students for the long term. Two Study Abroad students continued to work on developing and testing materials for the proposed resource over the summer. The project is on-going, although resourcing it sufficiently continues to be a challenge.
One of the main difficulties faced during the project was timing. As a result, the opportunity to use the full budget was missed, and a number of the resources that were originally envisaged could not be utilised within the time available.
The success of the project is fully attributable to the efforts of the three students who worked on it. As the project was student-led, this allowed the project to respond directly to the challenges that students face. The student team benefited from their involvement in the project as they gained experience of conducting end-user research, and using this research to iteratively design and develop a prototype, as well as developing skills such as the ability to work effectively within a team, and written and verbal communication within a number of different contexts.
Links
Assessing the impact of internationalisation on students, from both a UK student perspective and a NUIST student perspective
Dr Philippa Cranwell, School of Chemistry, Food and Pharmacy
p.b.cranwell@reading.ac.uk
Year(s) of activity: 2014/15
Overview
The project assessed the impact of the intake of a cohort of 16 3+1 BSc Applied Chemistry students on the existing undergraduate students on programmes within the Department of Chemistry (approximately 72 students, on both BSc and MChem programmes), and determined any preconceptions each cohort may have had about each other or the course.
Objectives
- Determine the impact of intake of students from Nanjing University of Information Science and Technology (NUIST) on existing Part Three Chemistry students.
- Discover what preconceptions current Chemistry students held.
- Determine what aspects of their year at the University of Reading students from NUIST found the most challenging, and what support could be offered to future students.
Context
The 3+1 BSc Applied Chemistry is a dual award degree delivered in partnership by the University of Reading and the NUIST. Students currently study for three years at NUIST in the People’s Republic of China before transferring to the University of Reading to complete their final year, with successful students being awarded a Bachelor of Science from each institution. This study was undertaken because integration of two cohorts on this scale had not been undertaken before in Chemistry at the University of Reading. There was a desire to ensure that existing students were not adversely affected by the intake of Chinese students because they would have to share resources, such as lectures, workshops and tutorials, and also to ensure the Chinese students felt they were adequately supported whilst in the UK.
Implementation
The findings were derived from focus groups held with Part Two students (30 students), Part Three domestic students (30 students) and the Part Three NUIST students (12 students). During the focus groups, the students were posed questions about different aspects of the year and wrote responses on giant sticky notes. The questions were designed such that they were open and allowed students to give as much information as they wanted. In the case of the NUIST students the focus groups were not very successful due to a reluctance to speak out. In addition, therefore, anonymous questionnaires that the students could fill in and return were distributed. During the focus group session, lunch was provided to thank the students for their time.
Impact
The study achieved its objectives, although not in the manner originally perceived. It had not been anticipated that the NUIST students would be so reluctant to speak out. It was quickly realised, however, that the best way to obtain meaningful data from the NUIST cohort was to offer anonymised questionnaires. This approach will be used in the future. Additionally, the study was useful in that it showed that there was one overriding theme for good integration; the importance of language skills. Although it was known that the students all fulfilled the University’s requirement for English language proficiency, it had not been anticipated how difficult it would be for them in a lecture situation.
Reflections
This project was successful in that it managed to gather the necessary information. If the project were to be repeated again, there would be more awareness of the fact that the Chinese students were less forthcoming with their views and anonymous questionnaires would have been used from the beginning. It might also have been useful to pose the questions in Mandarin, therefore avoiding any confusion or misunderstandings. With regards to the UK students, the session was well-received and students were happy to have the opportunity to give their opinions so no changes to this are necessary.
Outcomes from the activity have led to a reassessment of the way the initial three years of the programme are taught in the People’s Republic of China, and an emphasis on the importance of a good grasp of the English language; both in academic and in social situations. The Department of Chemistry is working towards providing:
- Additional exam-style questions for the students to practice while they are in the UK.
- Input into exam questions in the People’s Republic of China so students are better prepared for what to expect when in the UK.
- A greater emphasis on the technical language required for the study of Chemistry.
Developing a tailored study support package for postgraduate students
Dr Amanda Branson, Psychology and Clinical Language Sciences
a.branson@reading.ac.uk
Year of activity: 2014-15
Overview
This project focused on identifying challenges faced by mature students studying alongside full time employment, and developing ameliorative practices and resources. Structured interviews conducted with 40 current students and alumni revealed barriers including weak IT and study skills and role conflict. Modifications have been made to interview and induction processes, and online resources are being developed to support students; work continues in this regard.
Objectives
- To gain an understanding of the barriers experienced by postgraduate students.
- To design and deliver interventions aimed at supporting students to achieve their academic potential, and to foster a supportive, inclusive culture that promotes student well-being.
Context
The Charlie Waller Institute (CWI) in the School of Psyhology and Clinical Language Sciences delivers postgraduate training in Evidence Based Psychological Therapies. Trainees typically attend training at the University of Reading alongside full time employment as therapists in NHS Improving Access to Psychological Therapies (IAPT) services. Over the course of five years Amanda Branson observed that these students face considerable barriers to the successful completion of their training, with some reporting a negative effect of training on their well-being.
Implementation
Current students and alumni were contacted by email and invited to be interviewed. An overwhelming response was received, resulting in 40 interviews being conducted between July and September in 2014. Students were asked about their motivations to return to study, the challenges they faced (clinically and academically), the support they received and that would have been beneficial, their experiences of being a student at the University of Reading, and their emotional well-being. Those who had completed training were asked to reflect on their experiences and impart advice to future students and indicate improvements that could be made. Interviews were transcribed and explored to identify repeating themes. These themes were discussed with programme directors and clinical tutors, resulting in some immediate changes being implemented across the programmes, from interview to induction and across the training period. The process of developing on-line resources for students continues.
Impact
The first goal of this case study was to gain an understanding of the barriers faced by postgraduate students, and this goal was surpassed: while the initial target was to conduct 10 interviews, 40 interviews were conducted, giving both breadth and depth of knowledge and understanding of the experiences of a very heterogeneous population of students. The impacts were several fold:
- Staff obtained a greater understanding of the challenges faced by these students, enabling them to make some small, yet meaningful changes to the delivery of training, particularly in regard to student induction, and delivery of study skills/study support.
- Students have been given additional tutorials in accessing e-systems (such as Blackboard Learn and the library), and using Microsoft Word effectively for academic writing. Feedback from these training sessions is good. Examples of ‘exemplary’ work have been uploaded to Blackboard Learn to give students some insight into what makes a good piece of academic work.
Reflections
This case study was made successful by the level of support received by students, which exceeded expectation. Students appreciated that time and resources were being given to improving the experience of their peers. Course tutors were very engaged with the outcomes of the interviews, and were keen to implement changes immediately (particularly the induction process).
To date, fewer study support resources have been developed than were planned. Those focusing on clinical skill have been delayed due to confidentiality issues regarding patient data, but CWI are currently exploring alternative approaches. Over the time within which the project was being planned, the University study advisors developed some excellent video tutorials, including referencing and critical thinking. These topics have not been reproduced and links to relevant tutorials have been added to Blackboard Learn. The technology required to develop resources (such as Camtasia screencasting software) has been procured; therefore there are no limits to the on-going benefit of the project.
Follow up
Work continues on the development of study support resources. The ultimate goal is to create an e-library of resources to support academic and hopefully clinical work, though this may take a greater time than was initially proposed. The programme administrative team are due to receive training on the use of Camtasia, which will enable them to develop tutorials relating the coursework submission process, for example. Administrators have also been trained in the use of Endnote, so that a reference-bank can be developed.
Core Issues in English Language Teaching: building student autonomy, technology-enhanced skills and employability
Dr Clare Wright, School of Literature and Languages
c.e.m.wright@reading.ac.uk
Year(s) of activity: 2014-15
Overview
This project aimed to build student expertise in managing task-based approaches to learning, foster active engagement in seminars including international students, and support students’ development of Technology Enhanced Learning (TEL) skills, through student-led revisions of a popular undergraduate module, with the Employability and Professional Track staff at School level, the Generating Resources and Access to Screencapture Software (GRASS) team and the central TEL team.
Objectives
- Revise the module delivery to enhance student autonomy and academic development.
- Improve preparation for and engagement with team- or task-based work in seminars
- Improve the use of TEL in class and the related skills development of both staff and students.
- Build up student employability in teaching-related expertise by leading a team- or task-based teaching approach in seminars.
Context
Core Issues in English Language Teaching for Part Two and Three students aims to build awareness of professional language teaching practices in international settings, and has approximately 35 students. Students enrolled on the module learn about different language teaching approaches, including task-based learning, team-based teaching, and TEL. This project responded to student demand for clearer training to manage task-based approaches to learning, greater engagement in seminars including international students, and greater skill-development of TEL.
Implementation
Two Part Three students and two international students conducted this project, alongside the module leader, with the Employability and Professional Track staff at School level, the GRASS team and the central TEL team.
The project team worked through revisions to the existing module guide, held a student focus group to discuss possible changes with students across the university, attended tailored training sessions with GRASS and TEL team members, implemented their training by using various TEL products (such as Camtasia and Powtoon), prepared presentations for a University of Reading TEL Showcase organised by the Centre for Quality Support and Development (CQSD) and Reading University Students’ Union (RUSU), and wrote a final blog entry on the project. The project leader, Clare Wright, was awarded an inaugural national Jisc Change Leader Award for a portfolio based on this project.
Impact
The student project team members could show full satisfaction when reflecting on their progress in understanding more about learning processes, and in gaining greater employability as a result of developing TEL-related skills, delivering presentations to the Pro-Vice-Chancellor (Teaching and Learning) and a wider University audience, and writing up the project blog. The project revealed that students were generally happy with the way they were being taught, but that including more TEL and autonomous learning could seem a challenge, especially for Part Two students. The suitability of the project for the Jisc Change Leader Award was an unexpected outcome, and feedback from the project submission could be used to benefit University Teaching and Learning (T&L) stakeholders, for example at a T&L Showcase event.
Reflections
The positive engagement with the aims of the project, and the close interaction between the students and the project leader was a key element of the project’s success. Attempts to roll out discussions to a wider student base, through focus groups, were less successful, suggesting either that students felt they were too busy to attend such events, despite the incentive of a free lunch, or that they were already happy with the way they were being taught.
Follow up
The Core Issues in English Language Teaching module is being revamped for Part Two students for the 2016-17 academic year, and will take the findings of this project into account.
Links
Engage in Teaching and Learning blog post: The PLanT Project and ‘Core Issues in English Language Teaching’ by Jess Fullam, Emily King, Daria Pominova and Megumi Kuranaka
Campus Wildlife Champions
Dr Graham J. Holloway, School of Biological Sciences
g.j.holloway@reading.ac.uk
Year(s) of activity: 2014-15
Overview
Students (mostly BSc Ecology and Wildlife Conservation students) carried out species identification activities in The Centre for Wildlife Assessment and Conservation on Wednesday afternoons. Students were encouraged to ‘adopt’ and focus on a taxonomic group (or community) to acquire deep learning. As a result of this and other activities, we have increased the campus species list to nearly 1700 species, an impressive statistic that we use during Information Days.
Objectives
- To develop a sense of community amongst the BSc Ecology and Wildlife Conservation students.
- To increase appreciation of the importance of extra-curricular learning.
- To develop skills valued by many potential employers of these students.
Context
There is often a mismatch between what students value from their time at universities and what employers value. Employers in the conservation sector frequently value extracurricular skills in addition to modular learning. A skill that is acutely lacking in graduates is an ability to identify species and, therefore, to carry out surveys under field conditions. The Campus Wildlife Champions project offered a way for undergraduate students to develop their CVs.
Implementation
The Ecology and Wildlife Conservation programme adviser meets with the students on a regular basis. During these meetings the importance of a strong CV that provides evidence of understanding, interest and learning, vital for career development in the conservation sector, is impressed on the students. Funding provided by the Teaching and Learning Development Fund was used to provide resources to facilitate engagement of students with species identification. Every Wednesday afternoon the project leader, along with Mr. Chris Foster (a Teaching Associate of the School of Biological Sciences), worked with the students in labs to help them to get to grips with specialised identification keys.
Impact
One of the main objectives was to encourage students to appreciate that on completing their degree programme their CV becomes the most important element in their armoury. Students need to take charge of their own learning to develop their skills so that their CVs showcase who they are and what they can deliver to a potential employer. Relying entirely on learning through modular teaching is unlikely to make them special. For this reason I was not happy to make this activity compulsory; students had to choose for themselves whether they were interested and how far they would like to take the activity. Seventeen students engaged with the project but this number dropped off as the academic year progressed, in particular as the examination period approached. Several students though remain active and it is hoped that they will continue a second year.
Reflections
This type of activity had not been attempted before so we were unsure how it would unfold. To have several students still interested in the activity is great and I will be encouraging these students to resume engagement during the coming academic year and to interact with the new cohort of students starting in September 2015. Continuity was a major aspiration so to have a real prospect that Part Two students could become mentors for Part One students is excellent. It is not obvious how we could have done things differently or better while operating within the constraints.
Links
The Campus champion project has appeared as a university news item:
http://www.reading.ac.uk/news-and-events/releases/PR618137.aspx
Data from the project are contributed to the Whiteknights Biodiversity website:
http://blogs.reading.ac.uk/whiteknightsbiodiversity/