Improved Neural Network assessment by staged laboratory practicals

By edwardwhite Comment Closed

Professor Richard Mitchell, School of Systems Engineering
r.j.mitchell@reading.ac.uk

Overview

6470Adjustments were made to teaching, assessment, and feedback in a Part Two module within the School of Systems Engineering, Neural Networks (SE2NN11), successfully using three-staged laboratory practicals in order to encourage students to use neural networks on a ‘real world’ application. Making these changes saw an increase in the number of students successfully producing a neural network.

Objectives

  • Increase the number of students successfully producing a neural network.
  • Provide greater and prompter feedback to students.

Context

The major assessment for SE2NN11 requires students to write a program to implement a particular neural network and to then use that network on a ‘real world’ application. The students demonstrated their network by the end of the autumn term, where verbal feedback was given, and they then applied it to the real world problem of their choice in the next term. Previously, students had difficulty with the first stage, and so fewer moved on to the (more interesting) second stage, with only around 75% of students submitting a report.

Implementation

During the pilot year (2009), the tasks associated with writing the neural network were carefully divided into three, and three associated 90 minute lab sessions were organised, two weeks apart, for the work. The lecturer plus two laboratory demonstrators were available to provide help to the students at these sessions.

For each session, a Microsoft Word template file was provided, and the students copied and pasted relevant program output or small parts of the program (functions) into the appropriate parts of the template. A simple marking scheme was associated with each part, worth 30 marks: typically students could get 0, 1 or 2 for a piece of code plus 0 or 1 for comments; or a student could get 0 or 1 depending on whether the program output was correct. There was also space for comments to be written.

These files were then submitted to the lecturer who circled the relevant mark for each part and added relevant comments. Detailed feedback was thereby generated very easily and very quickly. The aim was to give feedback within a week of the session, allowing the student a further week to make any necessary corrections on one part before starting the next part of the program. In fact, the first week’s work was marked within two days.

Impact

Each year since this scheme has been introduced, around 95% of students have been able to produce a neural network, a significant increase in submissions.

Reflections

The impression obtained in the pilot was that a greater proportion of students had a working neural network compared with previous years, suggesting a great success of this scenario. As such the scenario has been used each year since with some changes to the templates (and to the program to help reduce plagiarism between years).

One problem is that the structure of the program is so tightly defined that there is little scope for variation in code – hence copying is difficult to detect. This is partly addressed by requiring the student to comment their code and to discuss the object-oriented aspects of their program in the final report. In addition students were expected to do experiments in their own time to investigate the effects of changing specific parameters in the program. The instructions for the final report were made clearer to try to ensure this happens.

One disadvantage of the approach is that it discourages independent thought more than is ideal. The much increased submission rate, however, is encouraging.

The important aspects of this scheme are the division of the project into suitable, easily marked sub-tasks, the extra support provided in the development of the program and the inherent feedback between sessions.

Follow up

Following the pilot year, it was realised that some functions were more complex than others, so the marking scheme was changed so that an appropriate number of marks for the code and comments was available for each function.

From 2015, with the move to online submission, students upload their document to Blackboard Learn, where the work is readily marked. Rather than circling the marks, the ‘insert text’ option is used to allow the marks to be entered. Comments on errors or suggestions for improvements are also easily added in an appropriate context.

The second part of the assessment requires the student to apply their neural network to a ‘real world’ problem of their choice, to see if the network can learn that problem. In effect the students are researching whether a neural network is appropriate. Given that, rather than asking the students to write a report on their work, they are now asked to present their research in the form of a four page conference paper. This tests them with a new skill, complementing the report writing skills they use elsewhere. This innovation has also proved successful.

Whiteknights biodiversity monitoring: building an app to collate long-term monitoring data of campus wildlife

By edwardwhite Comment Closed

Dr. Alice Mauchline, School of Agriculture, Policy and Development; Dr Alastair Culham, School of Biological Sciences; Dr Karsten Lundqvist, School of Systems Engineering; Professor Alison Black, School of Arts and Communication Design; Dr Hazel McGoff, School of Archaeology, Geography and Environmental Science
a.l.mauchline@reading.ac.uk

Year of activity: 2013-14

Overview
KiteSite
A mobile app was developed for the collection of field data, supporting the activities of the Whiteknights Biodiversity Blog, and providing a central database for students and staff to monitor long-term changes in the local environment on the University of Reading’s Whiteknights campus.

Objectives

  • To develop an app, suitable for use with Android and iOS devices, that was user friendly and had strong branding and identity.
  • To build a community of users for the app that would utilise and enjoy the app for biodiversity monitoring objectives.
  • To create the app as a tool that would support the teaching of biodiversity in a range of modules across several schools.
  • To create an app that could support the work of the Whiteknights Biodiversity Blog in monitoring long-term changes in the local environment of Whiteknights campus, including the University of Reading Phenological Monitoring Network (UoRPMN).

Context

The project to develop the app, which was named KiteSite, grew from Dr Mauchline’s involvement in Enhancing Fieldwork Learning, a Higher Education Academy funded project that sought to promote the use of technology in order to improve student learning in the conduct of fieldwork.

The need for the app grew out of the success of the Whiteknights Biodiversity blog. Since being established in June 2011, the blog generated increasing interest, and coordinated multiple records on biodiversity, including a growing phonological dataset, the UoRPMN. The app was conceived of as a field recording tool that would support the work of the blog in monitoring long-term changes in the local environment of Whiteknights campus. Crowd-sourcing data in the manner that such an app would allow will provide researchers with access to data on more species, over a greater area and period of time, than they may be able to collect themselves.

Implementation

First, a scoping study and literature review were conducted in order to identify existing apps, software and online resources that could be utilised.  Concurrently, six student champions, drawn from five schools across the University, interviewed staff members within their schools in order to establish the teaching needs that could be met by the development of the app.

As a result of these findings, a ‘HackDay’ event was held in December 2013 in order to decide upon the requirements for the basic functions of the app.  EpiCollect was chosen as an open source, generic, data collection tool that could be modified but already provided the functionality of sending geotagged data forms and photos to a central project website from mobile devices. The student champions modified EpiCollect to produce a prototype app, which was then tested by user-groups and refined by agile development.

In order to test the app, a mock species identification session was run, followed by field data collection using the app.  This and further data collection and feedback allowed the app to be refined and the database to be developed and enhanced.

In anticipation of the launch of the app, which was named KiteSite, a website and social media profile were set up, while promotional materials were printed and disseminated.

Ultimately the app was launched in June 2014, and a launch event was held, attracting a number of teaching and learning staff who expressed interest in using the app in their teaching and learning.

KS thumbnail

Impact

The project successfully created the KiteSite app that is currently being used by a small community for the monitoring of biodiversity on the University of Reading’s Whiteknights campus, and supporting the University of Reading Phenological Monitoring Network dataset.

Reflection

Those involved in the project felt that they benefited from working as part of a multi-disciplinary team, as they developed their skills in effective communication and learnt to avoid the use of subject specific jargon.  Given that team members also had other commitments besides the project, it was sometimes difficult for them to balance their workload.

While it was not possible to create a dedicated iOS app, as had originally been planned, a functioning equivalent within the existing EpiCollect app that operates on iOS was created.

The appointment of student champions was valuable, as by having the project led by the principle end-users, they were provided with the opportunity to shape how the final project could be used and developed most effectively for their needs. The student champions took the lead on developing the website for the app, and one of the student champions drafted the reflective paper that was then published in the Journal of Educational Innovation, Partnership and Change.

Follow up

The project team continue to seek further uses for the KiteSite app. While it is used in teaching, the current objective is to engage with student societies that might make use of the app, such as BirdSoc, an ornithology student society.

Other universities have expressed interest in the project, and are looking to set up similar resources mirroring KiteSite.

Links

From a traditional classroom to a flipped classroom

By edwardwhite Comment Closed

Dr Karsten O. Lundqvist, School of Systems Engineering
k.o.lundqvist@reading.ac.uk
Year(s) of activity: 2013-14

Overview

6477A flipped classroom approach was trialed for the Part Two Java module (SE2JA11) taught in the School of Systems Engineering. 

Objectives

  • Encourage students on the module to become deep-level learners, as they analyse, evaluate and create, rather than simply remembering and understanding.
  • Introduce a flexible teaching and learning style that students will find enjoyable.
  • Introduce flexibility that allows students to manage their time in a better way, giving them more opportunities to study the materials in a deeper manner.
  • Improve attendance and engagement with practicals.

Implementation

In the summer of 2013 videos were created for the module.  New slides to present the content were designed, with the fonts improved to make them easier to read on a computer screen.  While the content was based on that of the old slides that were available to students, practical screencasts were introduced in the video, whereby the students can see how the code behaves and how they are supposed to develop it practically.  Some slides were altered so that they presented difficult concepts in more easily understood ways, such as through use of analogies to the restaurant business and the automobile industry.

Feedback and feedforward videos were introduced to explain the progress through the course.  One of the feedforward videos was used to make the students aware of the object-orient programming (OOP) nature of the code, and that the weekly practicals would be building upon previous material.  Students were told that they could use the weekly practicals as a gauge to measure if they had problems with OOP, and should ask the teaching staff for help.

The videos were created using Camtasia, an tool for creating videos and screencasts from webcams and computer screens.  The software suite also has simple post-production tools, which allowed zooming to ensure that the small text of development environments could be viewed easily.  These videos were then embedded as items on the Blackboard Virtual Learning Environment.  Uploading the videos to a streaming service external to the University was considered, but was decided against in order to create a classroom feeling to the videos.

The flipped classroom method generally recommends that videos be simply bite-sized chunks of around 4 to 6 minutes long.  Several of the videos created for the module, however, were over 1 hour long, as a result of the amount of material that needed too be covered, the adherence to the lecturing paradigm, and the lack of time available to transform the material as much as would have been necessary in order to make 6 to 20 minute videos.

Impact

To obtain feedback from the students, two voluntary bespoke surveys were shared with the students, one available in weeks 2-3 of the Autumn Term, and one available in week 1 of the Spring Term. The first survey showed that 84% of students preferred videos over lectures, and that only 4% of students did not expect to watch the videos more than once. In the second survey, 100% of students now preferred videos to lectures, and 100% expected to watch the videos more than once.

Reflections

Flipping the classroom has been of great benefit. As the act of flipping cannot just be a case of replicating old teaching methods digitally, it promotes reflection on course content and teaching methods, and requires thorough planning. The initial investment pays off in the long term as the teaching materials produced can be reused, not only from year to year, but between different modules that have some overlapping content. While the creation of teaching materials may consume more time than the traditional delivery of content, it is flexible as it can be done when time allows, and does not require being present at an appointed time and location.

Despite concerns about the length of the videos, on the whole students expressed satisfaction about this.  The general response was that students expected the videos to be long, as they were replacing 2 hour lectures, and therefore students would feel cheated if the videos were not long and with a lot of content.  While it was agreed that students might benefit from having chapters within the videos to make them easier to search, none wanted the videos to be shorter.

In order to improve how the module is taught using the flipped classroom model in the future, the following recommendations were made:

  • Include a more self-regulated learning approach to the coursework, allowing students more flexibility over the weeks, and removing some of the summative pressures that might induce surface-level learning.
  • Change the module so that 100% of assessment is carried out through coursework. This should make students focus more on the practical work throughout the year, and help them focus more on the relevant material and learning it in a deeper way.
  • Introduce a level of self-regulated learning to the practicals, by introducing a logbook instead of weekly sign-off sheets. Students will need a number of signatures in their logbook to get 10% of their practical marks. The signatures will be given after a short formative discussion of progress provifnng useful feedback and suggestions of further work.

Follow up

The flipped classroom approach continues to be used for the teaching of SE2JA11, and has now been introduced for other modules within the School of Systems Engineering. In particular, videos on general coding theory are able to be utilised within many modules. Dr Lundqvist was able to draw upon the experience of flipping the classroom when creating the Open Online Course Begin Programming: Build Your First Mobile Game.

The recommendations generated by the pilot year have been carried out, with the exception of the introduction of a logbook, which proved impractical. While students still complete weekly sign-off sheets, the sheets are now 50% questions on the video, to ensure that students have viewed the videos and retained the information, and 50% questions on progress in their own learning, with the intention that students will reflect upon their own learning, and staff will be aware of students who are having difficulties.