The resource bank contains a selection of resources to help you get started with evaluation. There are also resources to help enhance your current evaluation practice.
Take a look through the different tabs to explore what is available:
The diagram below is based on five different levels of impact. The purpose of this is to support you to plan methodically for the evaluation of your activities or initiatives.
By working through these levels, your evaluation plan is therefore not only relevant to your context but ensures your work aligns with the University’s strategic priorities. Put simply, these levels encourage you to set objectives that go beyond numbers of participants (Level 1) and whether they enjoyed the activity or not (Level 2) to considering what has been learnt (Level 3), how the learning, knowledge and new skills are applied in different contexts (Level 4) and ultimately demonstrate how your work supports our institutional strategic priorities (Level 5). The diagram also shows some possible data/evidence sources which could be used to evidence each level.
To print the above diagram, you can download the PDF version here.
To further help you unpack different levels of evaluation, the following resource contains examples of questions you might want to ask to gather data tailored to different levels of evaluation. You can use these questions or alternatively design your own.
Forming objectives is a key part of the design and planning stage of an activity/initiative. The following guide assists you to understand the differences between goals and objectives, explains how to write SMART objectives for evaluative purposes and provides some examples and top tips to consider.
To download the PDF version, click here.
When thinking about impact, it's important to think about how you contribute to the success of the University more broadly. The diagram below shows areas you may be working to directly impact upon and how they feed into some of the institutions key performance indicators (KPIs). Hover over each area to view more detail.
The PDF version of the diagram above can be opened by clicking this link.
This tab contains a selection of generic guides which cover evaluation more broadly.
Evaluation Cookbook- A practical guide containing a 'pick n mix' of resources for those interesting in designing an evaluation study.
SAGE Handbook of Evaluation- A comprehensive overview of evaluation, available in the University Library.
Higher Education Outreach to Widen Participation. Toolkit 4: Evaluation- The following toolkit is particularly useful for colleagues who may be evaluating Widening Participation Activities. The toolkit does however contain information which all colleagues will find useful whether involved in WP activities or not.
You may also find the following TALIS reading list a useful source for further reading.
- Top Tips for informal light touch evaluation and closing the feedback loop in an online environment - This guide is designed for teaching staff . It provides, tips, tools and a conversation starter you could use to help you evaluate and respond to feedback in an online environment.
- Making use of Blackboard Learn data - This guide contains tips and tools to make use of data available through Blackboard Learn. It aims to help you to gauge if students are engaging with your course content which in turn can help you to tailor your teaching and provide further support or guidance as necessary.
When planning your evaluation, it can be hard to decide which method to choose. Here you can explore types of data, popular methods of evaluation and the pros and cons of each. These are some of the options you can pick and choose from depending on what you are trying to achieve and what resource you have available.
You may also find the following 'How to' guides helpful:
Qualitative methods focus on words rather than numbers. Good for highlighting illustrative or personal information and for giving rich insights into individual experiences. Typically takes longer to analyse than quantitative methods.
Quantitative methods are focused on numbers and quantification. Good for producing charts and diagrams. Can run statistical analysis of results which can help determine significance, differences and associations between results.
Mixed methodologies adopt elements of both of the above.
Focus groups are a series of audio-recorded interviews consisting of small groups (4-8) which are moderated. They focus on a fairly defined topic with questions that encourage participation. A topic guide should be used which helps explore a topic without leading the discussion. Diversity is encouraged via careful thought about selecting participants who have a range of perspectives and experiences.
+ Rich, participant driven data which can inform wider evaluations.
+ Discover ideas you hadn’t even considered.
– NOT generalisable to wider groups or a ‘back door’ to data collection.
– Time consuming (minimum of 4 groups recommended, at least 2 hours transcribing for each hour of focus group time).
– Not good for accessing individuals narratives (can’t get into individual stories).
– If poorly designed, the data will also be poor.
Want to learn more? – Book onto “Designing and Facilitating Focus Groups for Programme Evaluation” via UoR Learn.
There are a number of approaches towards conducting interviews but as an overall group, interviews involve meetings to gather the perspectives from stakeholder groups through a conversation. Interviews allows the evaluator to clarify respondent queries and result in the gathering of rich qualitative data. Interviews can follow an open, semi-structured or structured ‘agenda’, each of which has different benefits and limitations. A brief summary is as follows:
+ Questions are the same across target respondents, allowing comparability.
– Inability to capture unanticipated information a respondent may want to talk about.
+ Allows individual experience to emerge whilst maintaining a focus on similar ground.
– Can’t divert too far from questions without losing the ‘similar ground’.
+ Allows interviewer to respond flexibly to respondents and capture unanticipated information.
– Takes more time to gather systematic information about particular issues.
Questionnaires, Surveys, Smile Sheets. Broadly speaking these methods are all very similar and can be conducted both in-person and online. This method typically consists of a standardised set of questions of a qualitative, quantitative (or both!) nature.
+ Can access information from large groups, especially if dedicated time given in class.
+ Relatively quick and easy to collect and interpret information.
+ Responses are standardised and thus more objective.
– Typically occur after an event when participants may have forgotten important factors.
– Due to standardised nature, there is not opportunity to explain points participants may misinterpret.
– May receive superficial answers, especially in lengthy questionnaires or in times where students are completing multiple surveys.
– Some students may not be willing to answer questions under belief they won’t benefit or may be penalised for certain responses -> help avoid this by ensuring anonymity.
You could also encourage Student Reps to make use of the Rep Online Student Impact Evaluation tool (ROSIE) to collate feedback from peers.
In any evaluation, it is important to uphold ethical standards. The following guides will help you to do so.
When evaluating we always aspire to work in partnership with our students, its the foundation of our academic community.
Here are two templates which are designed to help you plan, reflect and report on your activity/initiative/project.
Evaluation Reporting Template- combines the processes covered in the template above.
The first template is an important part of your overall monitoring and evaluation plan and designed to support you complete the reporting template (if you are required to do so).
If you are expected to submit a formal project report
Complete the Evaluation Planning and Reflecting Template first. The information recorded can later be pasted directly into the Evaluation Reporting Template.
If you are not expected to submit a formal project report
You are not expected to complete these templates unless specifically requested to do so. However, we suggest that you use the Evaluation Planning and Reflecting Template to ensure that your evaluation is as complete as possible, and that all essential aspects are considered. We also suggest that using the Planning and Evaluation Template will help with your professional development through a considered review of the initiative and its evaluation. You may also consider using the Reporting Template on occasion to use as evidence of your professional development in this area.
If you want to disseminate your evaluation findings
Please submit an entry to the T&L Exchange. This is useful to take ownership of your activity and to share your findings with others.
Policy on student evaluation of teaching and learning- This policy covers three forms of student evaluation: mid-modular; modular; and, programme evaluation. The policy provides guidance and information on the purpose, frequency and administration of each these types of evaluation.
Guidance on Programme Evaluation- This guidance aims to outline what Programme Evaluation is, why it is important and how you can put the Policy on Student Evaluation of Teaching and Learning into practice for your Programme. This guidance, whilst not mandatory, is recommended as good practice and will help you to align your evaluation with University strategic priorities and provide evidence to meet the needs of other internal and/or external Quality Assurance processes.
Guidance on running a student survey- This guidance note sets out the process for approval to run a survey to seek feedback from students. This process is intended to ensure that any new surveys are designed for maximum impact and will lead to enhancement of teaching and learning, the student experience and external reputation.
Preparing for School Teaching Enhancement Action Plans (STEAP) (Online resources) - These resources aim to support colleagues preparing to complete the School Teaching Enhancement Action Plans (STEAP) QA/QE process. The screencasts explore key elements that UBTLSE have identified as priorities for an effective STEAP submission.
For Access and Participation areas
Access and Participation Plan (APP) Jargon Buster- This guide runs you through key terms used throughout access, widening participation and student success agendas.
OfS- Evaluation and effective practice guidance- This page links to further OfS resources which promote effective practices that can strengthen impact evaluation within access and participation areas. The resources also help support providers to make better use of evaluation which helps enable the outcomes of evaluation to influence practice. These evaluation practices can also be adapted or adopted into other initiatives in different areas and wider contexts.
OfS-Standards of evidence and evaluation self-assessment tool- This page contains a set of standards of evaluation and a self-review tool for access and participation evaluation. The resources aim to help higher education providers evaluate their access and participation activities and strengthen their evaluation of impact in this area.
OfS- The evaluation challenge: understanding what works in access and participation- This blog post details the move towards a step change in evaluation practice for A&P within the OfS.
OfS- Financial support evaluation toolkit- OfS Guidance on evaluating the effectiveness of financial support.
The following guide highlights some of the various support which is available to help you as part of your evaluation.
To open and download the PDF, click here.
Q: Do you have any examples of T&L Impact Case Studies?
A: Yes, there are some examples, now hosted on the T&L Exchange (Takes you to a new page).
Q: Is there anyone who can help me evaluate my activity/initiative?
A: Yes, whilst it is your responsibility to evaluate your work, there are lots of people within the institution who are able to support you. It will depend on what you are evaluating or need help with as to who is best suited to supporting you. The following guide sets out the various support which is available. Please also feel free to contact any member of the team and we will do our best to help.
Q: I'm worried about over surveying my students. Should I use a survey? What else can I do to evaluate?
A: It is a concern shared by many across the institution. When deciding whether or not to use a survey, it may help to refer back to the principles of evaluation and use these to consider if a survey is the most appropriate way of gathering data. There are a number of other methods which can be used to gather new data and you could also consider making use of existing data rather than gathering new data. For more information on evaluation methods see the 'Methods' tab on this page.
If you have any other question please contact any member of the team. We will do our best to answer your question directly or identify who is best able to answer your query.