Resource bank: clone
The resource bank contains a selection of resources to help you get started with and enhance your evaluation. Take a look through the different tabs to explore what is available:
The Impact Design Steps is for when you are starting to plan a new initiative or activity and when reflecting on the success of your activity/initiative after completing your evaluation.
It prompts you to think about impact at the design stage and by doing so, this allows you to track impact throughout the life-cycle of a change, initiative or project.
It contains a number of prompt questions, which help you consider essential factors at an early stage and enable considered reflection post-evaluation.
To consider and complete the steps, you may find the following template helpful. It contains further prompts and space for you to write down your thoughts as you plan and reflect on your activity/initiative.
If you are evaluating an activity relating to the Access and Participation Plan, you should refer to the Office for Students' Standards of Evidence. There are three Types of Evidence:
Type 1: Narrative. The impact evaluation provides a narrative or a coherent theory of change to motivate its selection of activities in the context of a coherent strategy.
Type 2: Empirical enquiry. The impact evaluation collects data on impact and reports evidence that those receiving an intervention have better outcomes, though does not establish any direct causal effect.
Type 3: Causality. The impact evaluation methodology provides evidence of a causal effect of an intervention .
The evaluation planning template helps you to design your evaluation, by leading you through each stage from defining why there is a need for the activity or intervention, planning your evaluation questions and data collection through to analysis and timelines. You can edit or adapt the template to your context as you require.
Creating this detailed evaluation plan ensures you have thought about each stage of the evaluation. The plan can be followed and referred to during the busy period of activity. It also forms the basis of an evaluation report which you will write later.
APP Evaluation Planning Template
T&L Evaluation Planning and Reflecting Template
You should also consider creating a Theory of Change. This helps you to map out how your activity would theoretically lead to the desired outcome. You can ask the Research & Evaluation team for support in creating a Theory of Change if your activity is related to the APP.
When designing both the intervention and the evaluation, you might benefit from engaging with our Student Inclusion Consultants, who advise colleagues on how to make their courses and services more accessible and inclusive to our diverse student community. You could also encourage Student Reps to make use of the Rep Online Student Impact Evaluation tool (ROSIE) to collate feedback from peers.
When planning your evaluation, it can be hard to decide which method of data collection to choose. Here you can explore types of data, popular methods of evaluation and the pros and cons of each. You may find the following 'How to' guides helpful:
See also validated scales from TASO and the Toolkit for Access and Participation Evaluation for APP-related outcomes.
Top Tips for informal light touch evaluation and closing the feedback loop in an online environment - This guide is designed for teaching staff. It provides tips, tools and a conversation starter you could use to help you evaluate and respond to feedback in an online environment.
Making use of Blackboard Learn data - This guide contains tips and tools to make use of data available through Blackboard Learn. It aims to help you to gauge if students are engaging with your course content which in turn can help you to tailor your teaching and provide further support or guidance as necessary.
Making use of institutional data that is already collected, such as module scores, NSS, graduate outcomes or continuation rates can be a great way to collect data without over-burdening students. Contact PSO to request the data you need for your evaluation.
In any evaluation, it is important to uphold ethical standards. The following guides will help you to do so. There is specific wording used for APP ethics statements on surveys, which we have included below.
British Educational Research Association Ethical Guidelines
Ethics wording for APP evaluations (please check with the Research & Evaluation Team for specific revisions for your evaluation):
The personal information you supply on this form will be used to *activity-relevant information* and for quality assurance and data analytics purposes. The contact information you supply is confidential and will be treated in accordance with the General Data Protection Regulation (2016), the UK Data Protection Act 2018 and all applicable Data Protection laws. Please refer to the University of Reading Privacy Policy for more information. The data collected may also be used to evaluate the impact of the activity. Only organisations that require the information for the funding, delivery, evaluation and tracking of these activities will be given access to the data. This data will be anonymised for reporting purposes. Your data will be stored on a secure database for future tracking.
Here are some templates which are designed to help you plan, reflect and report on your activity/initiative/project.
Evaluation Planning and Reflecting Template- mirrors the planning and post-evaluation reflection stages of the Impact by Design Steps.
Evaluation Reporting Template- combines the processes covered in the template above.
There are also related templates for APP evaluations, if your activity relates to under-represented groups.
There are several ways you could disseminate your findings.
- Submit an entry to the T&L Exchange. This is useful to take ownership of your activity and to share your findings with others.
- If your activity relates to the APP, it can be published on the Evaluation and Reports web page and the evaluation blog. Contact the Research & Evaluation team for more advice on disseminating APP-linked evaluations.
- Share at relevant conferences with colleagues from across the sector.
- Share with University of Reading colleagues at the relevant committees or community of practice.
Policy on student evaluation of teaching and learning- This policy covers three forms of student evaluation: mid-modular; modular; and, programme evaluation. The policy provides guidance and information on the purpose, frequency and administration of each these types of evaluation.
Guidance on Programme Evaluation- This guidance aims to outline what Programme Evaluation is, why it is important and how you can put the Policy on Student Evaluation of Teaching and Learning into practice for your Programme. This guidance, whilst not mandatory, is recommended as good practice and will help you to align your evaluation with University strategic priorities and provide evidence to meet the needs of other internal and/or external Quality Assurance processes.
Guidance on running a student survey- This guidance note sets out the process for approval to run a survey to seek feedback from students. This process is intended to ensure that any new surveys are designed for maximum impact and will lead to enhancement of teaching and learning, the student experience and external reputation.
Preparing for School Teaching Enhancement Action Plans (STEAP) (Online resources) - These resources aim to support colleagues preparing to complete the School Teaching Enhancement Action Plans (STEAP) QA/QE process. The screencasts explore key elements that UBTLSE have identified as priorities for an effective STEAP submission.
For Access and Participation areas
OfS- Evaluation and effective practice guidance- This page links to OfS resources which promote effective practices that can strengthen impact evaluation within access and participation areas. These evaluation practices can also be adapted or adopted into other initiatives in different areas and wider contexts.
The following guide highlights some of the various support which is available to help you as part of your evaluation. To open and download the PDF, click here.
Q: Do you have any examples of evaluations?
A: Yes, there are some examples of T&L Impact Case Studies, now hosted on the T&L Exchange.
There are examples of APP evaluations that can be found on the Evaluation and Reports page.
Q: Is there anyone who can help me evaluate my activity/initiative?
A: Yes, whilst it is your responsibility to evaluate your work, there are lots of people within the institution who are able to support you. It will depend on what you are evaluating or need help with as to who is best suited to supporting you. The following guide sets out the various support which is available. Please also feel free to contact any member of the team and we will do our best to help.
Q: I'm worried about over surveying my students. Should I use a survey? What else can I do to evaluate?
A: It is a concern shared by many across the institution. When deciding whether or not to use a survey, it may help to refer back to the principles of evaluation and use these to consider if a survey is the most appropriate way of gathering data. There are a number of other methods which can be used to gather new data and you could also consider making use of existing data rather than gathering new data.
Q: Can I see a full example report?
A: Here is an APP Evaluation Report FULL example.
If you have any other question please contact CQSD or the Research & Evaluation team (APP). We will do our best to answer your question directly or identify who is best able to answer your query.