We have written previously about how our outcomes assessment system, Stedfast, is able to support education and training providers in measuring programme performance. We wanted to continue this series of blog posts to delve a little deeper in measuring the outcomes on each standard.
Levels of Outcomes Measurement
Most organisations will have a series of outcomes that they work towards. Typically this will be:
The process for measuring outcomes applies to any level of the organisation – it does not matter if you are reviewing performance at the organisation or programme level, you will still have the same considerations:
What is useful with apprenticeship standards is that some of the work has already been done for providers. All apprenticeship standards have three key areas of delivery:
Mission, Goals, Outcomes
When conducting a programme review it is worth considering some standard headings:
- Mission – This provider an holistic vision of what you are setting out to deliver and what values you are working towards.
- Goals – This should be a general statement about the overall skills, knowledge and behaviours expected by learners on a particular programme.
- Outcomes – These are the outcomes contained in apprenticeship standards
Providers might also want to consider adding a statement about endpoint assessment as this may demand more than just achieving the learning outcomes listed in the standard.
Once you have an assessment plan in place then the next consideration is how to measure performance against the outcomes.
There are two types of evidence that need to be considered:
- Direct Evidence – This should focus on learner achievement and is easily measurable. For instance:
- Portfolio evidence
- Indirect Evidence – This is typically evidence such as:
- Self-reflection accounts
- Witness testimonies
Also, consider whether the evidence being collected is:
- Formative – evidence gathered during the programme
- Summative – evidence gathered at the end of a programme
Findings and Gradings
Once you have identified how you intend to measure each outcome and have distributed this review the next step is to add findings and grades. Adding findings is quite straightforward. The reviewer will have the expected outcome, the measure and can, therefore, make a judgment based on either direct or indirect evidence. Scoring or grading then tends to be a problem as without structure this tends to be left to how the reviewer is feeling.
We recommend that a scoring matrix is used to provide guidance to reviewers. Each outcome should have scoring guidance for reviewers.
Review and then review again
Once a programme review has been conducted and actions have been identified, staff then need to review and review again to make sure that findings and changes are being implemented. There will be a host of findings such as:
- Learning resources
- Evidence gathering
- Learner support
- Additional Employer engagement
- Assessment plans
- Individual learning plans
- Changes to the curriculum
But, the key message is that it needs to be constantly reviewed.