Formulating learning outcomes

Guiding the planning of education programmes

The focus of the learning outcomes leads to a number of consequences, where one of the more important is that assessment of fulfilment of learning outcomes becomes central. The examiner needs to know that the learning outcomes have been reached by the student, in order to give a pass.

In order to assess outcome fulfilment, the examination must measure or make it possible to judge fulfilment of the learning outcomes. The outcomes judge what has to be assessed and since the students need to prepare for examination, the learning outcomes will also determine which teaching-and-learning activities the students have to complete during the course. 

Biggs’ (Biggs 2003) model for alignment in education can be used as support material and the step by step guide Designing a course for meaningful learning describes course design.

Outcome based governance has a weakness that has often been criticised when used in the public sector: The follow-up, or the assessment of fulfilment of goals/outcomes, has not been up to scratch, mostly because of poorly formulated goals (Rombach 1991). 

Outcome based education requires a comprehensive commitment to the working process of aligning goals, activities and examination. It also requires active involvement from the students, thus facilitating their understanding of the outcomes, working process and expectations. Outcome based education is then model for enhancing student-centered learning.

Formulating learning outcomes

Formulation of learning outcomes is of course not only a question of wording. For the educational programmes that lead to a vocation, the demands within that vocation decides the content of the outcomes, but also the demands of generic academic knowledge that higher education is supposed to live up to.

On programme level, for a plan of education, the examination guides the content of the outcomes and for each separate course; the learning outcomes have to be included within the outcomes for the whole plan of education for the programme. Irrespective of which type of educational activity the outcomes are formulated, the essential part is what the students need to know and be able to do when the educational programme is finalised. In the education plan, the outcomes are divided into what the students should understand, what they should be able to do (skills) their ability to make professional and ethical judgements.

On course level, the outcomes do not need to include all those three aspects, for example one may have more focus on one of the three types of knowledge, but it is important to combine the three levels altogether when possible. For those who are involved in developing learning outcomes, the next step in your work can therefore be to disentangle what you want that the student should learn into what the student should know, be able to perform etc.

Besides the fact that the above can provide more ideas about what the outcomes would need to contain, it is also a good description to go back to in order to control that important aspects are not lost and thereby are left to the students’ responsibility only. 

Use active verbs and understandable substantives

When you have been thinking broadly and freely about what you want your students to know, it is time to develop the learning outcomes. They can be described as building on a verb part and a substantive part or subject part:

• A verb that describes what the student is supposed to be able to do when the course is finished, e.g. describe, critically assess, investigate, argue.
• One or more words describing the substance of what the student is working on, e.g. the circulatory system, theories regarding reasons for criminality, injuries, dementia and its effect on relatives

The verbs can be practical as well as theoretical/cognitive abilities and optimal clarity is provided when you use active verbs. Avoid verbs where the student will have or receive knowledge and overview. Also relatively common, but a bit vague verbs like understand can sometimes be clarified. The main thing is that the student is starting to get some kind of notion on what he or she is expected to be able to do with the course content, and not the least, that you as teacher can visualise how the outcome could be assessed. Could you use the same verb in the exam instructions?

The verbs in Biggs SOLO-taxonomy (Biggs 2003) has been used in the work of formulating outcomes at KI. Blooms taxonomy (Anderson 2001) can provide suggestions of active verbs and which demands on the student that different verbs provoke. Are they for example supposed to be able to describe a theory or to discuss it? Millers taxonomy for clinical training is used to describe clinical skills. The content should be described in a way that makes it comprehensible for the target group, which has not yet passed the course. Avoid specialised terms.

Effective outcomes

The student who receives the grade pass, or pass with distinction, or anything in between has reached the learning outcomes. In order to know whether the student really has fulfilled the outcomes, your outcomes have to be possible to assess.

The outcomes must also be possible to assess and reach within the time frame of the course and cannot describe what the student is supposed to be able to do during a future career.

That the outcomes need to be realistic may seem self-evident but sometimes outcomes are shaped more to show how much is done or could be done within the course. Let the number of outcomes and the content in these represent the teacher’s ambition with the course and also indicate threshold requirements . At least this is reached by all students.

The assessment criteria complement the learning outcomes and express the variation that is achievable within a course. That other interesting things probably also will happen during the course does not have to show in the outcomes. All learning cannot be governed, but the learning that is defined in the outcomes we need to govern towards - and therefore also be able to assess. 

In order to get useful information from student course evaluations, each outcome needs to be fairly well defined. If the outcome contains several different parts and the students believe they have reached certain parts but not others, they have difficulties responding to a course evaluation.

Outcomes in relation to aim

The learning outcome answers the question what the student is supposed to know. The aim answers the question why the course or course programme exists, which need it fulfils. The course plan should optimally include an aim which is presented to the students. The aim informs the student why the learning outcomes are important to reach and thereby the relevance can be reinforced. 

Outcomes in relation to grades and criteria

Fulfilment of learning outcomes shall be judged and graded. At KI, the main alternative is to use three steps, fail, pass, pass with distinction. Also two and seven steps are used. The students will want to know what differentiates the different steps; the assessment criteria. 

Criteria can be defined as how well the outcome was fulfilled and writing criteria will then be a further specification of the intended learning outcome: What characterises a pass in an examination? What separates the one who passed with distinction from the one with pass?

Outcomes in relation to course demands or activities

Taking part in a seminar or 80 % presence during the course is not learning outcomes but demands that we set up in order to support fulfilment of outcomes. They can be called course demands or mandatory activities.

Too many outcomes

If the outcomes are many, this can be caused by your eagerness to describe all areas, all content that is included. Maybe parts should be moved to the course content description instead.

Another way to sort out a high number of outcomes can be to ask yourself where these outcomes will lead: Why do the students have to know this? You may find that several of the outcomes lead to a common outcome, e.g. being able to perform a survey or to write an essay and that this common outcome is better to choose.

For vocational training, it is also interesting to look at what we include in formulations such as be able to perform a certain type of investigation. We could mean that the performance shall be founded on theoretical knowledge and be implemented in an ethically sound manner in order to say that the student can perform. We could also choose to make a longer formulation in the outcome: “Be able to perform a certain investigation with regard to xxx and based on knowledge xxx.” Independent of which formulations we choose, the outcomes need to be communicated. 

Besides formulating goals and discussing with students what the formulations really encompass, the best way to show the students what they are supposed to know by letting them perform, e.g. analyse or compare and assess their own success in the course as such.

Too vague or too specific outcomes

Verbs like describe or be able to identify, can be perceived very differently, sometimes depending on traditions within disciplines. The outcomes that you have set will not only need to be communicated, but also probably revised. Outcomes that have been shown to be too vague can maybe be changed to more stringent ones but also outcomes that have been shown to be too specific can need revision. Educational programmes are at risk of being too narrow and building on too much detail of the outcomes leaving very little space for variation.

Meaningful outcomes

The outcomes have legal implications since assessment of learning outcomes is an act of exercising authority. If the outcomes are providing a picture of what the future profession will demand, they will be meaningful for the professional development. If they verbalise demands of academic thinking, they will be meaningful from academic point of view.

The alignment with the examination makes the outcomes meaningful for the students’ learning. If the examination includes topics that are not included in the outcomes, it does not matter how well formulated the outcomes are. However, if learning outcomes, examination and teaching-and-learning activities are aligned, the learning outcomes will become an important tool for you as a teacher in your pursuit to facilitate your students’ learning.