Types of Evaluation

Health Promotion vs Instructional Design Evaluation
A number of years ago, I was lucky enough to conduct a needs assessment looking at how asthma was addressed in primary schools.  To guide me I used ‘Program Management Guidelines for Health Promotion’ put together by NSW Health Dept.  It had the familiar components of many evaluation models:  needs assessment, planning, implementation and evaluation, but also added a component of sustainability (surprisingly absent from many Instructional Design (ID) models).  Health promotion programs can span different settings, and emphasis is on population groups and collaboration across organisations, professions, community groups.  A course could just be one component of a program.
ID is about courses of instruction (of course!) and to me it differs in that it is likely to be used within a single organisation, concentrating on a program area or course within that organisation, and is focused on the individual.
This requires a reorientation of my concept and experience of evaluation.
Using An Instructional Design Model
I have chosen to use Reeves and Hegberg ID model for my evaluation project.  It has 6 components:
1. Review
2. Needs Assessment
3. Formative Evaluation
3. Effectiveness Evaluation
4. Impact Evaluation
5. Maintenance Evaluation
Each of these components has a specific purpose and has corresponding activities, procedures and tools which are appropriate to each component (some of which were highlighted by Duginan in my last post).  My evaluation project will be specifically looking at the  formative evaluation aspect of this ID model.

Formative Evaluation

It is not often that a course is spot on the  first time it is created.  The purpose of formative evaluation is to improve a course, ensuring that the kinks and bugs are ironed out before the final version.  Ideally it should happen early in the piece (and not only once) so that information can be fed back into the process.  Formative evaluation activities overarch the areas of program design, planning, development and implementation.

Reeves and Hedberg (2003) highlight the main factors associated with formative evaluation and associated questions to consider:

  • Functionality:  does the product work as designed?
  • Usability:            can the intended learners actually use the program?
  • Appeal:                do they like it?
  • Effectiveness:   did they learn anything?

Each of these factors can be broken down again depending on what is to be evaluated.  For example useability may include features of the interface such as the speed of loading, how it looks and ease of getting around etc.


2 Responses

  1. Sandra this is an excellent post about the types of evaluation, in particular formative evaluation. This type of evaluation is included in Tom Reeves’ evaluation model for instructional design – I don’t think it is an ID model as such unlike ADDIE. In Reeves and Hedberg (2003) they refer to evaluation functions which you have listed. Likewise, I prefer to think of them as types of evaluation – though I guess by calling them functions it helps us to understand how they are used. You are so right in saying that formative evaluation occurs all through the phases and should be done more than once. Otherwise, how can the course or product be deemed ‘good enough’ to be foisted on students? Though this is what we tend to do in the interests of time which is actually not a particularly sustainable model of working because it ends up taking more time to fix up the ‘bugs’. Do you agree?

  2. Yes I agree Bronwyn. Better to spend the time in the planning phase (encompassing formative evalutation) rather than as what is often done instead, jumping quickly to implementation. However evaluation is often in the face of time and budget constraints which puts pressure on the process. A management structure that understands and values evaluation is pretty important, otherwise it can become just a tick box exercise.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: