Resources > Evaluation

Evaluation

Notes on COSHG Forum workshop facilitated by Elizabeth Wheeler  at the National Self Help Forum September 14, 2002.

The following is one participant's recollection of some points made by Elizabeth and by workshop participants.

Doing evaluation involves finding the value in something.

Funding bodies may ask for evidence of things that can be counted, but remember that things that don't lend themselves to counting also have value - for example reflecting, introspection or changes in attitude.

Evaluation is sometimes seen as a chore or something to do when a group is in crisis, but people suggested a number of reasons for doing evaluation, including:-

• To improve
• To see if it's worth continuing
• To explore directions for the future, work out what to do next time
• To change and grow
• To get input, different points of view
• To find strengths
• To decide to work on weaknesses
• To prioritise
• To see if the group is meeting members' needs
• Are we achieving our goals?
• How far have people come?

Someone raised the problem of people saying they want something (eg training) and then, when you organise it, they don't turn up. Elizabeth said that you need to consider the cost to people of getting what they want. (It's one thing to say that training would be good, but another thing to make the time and effort to attend.)

Evaluation can be an opportunity to share ideas and values. Your measures of an activity might not be someone else's measures. It gives people a sense of being part of something. It can renew energy - by counting the value. And it can be part of finishing something to share what people will remember and take with them after the activity is finished.

Process evaluation is the quantitative side of things - Who came? What did they do? etc. There's a need to go further and look at the reasons behind these things.

Impact evaluation looks at things like - what was learned, what was the outcome. It's useful to have a baseline (At the beginning a person knew this or had this attitude)

Recording the facts - what is fact to one person is not to another. It's important to provide space for different narratives.

In doing evaluation, it is necessary to have openness and honesty

Strategies for evaluation

• Forms - they may have gradings or open-ended questions. It's important to have a space on the form for 'other comments'.
• Asking people informally - People may want to please, and some people may be left out, so you may get biased information.
• An external facilitator (asking someone who is not involved in the group to evaluate). Make sure the group still has control of the process.
• Shared discussion
• Build evaluation into every meeting
• A 'round robin' (Each person has a turn to speak briefly without interruption)
• A focus group (a group is given a series of questions to discuss)
• Brainstorms (people are asked to call out at random anything they think of about the topic and all the comments are written up)
• Looking at annual reports, records, databases, etc
• Any opportunities for reflection

Some evaluation strategies rely on people's verbal and writing ability. Other strategies may get a better response from people who communicate better in other ways. Whatever you do there are likely to be one or two people that it doesn't work for (some people won't fill in forms, some people 'don't get' activities with games, drawings, etc which we did some of in the workshop)

Ask, 'What am I seeing/hearing/feeling, etc? What does it mean?'

Acknowledge the possibility of having a nice time when doing evaluation.

Evaluation needs space and time to discuss. Everyone's perception needs to be understood. (Don't say, It wasn't like that, or You shouldn't feel like that. How it was for them is how it was for them.) You might say, What makes you see it like that? What makes you feel like that? How would you like it to be different?

Some other strategies:-
People spend five minutes writing their thoughts
Ask 'what do you remember feeling and thinking at our last session?'
People may write things anonymously. These are put into a hat and then pulled out and discussed.