This section takes a more in-depth look at the linked issues of monitoring, measuring, indicators and evaluation, with regard to social sustainability.
- Monitoring is a formal process of measurement during a project. It covers the choice of indicators and arrangements for gathering data.
- A measure is a quantified description of outputs or performance. Measures apply where there is a clear and direct relationship between the ways in which the inputs are applied and the outputs and outcomes that result.
- Indicators are tools that measure, simplify and communicate important issues. They can be a proxy measure used where project outputs are not directly measurable. They are useful in assessing project outcomes, and can often be usefully based on subjective assessment – asking people what they think. They are therefore particularly helpful when trying to measure a project’s social impacts.
- Evaluation is a formal process of measurement at the end of a project, which reviews and learns from the data (or information about the indicators) collected.
Why monitor and evaluate?
The monitoring and evaluating of a project is crucial to ensuring that a sustainable long term project is still viable and adhering to its original aims and objectives. It will:
- demonstrate value for money
- account for how resources have been used
- understand progress in relation to the original aims and objectives
- understand the wider context of a project (e.g. changes in quality of life)
- confirm that the long term strategy is still viable
- give you the information and knowledge required to improve the project
Monitoring and evaluating are therefore part of a learning process, and are central to community empowerment. A good evaluation, carried out with the local community, can show that the efforts you and they have made together really are making a difference. This will also give the project the boost it needs to secure its long term future and place amongst the community.
Monitoring and evaluating need to be built into a project right from the start and continue throughout the project's life. This will be time consuming and a cost that needs to be accounted for but it is crucial to involve the community to ensure that the evaluation is able to justify and encourage further community involvement. Developing good monitoring and evaluation processes takes time, especially if issues such as community involvement and participation are being assessed. It is also important to consider what may be influencing changes apart from the project’s activities. There will need to be both quantitative and qualitative assessment.
There is always a temptation to measure what is measurable rather than what is important, particularly with regard to changes in quality of life or social capital. Measures tend to cover two aspects of projects: physical changes and activities. This is usually because they are the most easily quantifiable. The effects on people who have taken part in a project and on the wider community tend not to be measured but this is crucial to the long term sustainability of the project. I cannot stress how important is to keep the community “on board” with the project and believing in its goals and ambitions.
Who to involve?
Measurement needs a commitment from key stakeholders that the work should be done. Stakeholders are people or groups who are either affected by an activity being measured, or those who can affect it. These include:
- project funders, who are ultimately paying for the measurement
- the organisation running the project
- staff carrying out the measuring
- the community who would benefit from the project and may be actively involved in measurement
Different stakeholders will gain different kinds of benefits from monitoring and evaluation, and different stakeholders will also be involved in different stages of measurement, from planning, to gathering information, communicating progress and taking action on the findings.
Involving local people
The conventional view is that evaluation should be objective and carried out by outsiders. In measuring the impact of a project on local people, however, involving local people directly can achieve better results. Evaluations of community projects that are done to the community, not by the community, can easily reduce the overall impact of the project. If the act of measuring is participative, it becomes part of the overall community involvement and in itself increases the project impact and long term sustainability.
If local people are to be involved it is best to do so right from the start. During community consultation you can identify what to measure and possible indicators because people will already be thinking about what matters to them.
What to monitor and evaluate?
It is important to be clear about the nature of an evaluation exercise as evaluation can take place on a number of levels:
Inputs are the resources, both material and human, spent by the project on different activities to achieve different aims. These are relatively easy to measure, and can be measured early in the project life, but are less meaningful in terms of social sustainability goals.
- Staff time and skills
- Skills of local community members
- Funds and support from partners
- Training provided by external trainers
Processes are the ways in which the inputs are used to move towards desired outputs and outcomes. These may include developing individual and organisational capacity and community empowerment. This covers how people work together, how they take decisions, how they get trained and how they find things out. This is not something that people often measure, but if the process goes wrong you usually know pretty quickly, because people stop helping or getting involved.
- Community group meetings to plan the process
- Raising awareness
- Training needs assessment
- Monitoring and evaluation
Outputs refer to specific project products or activities carried out with the resources. Outputs are often the means by which social sustainability aims can be achieved, rather than aims in themselves. Outputs are usually easy to measure and are a key part of any monitoring process.
- Number of people regularly attending group meetings
- Constituted group set up
- Number of people volunteering
- Qualification and training for numbers of people
Outcomes are the impacts of the project activities, or the effects that hopefully the outputs and processes will produce. They may be direct or indirect results of the project, as some outcomes may be influenced by external factors. Outcomes are most directly related to social sustainability aims, but measuring takes time. This tends to happen after the project.
- People gaining new skills and confidence
- Increased community pride and identity
- Empowered community
- Increased social capital
It can appear that participation and involvement are some of the hardest things to evaluate, not least because every case is different. However, some basic measures can be adopted to give a picture of what has been happening.
Ways of evaluating participation:
- Volunteer time sheets
- Percentages of questionnaires returned in target areas
- Number of people attending meetings
- Number of people in community steering groups
- Number of residents coming into group
- Number of events organised by the community
- Number of local people in formal positions
- Number of local people in informal decision-making
- Quality and accessibility of information provided
- Structure of group and % of local people with power
- Number of door-to-door surveys
Measuring outcomes is the most meaningful exercise if we are interested in how well the social goals of a project have been achieved or how long-lasting the difference will be. However, it is also the most challenging one. When deciding what outcomes to measure, it is worth considering that:
- Outcomes should be visible at the end of a project or soon after. It often isn’t practical to evaluate several years later. Even if it were, the ability to distinguish the impact of the project from other external factors would be diminished
- The impact on social capital, or the factors that enable people to work together, deserve special focus because they are often missed. The four elements of social capital om which measures and indicators can be based are trust, local rules and sanctions, respect and reciprocity and links and networks
It is likely your monitoring and evaluation will involve conducting a survey. There are two important points to be aware of at the outset of a project, which will impact on project planning and budgets.
When to survey
- A baseline survey should be done as near to the start of the project as possible. It is important to do an initial survey before the project has had any effect at all.
- Ideally surveys should be conducted at intervals to discern the evolution of social capital and effects. The second and later rounds should be conducted far enough apart that things have changed, but not so far in the future that everyone has forgotten about the project! The surveys should also be conducted at similar times of the year to avoid seasonal bias.
- A survey should, if possible, be conducted around three months after the end of physical work. This means that the evaluation aspect continues after the end of the rest of the project.
In order to understand the results of a survey fully, it may be necessary to get professional help with the analysis, unless there is sufficient experience of this kind of work within the group already. This help could range from advice from a local college or council department to engaging a market researcher to help design and analyse a questionnaire. Remember that it is important that community groups are able to show their data stands up to scrutiny.
How to build on the results?
The reason for monitoring and providing indicators is to promote positive change. The more people who are aware of them the greater the chance of this being the case. It is important to give feedback to those who have participated in the monitoring process. It is also important to communicate to people who can make a difference. A number of themes will emerge from the process, and some will be more or less meaningful to different stakeholders. It is therefore important to clarify the ‘story’ of the impact being made for each audience.
A basic visual report for distribution to local people should include:
- Explanations of what you hope will happen with each measure and indicator
- Ideas about what people can do next
- Ways for people to feed back ideas and get involved
Other communication methods include inviting participants to a presentation, producing a report, distributing results to local media outlets, community newsletters and via social media.
Taking action – using your results to improve human and social capital
With the knowledge taken from the monitoring and evaluation you can start to think about actions to change things for the better.
To give action a helping hand:
- Reflect on current actions/projects in light of what you have learnt about local human and social capital
- Think creatively about what new actions would enhance a project even more
- Plan and implement action. It may be a good idea to set targets to show how you hope the indicators will change as a result of the project, and consider how the project or future projects should change in the light of evaluation
This section is deliberately long and detailed because of its importance. A community project doesn’t work without the community. Even if they are not immediately enthused by the idea and its outcomes the monitoring and evaluating process will make them feel involved and give them a “voice”. The information received back, good or bad, will help guide the future of the project and ensure that the community feel genuinely integrated into the process. The connections and groups that this process creates will also help to sustain a community feel and approach to any future projects.
Go to the next section - Environmental sustainability
Go to the previous section - Monitoring and indicators