Evaluation methodological approach

.

What does this mean?

An evaluation can be performed before, during or after the evaluated intervention. Depending on the timing, its purpose and use will differ.

What is the point?

• To optimise the resources allocated to the evaluation by launching it at the time it is likely to have the most added value.
• To meet the needs of the main users of the evaluation at the most appropriate time.
• To ensure that a critical mass of results and impacts are already materialised in the field and they are ready for data collection.
• To avoid conflict with the concomitant exercises of review or audit.

An ex ante evaluation is performed before adopting or implementing the intervention. It gives support to the intervention design and contributes to ensuring the design quality. It is concerned with the following points:

• Need to satisfy in the short or in the long run.
• Objectives to be met.
• Expected results and necessary indicators for their evaluation.
• Added value of the community intervention.
• Risks linked to the proposals.
• Open alternative options.
• Lessons from similar experiences already undergone.

It aims at having a direct influence on the decisions upstream from the implementation, to the extent that it transposes lessons from past experiences into the framework of the new intervention.

Mid-term or final

An evaluation during or at the end of the implementation is intended to draw lessons from the first years of the intervention implementation and to adjust the contents of the ongoing intervention in relation to realities in the field and/or contextual developments. It often includes a report on outputs and an analysis of the first results and impacts achieved. It aims at improving the intervention under way and its conclusions may be supported by observations in the field.

Ex post

The ex post evaluation is performed right after or a long time after completion of implementation. It is mainly concerned with checking achieved impacts, identifying and judging unexpected impacts and assessing the sustainability of the intervention's benefits. 

It enables to detect the real changes in the field and, if the changes occur soon enough, they can be analysed to estimate those that are attributable to the intervention. 

The ex post evaluation often aims to report to the institutions that have allocated the resources. Likewise, it helps to transfer acquired experiences to other countries or sectors. 

• Mid-term evaluation offers a good compromise between utility and reliability, especially if it takes place in the second half of the cycle.
• If the evaluation is to draw on the experience of previous programmes in order to improve future ones, it is recommended to establish a multiannual evaluation plan covering several programming cycles.

.

.

What does this mean?

The evaluation can be scheduled:

• For the beginning of the cycle, in the design phase;
• During the cycle (mid-term or at the end of the implementation phase);
• After the end of the cycle.

Is there a best time to evaluate or should three evaluations be performed for each cycle?

What is the purpose?

Many interventions are characterised by successive cycles and show a relative continuity between cycles. 

A new cycle can start before the intervention of the previous cycle has yielded all its effects. 

In light of this, several cycles need to be considered when choosing the timing of the evaluation, in order to:

• Draw conclusions from an observation of the effects of the intervention in cycle "n-1".
• Feed into reflection on the intervention in cycle "n-1".
• Observe the implementation of cycle "n" and rapidly provide feedback.

How to take the cycles into account?For defining the purposes of the evaluation:

• If the evaluation takes place at the beginning of the cycle, it verifies the relevance and coherence of the objectives of intervention "n" by analysing needs and also the effects of previous interventions "n-1" and "n-2".
• If the evaluation takes place in the middle of the cycle, it aims to improve intervention "n" by taking into account its first observable effects, and also by analysing the effects of intervention "n-1".
• If the evaluation takes place at the end of the cycle, it draws conclusions from the analysis of the first observable effects and also from the effects of interventions "n-1", with a view to preparing the strategy and objectives of the next intervention "n+1".

For choosing evaluation questions

• Retrospective questions relating to the impacts and sustainability of preceding interventions.
• Concomitant questions relating to the direct results and relevance of the intervention under way.
• Prospective questions relating to the strategy of the interventions in the next cycle.

Recommendations

Where possible, ensure that the evaluation ends about a year before the end of the cycle. At that stage it will still be possible and useful to adjust the implementation because it always lasts for a year or two after the end of the cycle. It will also be possible to assist in planning the intervention strategy of the following cycle. Finally, it is not too late to perform a sound analysis of the impacts of the previous cycle's intervention.