|
Why is this tool used in evaluation?
Cost-effectiveness analysis is a decision-making assistance tool. It identifies the economically most efficient way to fulfil an objective. In evaluation, the tool can be used to discuss the economic efficiency of a programme or a project.
Focused on the targeted major result of the activity - the number of jobs created - the tool estimates the cost of each job generated by a specific measure. The comparison of various programmes with similar impacts enables the comparison of the costs generated by each job created and provides useful quantitative indicators for the selection of comparative methodologies.
The tool compares policies, programmes or projects. It presents alternatives in order to identify the most appropriate one to achieve a result at least cost.
Cost-effectiveness analysis may contribute to answer the following questions:
- How much does a programme or a measure costs compared with the cost of a particular component of its objective?
- Is it preferable to invest resources in an intervention, to the detriment of another, to achieve the target?
- What kind of intervention or group of interventions yields the best outcomes regarding the final objectives and available resources?
- How can the use of the resources be optimised, given competing needs between programmes?
- At what level of additional investment will the chosen intervention clearly give an improved outcome?
What are the possible uses of cost-effectiveness analysis?
Specificities of the cost-effectiveness analysis:
- Effectiveness is measured with a single outcome which stands as the main expected impact of the intervention.
- It is an economic analysis methodology which assesses the effectiveness of indicators highlighting results and outcomes. It does not evaluate the monetary value of the outcomes.
| Figure 1: The uses of cost-effectiveness analysis |
|
|
Cost effectiveness analysis is an efficient way to evaluate projects, programs or sectors evaluation when the main objective of the policy can be reduced to a single result. This tool is designed for the economic analysis of the operational objectives at different levels.
Cost-effectiveness analysis can be used in:
- Ex ante evaluations to support decision-making and guide the choices to be made.
Depending on the cases, it can be used:
- To foster the debate among decision-makers prior to the decision
- To highlight the preferences of the groups representing different categories of stakeholders or actors involved in the sectors where the intervention is planned
- Ex post evaluations to measure the economic efficiency of an intervention already carried out.
- Intermediary evaluation to update the ex ante outcomes and choose which options should be selected to continue the intervention.
How is a cost-effectiveness analysis undertaken?
| Figure 2 : Steps involved in cost-effectiveness analysis |
|
|
Check the relevance of the analysis to the objectives of the programme
If the outcome of a programme cannot be defined as a priority outcome, or if homogeneous and quantifiable units cannot be determined, the use of cost-effectiveness analysis should be avoided.
The method is adapted for actions in which expected outcomes are clearly identified and whose direct and indirect costs are easily measurable.
Identify the availability and reliability of the data
The analysis requires reliable data, i.e.:
- In ex post evaluations, a quantification of the outcomes
- In ex ante evaluations, a rigorous modelling of the most probable results
Determine the effectiveness criteria and develop the relevant indicator
The choice of the effectiveness criteria depends on the main objective of the intervention.
- In an intervention where the objective is clearly determined, the identification of the effectiveness criteria is straightforward.
- Conversely, when the intervention's objective is broad, the identification of the intervention's main objective should be discussed.
For example, when an intervention aims at improving the effectiveness of the basic education provision, the effectiveness criteria could be the increase in the average level of primary school's basic knowledge. Other criteria may be more relevant depending on the context in which the intervention is implemented.
This increase can be measured through the evolution of the grades obtained in all the courses followed by the students, or in the two courses deemed as the most important, or through the organisation of a single examination for all primary school students.
How is the total cost of the programme evaluated?
Add direct costs
In this type of calculation, only the direct costs invested in the intervention are considered. In the context of development assistance intervention, these costs are often financial: grant, financial transfers, decreases in taxes, financing of projects and activities, etc.
Examine indirect costs
Indirect costs indicate the value of civil servants' work in charge of monitoring the programme or intervention.
Example of indirect costs in the context of a health programme:
Production losses due to working hours lost against the benefit of a vaccination
Social costs: adjustments to the timetable of working conditions of a pregnant woman in the context of a premature birth alleviation programme |
Examine other types of costs
An additional level of complexity within cost calculations is required when other important costs are generated by the project's implementation. For example, cost calculations can include the loss of earnings and benefits due to the fact that public financing have been attributed to a specific objective (this is called the loss opportunity cost).
How is the impact of the programme measured?
Ex ante evaluations
The evaluator must forecast the quantitative results of the programme. Depending on the complexity of the intervention, the use of simulation techniques may be required.
Ex post evaluations
The evaluator can use empirical techniques if the primary data at his/her disposal are sufficiently numerous and reliable. If not, the evaluator needs to estimate the actual quantitative results from secondary data.
How is a costs-to-effectiveness ratio established?
The analysis requires stable elements to support the comparison between:
- Interventions of different nature, but similar objectives, occurring in the same country
- Similar interventions occurring in similar contexts
- The results and what would happen without the intervention, etc.
Comparison of programmes
When the analysis compares different programmes with identical outcomes, the chosen parameter is the cost comparison criteria.
When, for the same objective, the analysis compares different types of interventions with identical costs, it is supported by qualitative elements.
What are the preconditions for its use?
| Figure 3: the preconditions for its use |
The time span |
The required time span for the analysis part depends on the availability of technical and financial data. When available, it can be conducted within a few days. If not, the data collection can take a long time. |
Human resources |
The analysis can be undertaken by evaluators, the more so when one of them has already participated in such an analysis. The evaluators should ensure the collaboration of experts in the country under review, so as to check the relevance of the criteria chosen. |
Financial resources |
Significant resources may be required for the data collection and reconstitution, depending on the availability of the data. Do not forget to take into account the experts' cost. |
What are the advantages and limitations of the tool?
| Figure 4: The advantages and limitations of the tool |
| Advantages |
- A simple and effective ex ante evaluation tool which compares different measures or programmes with identical objectives.
- An educational and communicational tool which summarises the outcomes using a single quantifiable indicator.
- Visibility of the intervention's effectiveness.
|
| Limitations |
- Cost-effectiveness analysis focuses on the main direct outcome of the intervention. The measure of the effectiveness of the intervention's expected results is therefore simplified. When an intervention generates secondary and/or indirect results, the use of cost-effectiveness analysis may be irrelevant or counter-productive.
- Analysis of the effectiveness, not the relevance.
- Whatever the case, ex post situations are more challenging than ex ante situations because the implementation of the intervention generates unexpected costs and impacts. The data collection for these costs and impacts is difficult.
|
|
