Objectives diagram and the effect diagram
SUMMARY | ||||||||||||||||||||
Why are these tools used in evaluation? These diagrams are usually used as organisation tools. They provide a framework for the information collection and the undertaking of in-depth interviews during the assessment of the programme or strategy. They relate direct outcomes and field effects with the expected objectives and effects of the programme. The construction of the diagrams should therefore constitute one of the first tasks of the evaluation.
What are the possible uses of these diagrams? In the evaluation context, diagrams are used to reconstruct the intervention logic of the European Commission's support to the country. This reconstructed logic will be shaped into one or more logical diagrams of effect. Prior to the preparation of effect diagram(s), the team will have to prioritise the stated cooperation objectives and translate these into intended effects. These intended effects will form the "boxes" of the diagram(s). Possible "holes" in the intervention logic will be indicated and filled on the basis of assumptions to be validated by the reference group. The effect diagram(s) will help to identify the main evaluation questions.
The faithfully reconstructed objectives diagram displays the objectives system and provides the evaluator with a first approach to the strategy and policies inner quality. Indeed, an unclear, incomplete or incoherent diagram means a lack of relevance in the resulting planning or a lack of faithfulness to the initial objectives system.
How are they constructed? After determining the evaluation scope, the evaluators should construct a diagram illustrating the objectives presented in the strategy and planning documents. The objectives diagram is drawn from this diagram. When the evaluation scope covers one or more strategy papers (geographical) or strategic policies (sector-based, thematic), it is recommended that one diagram per document is created (unless there is a logical continuity in the strategy or the policy). When a logical framework has supported the drafting of a programme, it clearly states the various levels of objectives targeted by the programme. The logical framework is consequently a reference point for the evaluation, as a consequence of the presentation of the objectives diagram's basic constituents. Although the establishment of result-based logical frameworks has not yet been generalised, the effect diagram can usually be deduced from the logical framework of the intervention's objectives. The objectives diagram and effect diagram can also be used in evaluations for projects and programmes whose rationale is not explicit in the logical framework. In practice, the objectives and intended effects of complex policies and strategies often lack explicit presentation and logical structure, whereas the justification for an evaluation is to be able to answer the following question:
What are the preparation stages for the construction of the diagrams?
The terms of reference include information about the timeframe and the financial tools to be assessed. Note that projects and programmes originating from previous documentation can also be implemented during this timeframe. Stage 2: identify the objectives and effects Collect the documentation required for the establishment of the diagrams. It should comprise:
An overall analysis of the break points in the global strategy should be undertaken, without including the objectives of the projects and programmes in progress and scheduled for a different timeframe from the one assessed. Stage 3: construct a faithfully reconstructed objectives diagram A provisional classification could be carried out on the basis of the distinction between three levels of objectives:
La définition des relations logiques occupe une place centrale dans le classement. Elle est avant tout affaire d'expérience. Stage 4: convert the faithfully reconstructed objectives diagram into a logically reconstructed objectives diagram Some faithfully reconstructed objectives diagrams may reveal logical defects in strategy or political papers, such as:
In order to prepare a comprehensive and coherent objectives diagram, the evaluation team will need all available documentation, its own expertise and, if required, that of experts. Stage 5: construct the effect diagram Objectives diagrams and intended effect diagrams share the same rules of construction. The effect diagram is constructed from the conversion of each of the objectives into the corresponding intended effect:
Computer devices Most of the graphics can be addressed with software such as MS PowerPoint. How are the findings presented? Objectives diagrams are established during the organisation stage, where reports and notes should be provided. At this stage, the diagram's construction must be precisely described.
The process through which the logically reconstructed objectives diagram has been extracted from the faithfully reconstructed objectives diagram must also be clearly explained. Verbal presentations The evaluation team will need to present its work (methodology and findings) to different types of people (the evaluation reference group, participants in debriefing sessions). The objectives diagram and/or the effect diagram are very efficient tools for this purpose, providing that they are readable without being over-simplistic. What are the preconditions for its use?
What are the advantages and limitations of the tool?
|