Evaluation methodological approach

Monitoring, auditing and evaluation are all exercises in sound public management. They are similar and complementary but cover different issues. 

All three contain value judgements but the judgement references are not the same. 

• To avoid misunderstandings and deceptions. If the external evaluation team deals with questions that fall under auditing or monitoring, it may not answer them correctly because it lacks the appropriate resources.
• To reformulate a question formulated in a spirit of auditing or monitoring so that it becomes an evaluation question.
• So that informants adopt an attitude of cooperation with the evaluation, knowing that the relationship with an evaluator is not the same as one with an auditor.
• To promote synergy between the three exercises.

Subject of study

• An audit (in the traditional sense) verifies the legality and regularity of the implementation of inputs.
• Monitoring verifies the sound management of interventions and informs on the progress and use of inputs, and on the progress of outputs and results. Monitoring produces systematic information with a short periodicity, unlike auditing and evaluation.
• Evaluation judges the design and implementation of the intervention from the standpoint of results and impacts mainly, whether they were expected or not.

Judgement criteria

• Auditing judges on the basis of criteria that are known and clarified in advance (budgets, regulations, management standards).
• Monitoring judges progress in the use of inputs and the delivery of outputs, in relation to operational objectives. The quality of outputs is judged along professional standards.
• Evaluation judges the consequences of the intervention in relation to social needs, economic challenges, environmental problems, etc. It refers to objectives but often has to clarify them. The judgement criteria that it uses are seldom established in advance.

Time scale

• Auditing is carried out periodically, according to pre-established rules, or ad hoc if a risk analysis justifies it.
• Monitoring is a continuous process of collection, with frequent reports (one to four per year).
• Evaluation is an in-depth exercise that mobilises complex tools for collecting and analysing data, and that cannot be carried out annually.

Use

• Audit is intended to detect and prevent irregularities.
• Monitoring gives warnings and allows rapid feedback in case of problem. It is also used for reporting on performance throughout the management chain, from the field operators to the top managers.
• Evaluation furthers understanding on the effects of interventions. It makes recommendations for reform, suggests strategic and political options, and puts forward proposals to adjust the implementation.

The required professional qualities

Each exercise requires an appropriate professional capacity and distinct core competencies:

• The auditor has legal and accounting competencies. His/her action is authoritative.
• The monitoring manager has sound professional knowledge in the field of the intervention and management skills. His/her action is of a supervisory nature.
• The evaluation team has investigative skills and sector-specific expertise. It requires the cooperation of the actors concerned by the intervention to collect relevant data, interpret it correctly and formulate a legitimate judgement.

A good monitoring system considerably facilitates the evaluation. When an evaluation deals with intended effects, the first thing to do is to check whether these effects have been monitored, which indicators have been used, and whether baseline data have been recorded. In turn, evaluation reports often include suggestions for improving the monitoring system. 

Auditing and monitoring may detect apparent success or failure and recommend that the evaluation deepens the causes and effects analysis, clarifies the judgment criteria, or validates a good practice. 

An evaluation can show that the right application of a legal or professional standard is a factor of either effectiveness or ineffectiveness, and therefore provides feedback into auditing or monitoring practices.

• If illegal practices or bad management are assumed, then it is better to opt for an audit, to stabilise the situation and to postpone the evaluation.
• If monitoring information is absent or unreliable, it complicates evaluation but does not preclude it. The insufficiency of monitoring should never be used as a reason to refuse to deal with an evaluation question.
• The auditing practice described here is the traditional practice of regulatory audit. Sometimes professional auditors may be requested to judge results and impacts and to answer questions of an evaluative nature. This practice, often called performance auditing, can become equivalent to evaluation. In this case the auditors adapt their attitudes, judgement criteria and professional competencies. The absence of a clear boundary between performance auditing and evaluation is often a cause of confusion.
• The monitoring practice described here is the traditional one. Traditional monitoring systems are increasingly integrated into performance assessment frameworks. Performance monitoring involves the setting of performance targets in terms of outputs and results, the systematic measurement of progress towards targets, periodic performance-oriented discussions, and incentives for good performance.
• The monitoring practice described here is also different from the EC's Result Oriented Monitoring (ROM). The ROM system rates all external aid projects and programmes with a view to the achievement of intended results and impacts. However, the assessment is not continuous and does not rely on systematically quantified indicators.