Annex 7. Definitions and Data Quality Control
Definitions
To support monitoring and reporting on results, the following definitions have been agreed:
- Target group and beneficiaries - the target group consists of the intended beneficiaries of a development intervention. In other words, these are the persons or organisations for whom the intervention is expected to make a certain change in the future. The target group is directed towards the future. On the other hand, the beneficiaries are the persons or organisations actually reached by the intervention, in the present or past.
- Ex ante values refer to estimates of expected results. They are usually determined through feasibility studies, before the intervention is implemented.
- Ex post values refer to values of actually realised results. They are collected during the implementation of interventions, after or shortly before their completion. The term ‘value at the time of final control’ is used by some MS institutions - this corresponds to the meaning of ‘ex post’.
- Measurement - A measurement is available when results are collected through comprehensive monitoring (e.g. participant analyses), surveys or remote sensing methods. Measurements are primary data collections. Measurements are preferable to estimates, provided that they can be implemented with reasonable effort.
- Estimate - Estimation is the approximate determination of numerical values, sizes or ratios by means of visual inspection, experience or statistical-mathematical methods.
If measurements are not possible, estimates based on primary and secondary data or experience gained (e.g. from similar measures with own measurements) can be used in reporting. These data, which are considered to be contextually relevant, are transmitted or generated or extrapolated by mathematical-statistical-methods. In the case of estimates, a quality rating must also be taken into account: The more detailed the estimation method, the higher the chance that the estimate is a valid approximation of the true value.
Quality control (QC)
Results data refers to qualitative and quantitative information provided by TEI members on their joint contribution to results and objectives that were agreed in the JIL. This data will undergo quality control in the following manner:
| For qualitative Stories of Change | For quantitative results data following indicators in Table 1 above, for which data will be aggregated annually | For quantitative and/or qualitative results data for the other indicators agreed in the Joint Intervention Logic | |
|---|---|---|---|
| Timing | Annually, before submission to central level on 30 April | Annually, before submission to central level on 30 April | As agreed by TEI members |
| Responsibility |
|
|
EU and MS development counsellors and operational managers who oversee the relevant TEI components – with support of TA experts and Implementing Partners as needed. |
| Relevant template | Annex 4 | Annex 5 | Example in Annex 2, can be adapted |
| Standards that constitute the basis for the results data QC |
Requirements are explained under each field in Annex 4. QC level 1 will confirm the relevance, accuracy, completeness, validity, uniqueness and timeliness of the Story of Change167. QC level 2 will check the completeness (in terms of the template requirements) of data and investigate any data gaps. |
Requirements are explained under each field in Annex 5 and in the methodological notes for the indicators in Annex 6 - part 2. QC level 1 will confirm the relevance, accuracy, completeness, validity, uniqueness and timeliness of the quantitative data. Every TEI members (in the field and/or HQ level) will follow their own standard procedures for reducing the risk of double counting between interventions and years of reporting. At TEI level, members wishing to contribute data to the same indicator will investigate and reduce any risk of double counting of beneficiaries (guidelines for this will be prepared and can be added as an annex in the future). QC level 2 will check the completeness of data and further investigate any data gaps and risk of double counting. |
Progress values will come either from: (a) international databases whose quality is checked by their owners (i.e. the UN for any SDG indicator, national reports or similar sources), or (b) assessments and interventions funded by TEI members – in which case, the TEI members or their Implementing Partners (if any) are responsible for data quality. |
167 Adapted from “Quality Dimensions, Core Values for OECD Statistics”, https://one.oecd.org/document/STD/QFS(2011)1/en/pdf.