Skip to main content

Evaluation methodological approach

Group
public
46
 Members
2
 Discussions
210
 Library items
Share

Table of contents

Context indicators

Share
SUMMARY

Definition

A context indicator is a datum which provides simple and reliable information describing a variable relative to the context. It gives information about a situation and its evolution in a country, or an area relevant to the assistance policy.

Other types of indicators:

  • Programme indicators, which detail the resources, implementation, results, and, if possible, the impacts of an ongoing activity
  • Evaluation indicators, which, in addition to the evaluation criteria, enable the evaluator to judge the programme's relevance, coherence, efficiency and effectiveness, and support answers to evaluation questions

Indicators are designed by national statistical services and in the context of specific programmes. National statistical services, many donors and international organisations have co-operated to establish standard indicators, to ease comparisons over time and between geographical areas.

The European Commission's use of context indicators in country evaluations:
Economic indicators: GDP, growth, balance of payments
Social indicators: population, unemployment, educational level, health
Indicators of services provided to the population: education, health, drinking water, electrification
Others: indicators for the analysis of poverty in ACP countries

These indicators are often designed to highlight the specificities of a local context without, however, enabling the evaluator to make a comparison between countries (for example, information by comparison of the level of the indicator in another country) or get a global and normative view of the country's situation.

What use can be made of context indicators?


To present the country

Context indicators are usually displayed in the introductory chapter of the evaluation. They deal with:

  • Economic and financial fields (GDP, trade flows, debt)
  • Social fields (demography, occupation, gender)
  • Specific important sectors (education, health, environment)

In certain cases and countries, the emphasis may be put on particular sectors or issues (for example, poverty analysis, conflict analysis). 

To portray the country's level of assistance

Context indicators also describe the nature of the assistance provided to the country. They should indicate the type of assistance, the amounts disbursed, the sector-based allocation of assistance and the European Commission's activity, compared with that of other donors. 

To answer evaluation questions

Context indicators can also be used to answer evaluation questions which need a preliminary presentation. They facilitate the understanding of the country's situation for the readers.

The following table shows a selection of indicators which are internationally comparable. Tanzania's situation is compared with the situation of a group of 7 African countries considered to be similar: Burkina Faso, Gambia, Ghana, Madagascar, Mali, Nigeria and Zambia.

Figure 1 : Governance Research Indicator Country Snapshot (GRICS) in 2004

Source: World Bank

* Progress is assessed from the most positive (++) to the most negative (--) evolution with respect to the indicator's evolution over the last 3 to 10 years

How are context indicators selected, collected and used during an evaluation?

A good indicator should:

  • Be relevant: it should correspond to an interest, an objective or a need.
  • Be sensitive: the quantity of measurement must reflect, to a sufficient degree, a change that has occurred in the variable to be measured.
  • Be practical: reliable data should be collected quickly and at a reasonable cost through a rigorous process of construction.
  • Be easy to interpret and use: the concrete, visual and intellectual use of the indicator should be straightforward for the user.

European Commission indicators

DG Dev indicators for the appraisal of country assistance performance 

The indicators used for the appraisal of country assistance performance by the European Commission's DG Dev have been constructed with the assistance of various donors, including the European Union Member States, the World Bank, the UNDP and the OECD/DAC).

Their construction is mostly based on the following typology, and particularly on impact indicators.

Figure 2 : Typology of indicators

 

The construction and implementation of such indicators target two goals and require the monitoring of two distinct series of indicators, in order to:

  • Measure the results of the country's policies in terms of poverty alleviation and improvement of living standards
  • Detail the outcomes of sector-based policies which are targeted by the Commission's assistance

EUROSTAT data

EUROSTAT provides Member States with numerous data on several topics which are partly expressed by indicators and ranked into short-term (balance of payments, consumer prices, etc.), long-term (economy and ecology, the business structures, etc.) and structural factors (employment, general economic background, environment, etc.). Moreover, EUROSTAT holds data on trade flows between European Union Members and the rest of the world. 

United Nations indicators

Indicators related to the Millennium Development Goals

For the Millennium Goals, the levels to be targeted and 48 Indicators Development Goals have been identified. They are available on the websites of the United Nations and the World Bank. They deal with more than 200 countries, and their methodologies and their precise definitions are presented to ease their understanding and use. These indicators focus more on outcomes than inputs. 

For example, the progress accomplished in the implementation of a programme aiming at promoting gender equality and women's empowerment is measured with 4 indicators:
 

  • Girls' primary, secondary and higher education enrolment ratios in relation to boys
  • Literacy rate of women aged 15-24 in relation to men
  • Share of women in non-agricultural wage employment
  • Share of women in single or lower houses of parliament

These indicators assess the progress of a country in the achievement of an objective year after year. They can also facilitate comparisons between countries.

Common Country Assessment indicators

In 1997, the United Nations decided to create a system of Common Country Assessment (CCA). In this system, the CCA is used as a tool for analysing the country's development situation and identifying the main development challenges.

Development index

The United Nations database is one of the most developed in the world. Approximately 200 indicators have been developed.

This database is less developing country-oriented than other databases; yet it presents indicators and development indices, which are designed by the United Nations. 

Human Development Report

Each year, the UNDP publishes its Human Development Report, which includes a large proportion of the United Nations indicators. 

World Bank indicators

International Development Association indicators

The International Development Association has recently presented in "IDA Results Measurements System: Progress and Proposal, April 2003" a series of indicators aiming at improving the monitoring of the countries' development outcomes, and particularly for countries benefiting from a Poverty Reduction Strategic Programme.

IDA indicators cover the following fields: income and poverty, malnutrition, maternal and child health, HIV, gender, primary education, drinking water, infrastructure, private sector development, public sector management, and economic growth.

International Comparaison Program

Founded in 1968, the International Comparison Program is a statistical system used to produce data by country. These data facilitate international comparisons based on prices, expenses value and purchasing power parities.

Because of the information about the purchasing power, this statistical system provides the evaluators with comparable data that are valuable for economic and social topics.

Purchasing Power Parities (PPP) are monetary conversion rates which express the purchasing powers of various currencies in a common unit. In other words, PPPs can determine how much national currency is needed to buy the same amount of goods and services in any countries. In that sense, PPPs are monetary conversion rates which erase price differences between countries.

OECD indicators

In its statistic portal, OECD offers a range of precise and updated information about its member countries, from which indicators can be constructed. This descriptive information covers sector-based, social and economic fields.

Precise data (geographical, such as country-, regional-, or sector-based) dealing with non-member economies and their development are available on the OECD website. They describe the context in which assistance to a country is carried out.
The OECD series of thematic and environmental indicators may be usefully consulted.

Other sources

Transparency International indicators

Transparency International seeks to provide reliable quantitative diagnostic tools regarding levels of transparency and corruption, both at global and local levels.

The best known of Transparency International tools is the annual Corruption Perceptions Index. It ranks more than 150 countries in terms of perceived levels of corruption, as determined by expert assessments and opinion surveys. The Global Corruption Barometer (GCB) and the Bribe Payers' Index (BPI) complete the CPI.

NGOs such as Human Rights Watch and Amnesty International publish reports on a large number of countries dealing with human rights and other important issues.

What are the preconditions for its use?

Figure 3 : the preconditions for its use
The time span
  • The time span is the time dedicated to finding available indicators. The evaluator can quickly complete this task using the Internet.
Human resources
  • The human resources required for the collection and selection of indicators differ largely with the themes under study. 
  • The selection of an appropriate indicator, such as maternal health progress, may require the advice of a health expert.
Financial resources
  • The financial resources required for data collection are very limited because indicators are available on Internet, where access is free most of the time.

What are the advantages and limitations of the tool?

Figure 4 : The advantages and limitations of the tool
Advantages
  • A way to quantify information, preferably in a standardised form, in order to make comparisons in time and space. 
  • A way to simplify situations in order to understand them better. 
  • Elements that can be used as evidence for presentation.
Limitations
  • A simplistic explanation of the situation, which, in turn, becomes exaggerated. 
  • Errors of measurement. 
  • Difficulty in developing indicators which are sensitive to slight changes in the context at the macro-economic level. 
  • Problems with the availability of reliable and standardised data over a long period of time. 
  • Differences in the understanding of the meaning of an indicator between various users, and particularly between the donor and the beneficiary country.