INTPA/ESS Initiatives

A DEVCO/ESS INITIATIVE

Conferences on Evaluation in hard-to-reach areas

This space is dedicated to a cycle of lunchtime conferences organised by INTPA/ESS that took place between February and June 2019. The initiative was developed to share the learning from innovative techniques and methods used in evaluation in hard-to-reach areas, where traditional methods tend to fail.

You will find here all relevant information on the cycle of conferences, including its final product, the Call-to-Action paper on Evaluation in Hard-to-Reach Areas, published in January 2020, laying out the key takeaways and techniques presented in the conferences. Following the Quick Access Menu is a referenced summary report on the Cycle of Conferences.

For more information on these conferences, you may contact the ESS team at conferences@evaluationsupport.eu

Quick access to the conference resources

1. Combining satellite imagery with traditional evaluation techniques

2. Improving evaluation through geospatial data and analysis

3. Use of mobile phones and evaluation in the contexts of fragility, conflict and violence (IDEAS)

4. Qualitative and quantitative techniques in remote evaluations

5. Conflict sensitivity: evaluating in the contexts of violent conflicts

6. Technologies for monitoring in insecure environments / Project Drawings, an innovative remote evaluation technique

7. Use of innovative technologies: new ethical standards in evaluation?

    

1. Conference presentation paper

2. The end-of-conference Call-to-action paper

3. Referenced literature & other useful documentation

The Media section comprises photos that were taken during the cycle of conferences.

REPORT ON THE HRA CYCLE OF CONFERENCES

Evaluating in a fragile world

Around 2 billion people live in countries affected by fragility, conflict and high levels of violence, and around half of the world's poor live in fragile or conflict-affected states (FCAS)[1]. The percentage of the poorest living in FCAS increased from 20% in 2005 to 43% in 2015[2], and the World Bank estimates that, by 2030, their number will rise to nearly 50%[3] while OECD considers that - without action - this number will be higher than 80%[4].

The EU and its Member States have committed to pay particular attention to fragile and conflict-affected states and to support the most vulnerable[5]; this has been one of the key priorities of both EU development and foreign policy over the last decades. In 2016 the EU's development cooperation with countries in situations of conflict and fragility represented EUR 4.970 billion in commitments, or 52.8% of total commitments of DG DEVCO.

In terms of commitments, the yearly engagement in 2016 was 76% higher than in 2015 and 249% higher than in 2014. To limit the analysis to just five fragile countries (Afghanistan, Central African Republic, Somalia, South Sudan and Yemen), the EU disbursed over the period 2010 to 2018 EUR 4.195 billion; with 73% financed by DG DEVCO and 27% by DG ECHO[6].

Adapting the evaluation methods to fragile contexts and hard-to-reach areas

A traditional approach to evaluation in fragile or conflict-affected states and, more generally, in hard-to-reach areas is destined to fail: the number of professional evaluators available to travel to these countries is limited and the security risks during in-country travel make conventional field missions unrealistic, particularly in the most remote areas.  In response to this, various development partners have started encouraging the use of methods and techniques that are innovative in an evaluation context.  These include use of geo-spatial data, surveys administered by non-specialised local enumerators, phone/tablet voice or data surveys, location tracking, communication through online platforms, etc.

Pioneering Actors and Techniques in Evaluation

It is now time to learn from the experiences of these early pioneers and reflect on some paradigms that are often overlooked, such as the importance of integrating conflict sensitivity as an area of analysis when evaluating in FCAS, and the impact of these techniques on the ethical standards of the evaluation profession.

With this initiative, organised by the Evaluation Support Service of DG INTPA, the Unit ‘Results, Monitoring and Evaluation’ of DG INTPA has invited interested colleagues in both Headquarters and EU Delegations to reflect on the possibilities offered by innovative evaluation methods to respond to the difficulties that traditional evaluation missions inevitably face in hard-to-reach areas, and discuss the use of such methods also beyond the FCAS.

The Cycle of Conferences

This reflection began with a cycle of lunchtime conferences that took place between February and June 2019, hosted by at the EC InfoPoint (43 Rue de la Loi - 1040 Brussels, Belgium). The video recordings and conferences materials can be found on the conferences' page via the menu on the left.

Download the conferences' presentation paper from https://europa.eu/capacity4dev/devco-ess/documents/evaluation-hard-reach-areas-presentation-paper.

The conference cycle was a great success with more than 200 participants and 5000 online spectators.

Final Publication

The lessons learnt from this initiative have been gathered in a final Call-to-action paper published in January 2020. This paper provides is a call to DEVCO Operational and Project Managers to evaluate their interventions even in Hard-to-Reach Areas, and to evaluation contractors 

This paper is addressed to evaluation managers in the EU Delegations and in INTPA headquarters, as well as partners and professional evaluators in the development field. It may also provide useful insights to colleagues from other Commission DGs and services.

It consists of the following sections :

1. Conflict Sensitivity
2. Preparation
3. Tools (tech and non-tech methods)
4. Reporting

Conception and design of the cycle of conferences

Marco Lorenzoni – Evaluation Support Service, senior evaluation expert

With thanks to all the speakers that enthusiastically accepted to share their experience; the entire ESS team; the Sector ‘Results, Monitoring and Evaluation’, Unit ‘Evaluation and Results’ of DG INTPA; and the different officers of DG NEAR, ECHO and FPI met during the preparation of this cycle of conferences for their ideas and suggestions.   Additional thanks to the European Evaluation Society (EES), IDEAS (International Development Evaluation Association), IOCE (International Organization for Cooperation in Evaluation), EvalPartners and Monitoring and Evaluation News (https://mande.co.uk/) that kindly helped us in spreading the voice about this cycle of conferences to their respective communities.

Should you need any further information about this initiative, please contact the ESS at conferences@evaluationsupport.eu

Footnotes and references

[1] https://ec.europa.eu/europeaid/policies/fragility-and-crisis-management_en

[2] Same as footnote 1

[3] http://www.worldbank.org/en/topic/fragilityconflictviolence/overview

[4] OECD (2018), States of Fragility 2018, OECD Publishing, Paris, https://doi.org/10.1787/9789264302075-en    

[5] European Consensus on Development, 8 June 2017, https://ec.europa.eu/europeaid/new-european-consensus-development-our-world-our-dignity-our-future_en

[6] DEVCO’s Statistical Dashboard, viewed on 12 November 2018 (EC internal)