Working Better Together in a Team Europe Approach

The new European Consensus on Development establishes joint monitoring and results frameworks as core elements of the joint response¹²¹. This reflects the commitment to ensuring a sustainable impact in the use of resources for the SDGs, as well as to implementing development effectiveness principles such as the focus on results and transparency.

Building on its work to strengthen policy coherence for development, the EU should better identify impacts of EU and member state policies on developing countries in its reporting, beyond actions taken’.

OECD peer review, 2018.

Aligning with partner countries’ statistical, monitoring and evaluation systems can provide a stronger and more trustworthy platform for subsequent policy dialogue. However, it may prove to be a challenge. UN Rapid Integrated Assessments¹²² and the Voluntary National Reviews¹²³ completed to date suggest that many partner countries will not be able to report on all SDG indicators, either because some aren’t relevant, or due to insufficient reporting capacity. This highlights the importance of including support to national monitoring processes in discussions about a joint response.

Joint programming experience to date shows that the need to elevate the focus on results has become clearer. The 2017 evaluation highlighted that joint programming¹²⁴ had not yet been translated into results at country level, a view that was shared by the OECD in its 2018 peer review of the EU¹²⁵.

Learning from country experience has shown that:

• Joint results clarify the common goals and intentions of the joint programming partners, helping to raise visibility and structure policy dialogue with partner governments¹²⁶.
• Linking joint results to the joint programming document’s strategic objectives from the planning stage ensures credibility, transparency and accountability.
• Reporting on a common set of results promotes European visibility, maintains momentum in strategic joint planning and enhances mutual accountability.
• The process of developing a set of joint results is itself valuable as it facilitates consensus on underlying challenges and informs resource allocation and activity design, paving the way for thinking about how European partners can implement jointly.
• Clear and visible results can help to identify and establish partnering arrangements with other development actors, including civil society and the private sector.

Furthermore, it has been established that high-quality results frameworks display the following features:

• an explicit underlying logic informs the results chain and planning of activities;
• they align to national monitoring and processes for results management;
• they integrate universal and European values and approaches, such as gender equality and support to civil society;
• indicators, baselines/targets, data collection, data quality and monitoring responsibilities have been carefully considered to confirm the feasibility of joint programme monitoring;
• monitoring is informed by partner country processes so that results reporting can contribute to policy dialogue and programming work; and the results and monitoring processes are reflected in the joint communications strategy to raise visibility of European partners’ support.

The joint results framework should, where possible, draw on existing national indicators measured through national systems which reflect the strategic priorities of the joint programming document and its intervention logic (i.e. the links between resources, actions and expected results). Alignment of selected indicators with the SDG targets and indicators should be highlighted where possible.

Activities to prepare a joint results framework

Establish strategic objectives for the joint programming document

The strategic objectives will be determined as a clear picture emerges regarding the specific challenges and issues to be addressed by the joint programming process. Once they are agreed, they can provide an entry point to defining higher-level results, especially outcome and impact, and to anticipate the link between resources and results. See previous chapter on how to jointly identify strategic objectives.

Identify stakeholders and incorporate results into preliminary consultations

To frame the discussion at an early stage, participating actors following a Team Europe approach should share current monitoring arrangements (policy, process, level, etc.). This means that Member States’ implementing agencies, European development finance institutions, Member States’ export credit agencies and Programme/ Task Managers should be involved in discussions about indicators related to their specific areas of expertise. Consultations with national stakeholders (government, parliament, CSOs, academia and the private sector), especially concerning the national results framework and the SDGs, will inform understanding on priorities, policies and capacity concerns at an early stage.

Agree on the purpose of the results framework

The purpose of the joint results framework and its relationship to other monitoring obligations should be discussed at an early stage to ensure a common understanding of its role and to encourage all participants to think critically about its structure and content.

Experience to date has shown that joint results frameworks can be used to:

• underpin policy dialogue by signalling key priority messages and providing an evidence base;
• establish a logical and verifiable link between resources and activities with higher level results;
• improve accountability in terms of project/programme results;
• contribute to overall monitoring of national development goals; and
• to provide visibility for the joint programming process’ collective impact.

Define key results and the scope of the results framework

The scope of the results framework should be agreed based on the activities prioritised in the joint programming document. It will be necessary to take a strategic view on result and indicator selection, also taking into account other joint strategic documents produced at country level, such as the CSO Roadmap and the indicators proposed in the staff working document of the Gender Action Plan. Participation of CSOs and the private sector in the monitoring process should be considered. Depending on data availability and timeliness, a smaller number of representative results might be easier to monitor and communicate than an extensive list.

Experience to date indicates that including ambitious, higher-level results indicators can help to focus dialogue on the bigger picture. In order to account for potential attribution difficulties, a number of countries (e.g. Honduras, Ecuador) decided to complement what they saw as country impact indicators through a set of more tangible joint results indicators (showing how the group collectively contributes to the impact indicators through their programmes) or development effectiveness indicators (showing how joint programming has led to working better together at partner country level, e.g. number of joint positions taken to the policy dialogue; joint missions; joint evaluations, etc.). This approach is in line with the model used for the EU’s Global Results Framework, which proposes three levels of results indicators (1. Country progress/impact indicators; 2. EU contribution indicators; 3. INTPA institutional performance indicators). It also reflects the approach taken for the TEI intervention logic – thus, following this approach would make it easier to integrate TEI intervention logics into the JP results framework.

Image

In Cambodia, the European partners did not wish to create a parallel system for monitoring results, but rather boost existing national processes. The European Strategy results framework was therefore aligned with the Government’s own strategy and associated national processes for results management including the national process of agreeing joint monitoring indicators between government and the development partners in 19 sectors. The results framework in Cambodia was intended as a concise framework that includes the outcomes which are of particular importance for the European external cooperation programmes and which provide a platform for policy dialogue. In this context, one representative output indicator was selected for each area of European partners’ focus and support, as well as some additional indicators in areas European partners felt were not covered by the 19 sectors but which were important for the group. The results framework was originally 14 output indicators although after the first monitoring report this was expanded to 21 to better capture the breadth of European support in Cambodia (the added indicators covered higher education scholarships and TVET, anti-corruption, green energy, sustainable production, social protection, support for the CSO enabling environment and climate change).

Review potential links with national monitoring systems and results frameworks

The following principles were key for Cambodia:

• use of the joint results framework as a tool for policy dialogue;
• importance of alignment to country priorities;
• selectivity in the choice of results to be monitored;
• references to the SDG targets that the joint programming priorities contribute to; and
• keeping the joint results framework comprehensible for external stakeholders and manageable for the European group.

Explicitly address assumptions and risks

A poorly designed results framework represents a risk to the credibility of the entire joint programming process. In terms of the monitoring system, attention should therefore be given to risks and assumptions related to country level data collection, evidence generation and data processing, and support for national statistical and information systems. The accessibility and reliability of data sources should be confirmed, along with the timeliness and frequency of data collection and analysis, as the joint results will provide the basis for dialogue, reporting and advocacy work.

Include cross-cutting issues and joint values

Joint programming documents are underpinned by the common values of participating partners: commitments to promote gender equality and women’s empowerment (in line with GAP III commitments), environmental sustainability, an enabling environment for civil society (in line with the priorities of the EU Civil Society Roadmap), protection for human rights and democratic governance.

These common and crosscutting issues should be incorporated into the joint results framework. This can be done by ensuring they are reflected in main sector priorities and/or as standalone results. Existing agreed priorities/indicators listed in other policy documents could be drawn on: EU Civil Society Roadmaps, CLIPs/ Gender Action Plan, Human Rights and Democracy Country Strategies and the Humanitarian-Development- Peace Nexus.

Formulate the joint results framework

When putting the final results framework together, consider the following:

Overall approach:

• Place sufficient emphasis on higher-level results, as this will be of more interest to national counterparts and can provide a better basis for policy dialogue. Complement such country progress/ impact indicators through more tangible ‘Team Europe approach contribution indicators’ where needed and applicable, to achieve better attribution and visibility. Here, the TEI indicators – namely the TEI MORE table 1 and GERF level 2 indicators can serve as a source of inspiration (also see chapter on TEI JIL about this).
• The Joint Results Framework of the joint programming Documents and the Joint Intervention Logic(s) of the TEIs should be linked either in a coherent manner or in a single results framework applicable to both JPD and TEI. This can then be annexed to the joint programming document. An example of how the JRF and JIL link up is shown in the diagram below.
• Consider developing a narrative around the Joint Results Framework in order to explain the choices of the actors following a Team Europe approach.
• The joint programming document’s outline of potential communication activities should include links to monitoring work so that results can be used to form the evidence base in reporting and advocacy.

Indicators, monitoring and results:

• An ambitious yet attainable joint results framework rests on the careful consideration of result levels (output/outcome/impact), the quantity of indicators and their measurability.
• Select quality SMART (Specific, Measurable, Achievable, Relevant, Time-bound) indicators, including, at least, disaggregation by sex, age and disability, that can be reliably collected on a timely basis, collect baseline data on these and agree on how to monitor annual improvements with regard to the baseline data. If baseline data is not available, assess whether it can be made available at some point in time (e.g. with the elaboration of the first national results report by the partner country – or with the help of an experts team if it is about results of TE approach actors programmes. In this case, the baseline should say ‘will be determined at the end of year XX, on the basis of XXX’.
• Consider a division of labour in monitoring responsibilities, by selecting specific leads, ‘Chef de file’ responsible for monitoring each objective, their results and indicators (see chapter on JP M&E of this annex).
• Development effectiveness indicators can be a driver for improvements in delivery. They also include measures of the enabling environment for the private sector and civil society, as well as gender equality and women’s empowerment.
• Include measures to ensure that joint programming activities comply with policies on environmental, social and economic sustainability and the protection of human rights.
• Where possible and relevant, incorporate results indicators from other joint strategies such as the CSO Roadmap, the EU Gender Action Plan and Human Rights and Democracy Country Strategies, and the joint Humanitarian and Development Framework documents and TEIs (if not directly incorporated)
• Reference the respective SDGs in the results framework to facilitate national and development partner reporting. Where applicable, show how the selected joint indicators align with SDG indicators (e.g. if the formulation is the same or compatible).
• Consider integrating international indexes and indicators developed or used by the UN, the Word Bank, the African Development Bank or the IMF in order to demonstrate integration among development partners, facilitating common monitoring among them when using similar indicators across all interventions. Indicators should be quantifiable and process based.

TEI Joint Intervention Logic and links to JRF

TEI´s ‘high-level’ impact and specific objectives should match the overall joint objectives/ priorities as set out in the JP document.

The TEI´s ‘specific objectives’ should ideally match the specific objectives of the JP document, under the relevant joint priority – although actors following a Team Europe approach may choose to focus the TEI only on 1 or 2 (not all) JP sub-objectives. In that case, the missing indicators for the other sub-objectives will need to be identified for the overall JRF in annex to the JP document.

Possible JRF template (non-prescriptive)

If there are TEIs with a link to one specific objective, the results and results indicators of that objective should ideally be linked to and reflect the agreed TEI pillars. As a minimum, one outcome/output indicator will have to be drawn from the MORE guidance (annexed to this guidance) Table 1 on Aggregated indicators for global TEI reporting. Moreover, one additional indicator of that list will need to appear at the impact level of the overall objective. If a specific objective has no connected TEI, Team Europe can decide whether to use SDG/ NDP results and related national/ SDG indicators for each result or whether to replicate the more output-based TEI model for those specific objectives.

Image

The below example is an extract from Bolivia’s Joint Results framework – the first objective, highlighted below, reflects the overall objective of Bolivia’s TEI. MORE indicators were not incorporated at this stage, because the MORE framework had not been elaborated, at that point in time. However, the table shows that the objective and sub-objectives are matching for both the JP and TEI. The selected results and indicators also both apply to JP and the TEI. In addition to the ‘country progress’ indicators, baselines and targets taken from the national development plan, another level was added to measure TE approach actors+ contributions – here the baseline states that it would be elaborated in 2022. The last column indicates the JP/ TEI participating countries which will contribute to monitoring these indicators. In red: The lead development partner who will coordinate the monitoring exercise.

Objectivo 1: Impulsar el desarrollo verde y sostenible para ‘vivir bien en armonia con la Madre Tierra’ Logica de Intervencion del TEI.

Indicador de impacto:

• Indice de Desempeno Ambiental (Environmental Performance Index) - Rango Bolivia: 88. Punctuation: 44.3 (2020).
• El equipo Europa+ se ha convertido en el principal socio de Bolivia en materia de Medio Ambiente y Cambio Climatico.

121 OJ C 210, 30.06.2017
122 See Rapid Integrated Assessment (RIA) Tool available here: https://www.undp.org/publications/rapid-integrated-assessment
123 https://sustainabledevelopment.un.org/vnrs/
124 This evaluation evaluates the EU's Joint Programming process during the period 2011-2015. It covers aU regions and countries of EU development cooperation. https://international-partnerships.ec.europa.eu/policies/monitoring-and-evaluation/strategic-evaluation-reports-deprecated/strategic-evaluation-eu-joint-programming-process-development-cooperation-2011-2015_en
125 https://www.oecd.org/en/publications/oecd-development-co-operation-peer-reviews-european-union-2018_9789264309494-en.html
126 See Council conclusions on the Revised EU International Cooperation and Development Results Framework,
14553/18, 26 November 2018, available at: https://data.consilium.europa.eu/doc/document/ST-14553-2018-INIT/en/pdf
127 Developed by Germany on the basis of this Team Europe Approach guidance.