Working Better Together in a Team Europe Approach

#borderbox { border: solid 1px #1196b4; padding: 2% }

To enable decision-making and promote learning during implementation, actors following a Team Europe approach can choose to use technical assistance for conducting an external, objective and impartial assessment of the TEI relevance, efficiency, effectiveness, its value added and adequacy of joint intervention logic.

Closely linked to other monitoring and evaluation activities described in the MORE framework, this joint monitoring exercise is:

• a complement to the internal monitoring conducted by the EU operational managers and the EU Member States’ development counsellors, as well as to any external monitoring employed by MS and the EU.
• an input for the upcoming evaluation(s) of the TEI in question – providing insight into the progress achieved by the TEI, the evolution of coordination between its members, as well as the extent to which national partners are engaged.
• a way to log lessons learned during TEI implementation.
• an opportunity to update the TEI Joint Intervention Logic if needed.
• a way to capture Stories of Change that the TEI contributed to, which could then be submitted for the annual report (as described in Chapter 3 of the MORE framework).

The draft Terms of Reference presented below are provided as guidance for this type of support and the details should be adapted by the individual TEI as relevant.

Background on the TEI [add title]

[add a short description of the TEI in question, its main partners and a summary of the context]

Methodology

The assessment will be conducted following the methodology outlined in Annex 2 of the ROM Handbook used by EU-funded interventions, adapted to the TEI context.

In particular, the criteria listed above will be assessed using the monitoring questions selected from the following list (taken from the EU ROM Handbook and adapted to TEI):

• Relevance (OECD DAC, 2020: ‘the extent to which the intervention objectives and design respond to beneficiaries’ global, country and partner-institution needs, policies and priorities, and continue to do so if the circumstances change)
  • Does the TEI adequately respond to the priorities of the partner country/region?
  • Does the TEI members/actors adequately consult with the partner country?
  • Does the TEI constitute an adequate response to the current needs and rights of the target groups / end beneficiaries?
  • Is the TEI adapted to the present institutional, human and financial capacities of the partner government and/or other key stakeholder(s) with a role in implementation?
  • Are the chosen Implementing Partner(s) proving to be appropriate?
  • Do all key stakeholders demonstrate effective commitment to the TEI specific objectives (ownership)?
• Efficiency (OECD DAC, 2020: ‘the extent to which the intervention delivers, or is likely to deliver results, in an economic and timely way’)
  • Are the implementation mechanisms proving to be appropriate to realise the planned TEI results and specific objectives?
  • Are the inputs/resources provided by various stakeholders (still) adequate for realising the planned TEI results and specific objectives?
  • Has implementation suffered any delays?
  • Is spending in line with the planned TEI contributions?
• Effectiveness (OECD DAC, 2020: ‘the extent to which the intervention achieved, or is expected to achieve, its intended objectives, and its results, including any differential results across groups)
  • Are the results and outcomes/specific objectives likely to be achieved?
  • To what extent are the results so far inclusive (i.e. ensuring the fair distribution of effects across different groups of the population)?
  • Does the TEI effectively influence the partner’s relevant policy and interventions?
  • Is the TEI implementation having any unintended positive or negative effects? Were the negative effects considered for possible (risk) mitigation?
• Update of the TEI Joint Intervention Logic (JIL)
  • Does the Joint Intervention Logic still reflect the main results, specific objectives and impact that the TEI in question plans to contribute to?
  • Have any new components been added to the TEI that need to be reflected in the Joint Intervention logic, in terms of (a) results, specific objectives or a refinement of the impact statement, or (b) indicators?
  • Are there any indicators in the Joint Intervention Logic for which it has proven impossible to obtain progress data and if yes – what indicator(s) can replace them?
  • Should any indicators identified in the MORE framework for aggregate result reporting be added to the Joint Intervention Logic?
• TEI added value: The positive results the TEI achieves above and beyond what could have been achieved by the sum of its individual components.
  • Are the interventions/components included in the TEI creating complementarities or synergies during implementation?
  • What is the TEI added value, compared to the implementation of individual (parallel) interventions? For example:
  • Did the TEI members agree on and use joint messages in policy dialogue with stakeholders and/or in public relations campaigns?
  • Has Team Europe approach development cooperation become more coherent and less fragmented? Have there been more joint contributions and larger overall mobilised capital?

Prior to the signature of the contract, EU operational managers and MS development counsellors will confirm that the proposed expert(s) do not have a conflict of interest.

In order to ensure that this external assessment is objective and impartial, the experts must not have been involved in the preparation or management of any part of the TEI interventions monitored.

Organisation of the assessment

The expert will analyse documentation and conduct a field mission (if possible, alternatively interviews can be done remotely). They will interview all relevant stakeholders, including EU and MS representatives, national partners and beneficiaries.

After collecting and analysing data, the expert will provide recommendations to improve implementation and increase synergies within the TEI. A summary of key steps is provided in the following graphic.

Image

As outlined in the workflow presented above, activities will be organised in three phases:

I. Preparatory phase:

Objectives of the phase: to structure the assignment, clarify the key issues to be addressed and contract the expert. Main activities during this phase:

• Identification of key documents to be provided to the expert (jointly by the TEI members). This can include the TEI Joint Intervention Logic, monitoring matrix with the latest progress data (if applicable), list of interventions included in the TEI and a summary if available, etc.
• Identification of key stakeholders to be interviewed during the field phase (jointly by the TEI members). This will include: EU and MS representatives, national partners and beneficiaries.
• Informing the stakeholders about the planned study and collecting their inputs on the questions the study should address and logistical organisation (for example, the feasibility of conducting in-person interviews, periods of unavailability, etc.).
• Agreeing on the Terms of Reference (jointly by the TEI members).
• Contracting of the expert.

II. Mission implementation:

Objectives of the phase: collecting and analysing monitoring data in response to the agreed criteria and questions. This will be divided into three sub-phases:

• Desk phase – including an initial kick-off meeting between the expert and TEI members, the expert’s review of background documents provided by TEI members, and the expert’s suggestion for the outline of the final report, lessons learned presentation and field work agenda (to be agreed by TEI members before field work begins).
• Field phase – including an initial briefing of TEI members on the planned monitoring work, interviews, and a debriefing of TEI members at the end of the mission.
• Reporting phase – including analysis of data, writing a report that answers the questions agreed in the Terms of Reference and identifying lessons learned and Stories of Change.

III. Comments and follow-up

Objectives of the phase: finalising the monitoring report and discussing it with TEI members with a view of agreeing on a follow up plan.

TEI members will comment on the draft monitoring report provided by the expert.

If needed, TEI members can share the report for comment with their implementing partner(s). The expert will revise the report in response to these comments. Following the approval of the report, TEI members will agree on a follow up plan, which can include the sharing of conclusions and recommendations with national partners and submission of Stories of Change during the annual aggregate reporting on TEI results.

Working days

As a guidance, please see below the estimated number of working days required for this type of assessment, based on the EU’s ROM experience.