Development professionals are regularly asked to assess results of Capacity Development programmes – a challenging task that is not always easily carried out. But a new tool, called the Rapid Assessment of Capacity, or RAC, has been designed with the intention of being fast, effective and low cost to implement.
Assessing programme outcomes is difficult and becomes even more difficult when measuring Capacity Development, where areas of action and deliverables are not always easily calculated.
However, the new RAC tool assesses the Capacity Development gains of a programme, making it easer to follow the contribution between Capacity Development support (like Technical Cooperation) and the chain of results.
The RAC is built on the logic structure of a more elaborated evaluation methodology and uses a mix of periodic assessments and ad hoc rapid survey methods so as to allow a standardised and recurrent assessment of the results of Capacity Development programmes.
The recent application of the RAC in Bolivia, Chad and Ukraine also offered useful inputs for the finalisation of the methodology.
The three cases presented very different opportunity frameworks and the motivation of the institutions and staff involved was very different in each instance. Thus, they provided a diversified sample for testing.
Jaime Rojas Elgueta, a consultant working with the Development Researchers’ Network used the new tool in Bolivia and is positive about its advantages.
“RAC is not an evaluation of the results of a given programme but rather an evaluation of how the programme has influenced the development of capacities of both the institutions and the individuals that interact with the programme,” said Mr Rojas Elgueta.
“This allows having, in a short time and with reduced costs, a concrete overview on how the capacities of cooperation programme stakeholders’ are being developed,” he said.
The interesting feature, which worked well in Bolivia, is that the coaching approach allows the evaluated persons and organisations to take an active part in the evaulation exercise.
The approach consists of creating a space for dialogue between different persons that have a common objective. The questions are not only asked by the evaluator but also between the colleagues themselves. No one is better placed to help identify new capacities than colleagues who work together. As a result, such participation can be an eye-opener for the evaluated persons and therefore be beneficial for the organisation itself.
The term Capacity Development, which emerged in international development in the 1990s, has since grown to be included in most cooperation programmes run by international development organisations. Greater information on Capacity Development trends can be of considerable assistance to development professionals, said Mr Rojas Elgueta, as it enables better forward planning and efficiency.
| For more information.... |
|
Go to the Public Group on Capacity Development & Technical Cooperation Reform or on the Capacity Development & Technical Reform topic page. For more details and downloads directly pertaining to RAC, visit this recent event page: http://capacity4dev.ec.europa.eu/public-cd-tc/event/capacity-developmen… |
A full transcript of the interview with Mr Rojas Elgueta is available below:
JAIME ROJAS ELGUETA INTERVIEW ON THE RAC METHODOLOGY
Jaime Rojas Elgueta works for the Development Researchers Network, DRN.
1st question: What is the rapid assessment of capacity, or ‘RAC’?
The RAC (Rapid assessment of Capacity development) is a new evaluation tool that aims at evaluating the impact of the effects of a particular programme within the interaction processes of the individuals that participate in the given programme.
The RAC is not an evaluation of the results of a given programme but rather an evaluation of how the programme has influenced the development of capacities of both the institutions and the individuals that interact with the programme.
This allows having, in a short time, and with reduced costs, a concrete overview on how the capacities of cooperation programme stakeholders’ are being developed.
2nd question: can you please tell me how this method worked in Bolivia?
In Bolivia, we had the opportunity to test this new tool, the RAC, within the framework of a budget support programme to one of the key policies of the Bolivian Government, which is the Integral development with Coca policy. This is a four-year programme, with an EC contribution of 26 million Euro.
The characteristic of the RAC test was the possibility to work closely with the persons involved in the programme. The main characteristic, or the first step, was to make the interlocutors understand that we were not evaluating the results of the budget support programme but that our objective was to evaluate the impact of the activities and of the interactions undertaken by the programme with the different governmental interlocutors and with the final beneficiaries, which were the coca producer communities.
This allowed introducing a new tool into the RAC, that can be called either co-coaching or direct dialogue. That is to say that the evaluated persons had an active role in the evaluation, therefore obtaining immediate results in terms of identifying, detecting and observing which individual and organizational capacities were developed thanks to the interaction with the cooperation programme.
Furthermore, it has to be added that the interlocutors’ response, both from the political interlocutors (our interlocutors were the vice-ministry for integral development with Coca, the staff from the vice-ministry, other ministries, such as the Public Finance Ministry) but also from the communities, was an instant interest to participate in the evaluation; they perceived the advantages of this type of instrument.
3rd question: what’s the ‘value added’ of this type of coaching?
The value added of this instrument, other than what we have already said (which is the rapidity, the short period of time required to apply the RAC, the reduced cost and the immediacy of the results) is the possibility to have an exchange and a direct dialogue between the evaluators, the evaluated persons and institutions and the donors themselves. In the specific case of Bolivia, there was an active and direct participation of the EU Delegation in the country, in particular of the Head of Cooperation and of the persons in charge of the programme. We can therefore state that one of the values added is that of having an immediate and non intrusive instrument that allows the participation of all stakeholders.
Another value added is that the RAC results are both qualitative and quantitative. This allows having the necessary elements to observe, correct or reorient the cooperation strategies and policies in a given sector of intervention or in a given country. While the evaluations of results provide information on a specific programme, the type of information on the capacity development allows having a more political and strategic overview: which are the trends, what is happening with the development of capacities, which are the capacities that allow strengthening or increase the partnership relationship with the institutions and governments. This would be the main value added of the instrument.
The co-coaching, that allows the evaluated to be an active part in the evaluation exercise, introduces the dialogue, which is nothing more than an open dialogue between the evaluators and the evaluated but also between the evaluated themselves. It consists on creating a space for dialogue between different persons that have a common objective. For example, when we ask the persons that have participated in a programme, which are the capacities, the new capacities, what is the learning process… it is not only asked by the evaluator but also between the colleagues themselves. Who is better placed than the persons working together to help the others establish, to identify, which are the new capacities? One of the instant results to the participants is to be able to observe, to be aware of the new capacities: what have I learned from this programme, how is my career path evolving… And this also applies to the institution, because it not only assesses the individual capacities but also those of the institution, those of an organizational nature.
This collaborative piece was drafted with input from Jaime Rojas Elgueta, Maria del Carmen Bueno, Nicolaus Hansmann and Adrian Constandache, with support from the capacity4dev.eu Coordination Team.
Comments (5)
Log in with your EU Login account to post or comment on the platform.
This article makes it sound like the tool exists and is ready to use, but the tool itself is not available and some of the documentation says that it is just in the pilot stages and will not be ready until next year.
Hi Brian, thank you for your interest. The final document on the methodology should be published on the EC website in no more than one month. However, it is possible to acquire detailed material, already on this web site and on demand. Best. Enzo
It would be useful to share a bit more technical details on RAC to help understand what exactly it does and how. Including the definition of "results" as it is not quite clear from the text what it means when it says: "The RAC is not an evaluation of the results of a given programme but rather an evaluation of how the programme has influenced the development of capacities of both the institutions and the individuals that interact with the programme." How do you do the attribution to these broader development outcomes?
Thank you Sandor for your remarks. I will try to pick here and there few specifications in the documents to respond to the issues you raise.On the difference between CD and standard programmes’ evaluation. Indeed, the two evaluations are not superimposable. A clear distinction should be made, in the short term, between the evaluation of an institutional CD process and the evaluation of the performance of the same institution vis à vis a set of ‘externally’ given objectives, as it is the case when evaluating a development programme.The CD evaluation aims at identifying the progress achieved, within the institution, in terms of skills, competencies, strategic initiative, implementation capacity, etc., in view of a long term fulfilment of the institution’s mission. The standard programme evaluation aims at identifying the progress achieved, during the life of the programme, toward a set of objectives and performance indicators, which are supposed to be coherent with the institution’s mission.The key value-added of a CD evaluation toward a standard evaluation concerns the assessment of the sustainability of the ‘induced’ outputs and outcomes of the programme. Different cases may arise, viz.:§Both the standard and the CD evaluations give compatible positive or negative results: this means that the induced policy outputs and the related outcomes of the standard programme are both positive and institutionally sustainable, or negative and institutionally unamendable. §The CD evaluation is positive, while the standard programme evaluation is negative: this implies a question of time. The new capacities are not yet translated into new induced outputs and development outcomes, or else the latter were badly formulated.§The standard programme evaluation is positive, while the CD evaluation is negative: the induced outputs and development outcomes are not likely to be sustainable. This is for instance the case in many countries where intensive TA programmes are implemented.On the identification of the outputs and outcomes of a CD process. A big part of the methodology regards such identification. In few words, the idea is that the interaction of an institutional system with a support programme, combined with an existing opportunity framework, will generate a number of expected/ unexpected capacity outputs (knowledge and abilities), which under adequate conditions may be mainstreamed and combined with other factors to generate new sustainable behaviours in the system. The identification of some broad categories of capacity outputs and outcomes is part of the methodology to help on an open and flexible base the participatory research process.On the causality links between the capacity outputs and outcomes on one side, and their determining factors on the other side, the RAC uses standard methods, with only one strong limit: the fact that the acquisition of data and – in large part – also their processing is done through the participation of the stakeholders: identifying examples, re-building stories, cross checking through group work the individual reconstructions and so on. An additional methodological specificity is the strong emphasis of the method on the so called Opportunity Framework, which summarises the positive/ negative driving factors to which the institutional system is submitted.
Nice interview, very insightful. Thanks for sharing!