Development Cooperation Handbook/How do we Monitor and Evaluate Projects and Programmes?

How do we Monitor and Evaluate Projects and Programmes?

We said before that in case of development cooperation activities, where there are a number of actors co-operating in a non-hierarchical structure, "planning" and of "working along the plans" is an essential prerequisite for success. And that actions should be "beneficiary based", i.e. designed and managed to serve the need of the target population (and not the needs of the donors or of the employed organizations). Now the question is: how do we verify that plans are really designed through participated approaches? How do we know if the organisations are really "walking the talk?". How do we find out is the project results are being used for sharing knowledge and improving the program cycle management?

The subject dealt with in this chapter of the handbook is more extensively debated in a separate Wikibook Evaluating Development Cooperation

Monitoring

Project monitoring is done by project managers who need to know if activities are being undertaken along the plans, within the time and resource allocation that were decided and if they are delivering the expected outputs. In this sense monitoring is identical with performance measurement and belongs to the project execution and control phase of project management. Monitoring reports are done by comparing the project achievements against expected results, timelines and costs. It answers to questions like:

Is happening what is supposed to happen?
Do the actions follow the plan?
Are the employees doing their tasks as they were expected to do?
Are the tasks managers working efficiently?
Is the team working efficaciously?
Are the employees doing their tasks as they were expected to do?
Are the tasks managers working efficiently?
Is the team working harmoniously?

Evaluation

Project evaluation is different: it is a judgment about the capacity of the project to contribute to the program objectives. In the project evaluation we see if the outputs delivered are consing to the expected outcomes. In the project evaluation we see if we are really inducing the changes that we would have liked so that the factors determinate poverty and suffering are diminished and development is facilitated. Evaluation is based on monitoring but adds a "judgment" on the correlation between activities performed, outputs delivered, changes induced objectives achieved and impacts obtained upon the factors generating the problems and opportunities that motivated the project. In evaluation we want to see if we are having the outcomes and the impacts that we wanted to have. While the delivery of project outputs is the responsibility of the project team, the achievements of project outcomes and objectives depends also by the way stakeholders utilize the project outputs to interact with the rest of the community and contribute to achieve the project objectives.

From the monitoring we learn if the activities are producing the expected outputs, that is a way of finding out if people employed are doing the job they were asked to do. When we evaluate we want to see if the project deliverables are leading to the expected project outputs.

Are the products of activities contributing to solve the problems that have prompted us to undertake this project?
If not, why not?
Are the beneficiaries on board in the project g of decision making processes?
Was the plan well conceived?
Are there new factors taking place that require modifications in the plan?
Is the project contributing as intended to the wider programme to which it was a component?
Are we learning from the experiences?
What we are doing is it the right thing to do?

The importance of evaluation for learning what we have done and improve the quality of what we will do - Julian Parr

These judgment are possible as far as there is the possibility of comparing what is happening to what was expected to happen: therefore in order to carry out an evaluation we need at least two commonly visualized scenarios: the one that was expected (the program/project plan) and the one that is factual (the facts resulting from monitoring). When comparison highlights difference between expectations and results, evaluation will attempt to find the causes of such discrepancies, attributing them either to lack of capacity of the implementing team, or lack of vision of planner, or to unexpected external factors, or other factors as it may be the case. If the project is still running evaluations report will possibly indicate how to redesign the remaining actions in order to conciliate expectations and results. If the project is already completed, evaluations will indicate the lessons learned from the project to be considered in the successive planning and implementation of similar projects. Evaluations are therefore an indispensable means for programme cycle management and constitute an essential element of its actions.

Authenticity

Since evaluation is a highly technical job, in development cooperation actions specialized professionals are employed as consultants by sponsors and donors. The problem however is that they are economically dependent to the same stakeholders who would like to be justified by the evaluators. They approved plans, the spent money for them ... and they want to say that projects worked! Traditionally from the evaluation reports they want data to pass over to communication experts that will have to convince the public of the results obtained by the projects and the positive changes that occurred in the lives of the beneficiaries. Hence big use of rhetorical language, which however often leaves the public quite cool, because the public do not trust those who justify themselves.

Communication

If project sponsors really want to learn about the project results and if they want to establish a healthy communication climate with all stakeholders then the rethorical approach should be removed and the dialogue modality should be chosen, which is really in line with the spirit and the principles of development cooperation. The persons who are trusted by the stakeholders should be enabled to build informed opinion and freely express independent judgment. In this manner they can learn about the projects and they can share their learning with their public.

As it is becoming ever more evident that communication has no authority if its content is not developed in a process of reciprocal empowerment, then project donors are increasingly oping up to really independent evaluators and are asking independent media to come on board and express their unpiloted judgments.

Learning

Responsibility and challenges of mass media

While working for Eugad we learned that in order to adequately inform the public about development cooperation programmes it was not sufficient to take along in the field independent journalists and leave them free to express their views because in any case journalist tend to follow the conventional logic of the main media channels that tend to repeat used stereotypes and are mainly at the service of advisement sponsors rather than the public itself. (see the issue ⇒ Media and the international communication climate).
In order to bypass the media gatekeepers we decided to avail of the potentialities of Internet and start The Vrinda project where development actors share their views and their knowledge in order to inform each other and enable better informed decisions. The final aim of evaluation is to produce the learning that enables decisions based on informed judgments. We then learned that evaluations are really possible only if carried out in a dialogical communication climate.

The rhetorical modality by seeking consent (from followers) does not really generate new learning, even if carried out by highly skilled professional evaluators; the participated approach to evaluation fosters a dialogical modality of relationship amongst all development stakeholders and enables the articulation of learning setting the stage for increased cooperation and dialogue.