Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response
The recent special report on the 'leaked' DEC (Disasters Emergency Committee: http://www.dec.org.uk/) evaluation report that appeared on the BBC television's Newsnight (http://news.bbc.co.uk/2/hi/programmes/newsnight/default.stm), was not good news for public confidence in humanitarian agencies. Anyone who saw the programme was probably left with three main impressions. First, agencies had been greedy and had accepted much more money than they could spend properly. Second, in spite of the extra cash the job had not been done well and thousands of needy people were still living in temporary shelters. And third, and perhaps of most concern, agencies were not open to admitting their mistakes. Presenter Martha Kearney was in a particularly adversarial mood and Sir Nicholas Young, Chief Executive of the British Red Cross, found himself in a defensive mode which, at times, could have been perceived as evasive. All in all, not good profile for humanitarians.
So, what went wrong? By all accounts the evaluation report simply says what one would expect, painting as it does, a mixed picture of what went well and what did not go so well. It is no surprise to learn that replacing housing after a major natural disaster is difficult and that land rights are a particularly tricky issue. Equally it is common knowledge that the so-called 'flag flying' and 'competition' between agencies in the really big disasters does take place. We can find countless examples in similar evaluation reports in the ALNAP (Active Learning Network for Accountability and Performance in Humanitarian Action - http://www.alnap.org/) database. The focus of any constructive discussion around these issues should be how to deal with them more effectively, not about whether they happen or not.
So, why has old news suddenly become a mini-scandal? One of the reasons is to do with changing attitudes of the media to humanitarian agencies. Until relatively recently humanitarians could do no wrong in the eyes of the media. Stories from the 1980s of famines in Ethiopia and Sudan were about 'angels of mercy' feeding the hungry and generally saving the world. The Evening Standard even ran a front page piece on me as a 26 year old with a nutrition background going out to 'sort out Ethiopia's foods supplies' in 1984. We were all saints or saviours in those days. It was hard not to be.
Of course, the media had it wrong. All of us then were a lot less accountable than we are today. Evaluations were rare. We all made mistakes and we were not encouraged to learn from them. Gradually, the climate has changed and we now work in a culture of accountability where evaluation and learning is engrained in the system. And rightly so. The media has also changed in their attitudes and role. Whilst they do help instigate quick, effective responses to emergencies and provide valuable publicity and information, they also appear to find it hard to resist the temptation to drum up a bit of scandal. The John Vidal Guardian article about the Southern Africa crisis (UK charities exaggerated Africa crisis, 16 January, 2004: http://www.guardian.co.uk/uk_news/story/0,3604,1124355,00.html) may have been well received by his editor but I suspect the DEC trustees were less pleased to see selective quotes from their evaluations used as a big stick to bash the system.
Up until September 2004, all DEC reports were automatically published in full. This served to improve the quality of programmes and indeed DEC evaluations have always scored very highly in the ALNAP meta-evaluation and are cited as examples of evaluative best practice. It may be no coincidence that, after the Vidal incident, the DEC modified their policy and are now reluctant to put the full evaluation report in the public domain. Instead, the conclusions of the evaluation (and the formal response to them by the members) are published - but not the full report. There is nothing wrong with this except that if anyone gets hold of the unpublished material it becomes 'leaked' and as we saw on Wednesday evening leaked materials = scandal = good ratings = good TV.
Better accountability does require elements of trust between different stakeholders and it is worrying to see a hardening of the media attitude to humanitarian agencies. The consequence is that agencies are much more reluctant to 'wash their dirty linen' in public. Neither trend is welcome and not conducive to improving the system.
Comments
Comments on the ODI blog are moderated. ODI will post as many of your comments as possible but we cannot guarantee to publish them all.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Tuesday, January 17, 2006 4:28 PM
An interesting analogous matter also in the papers at the moment. The revelation of the fraud committed by Dr Woo Suk Hwang in claiming to have achieved scientific results in cloning is on the surface a story of failure – the failure of the scientific establishment to behave honorably, or at least according to the standards expected of science, and by implication an inability to ensure, monitor and if need be ‘police’ the implementation of those standards.
Yet the editor of The Lancet, Richard Horton, claimed in an article in the Guardian last week, that “the lesson to be drawn from the cloning fraud is that science has succeeded not failed”, in the sense that “[s]cientists have quickly rooted out a fabrication … a self-correction of which is to science’s credit, not shame”.
What the ‘scandal’ over the DEC evaluation has now overshadowed is that agencies actually do employ independent evaluators to bring to their attention their (relative) successes and failures. Such evaluations normally try to perform an uncomfortable balancing trick, using such observations and value-judgements about good practice and improvements to be made (known as ‘lessons’), together with making the agencies ‘accountable’. Such evaluations unsurprisingly vary in both quality and usefulness.
However humanitarian action is not a scientific enterprise, and the independence of an evaluator does not guarantee that the findings are accurate nor that recommendations made for improving practice will be well-founded, nor agreed by those who have the job of implementing them. What is necessary is that agencies consider what they are told about their performance and decide what to do about it. The implication in the current debacle for the DEC is that it is made to appear that agencies are not undertaking that consideration, or are certainly not willing to be visible in undertaking it.
While it is certainly unhelpful to have an adversarial media that is not, it appears, looking for balance in this reporting, agencies should also acknowledge the fact that they often get uncritical reporting from the media. What would be more helpful than the two extremes would be a profile in the media that helped agencies be accountable to the public by increasing understanding of the difficulties and the achievements and failings of the humanitarian enterprise, which would in turn be a more fertile environment for transparency and consideration of lessons.
‘Science’ may well be stronger for having had to root out the false claims of Woo Suk Hwang, but popular faith in the scientific enterprise is probably not strengthened by it, nor is popular understanding of that enterprise. And the role of the media in hyping scientific claims remains largely unexamined. This benefits neither the public in whose name the media claim to operate nor the scientists. I wonder if the same will hold as the fall-out of the BBC-DEC scenario, a lose-lose situation once the sexy glare of exposure has faded.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Monday, January 23, 2006 5:36 PM
In evaluation there is much talk of ‘lessons learned’ but one of the fundamental ones is that you should not hide facts. To come clean in the first place seldom gains much attention, but when you hold back information the media just love to print it. So, small stories and setbacks in humanitarian action, quite commonplace for all of us, get blown out of proportions. Why? Because disgruntled evaluators or civil servants might think that the public deserves better than half-cooked truisms telling how well the humanitarian agencies spend their money. Surely have agencies improved their operations the last couple of decades. But neither the public, nor evaluators expect them to be flawless, thus trying to make them look so is probably not a good idea.
Independent evaluators might take a more critical stance but the jury is still out on this issue, also independent evaluators depend on keeping a good relation with their employer and not gaining a reputation for being to gung-ho. In the end both ‘independent’ and internal evaluators present lists of ‘lessons (to be) learned’ which often seems to have disappeared when the next operation is implemented. International humanitarian assistance evolves in an arduous manner, and many setbacks are still to come. However, to actively construe a policy of concealment invites to distrust and is a very sad way of weakening the accountability of humanitarian action.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Monday, January 23, 2006 8:14 PM
In the various published evaluations of the tsunami response(s)and off-the-record discussions, a striking tendancy is too often that of self-congratulation and self-indulgence that most humaninitarian organisations have shown. At least, in UK there has been a go for publicised evaluations, which is not the case in France (where I am from)...
Whether the media in both countries are "turning against" the NGOs or not is maybe not the most important question ; media are doing media's job...
In France, the worrying point is very much laying with the fact that NGOs in general have played with the media for fund-raising purposes whilst they did not command the game. Apart from... well MSF alone I am afraid, the key points, issues and challenges behind all humanitarian crisis, be they political, economical.. or natural are not communicated upon, and actually often hidden on the ground that "the public cannot understand so complex dynamics"... This in turns contributes to the "angelisation" of NGOs untill such a crisis as the tsunami occurs. Most of the humanitarian organisations did not question whether they should intervene in Asia or not. They rushed to Asia partly for humanitarian reasons, partly pushed by the media pressure, and very little of them ever took the time to wonder what was going on, what was their added value, how they should proceed (or not) from there. Eventually, it was highly dubious that so many organisations could all do a great job.
As John was saying, NGOs actually gave the media the stick to be bashed eventually. Why ? Not because they are not “angels” anymore. Not because they are more accountable. Not because they are sometimes doing mistakes and provide “weak” responses. Most certainly because somewhere on the way, they may well have turned themselves into business practionners, fund-raisers, and may have forgotten to put the wellfare and wellbeing of the populations they aim to assist at the forefront.
Evaluations are key to – much needed – strong evolution and better positioning of NGOs, as long as they indeed accept to be questionned by their “beneficiaries”, by the public, by the media, and accept that their learning process is made public, rather than playing with the facts to show an ever-angelic face, that nobody really believes in anymore.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Tuesday, January 24, 2006 9:53 AM
John and Maurice - thanks for the thought-provoking posts here. It's good to see a well-thought out response to the Newsnight incident, which I watched with an increasing sense of horror. Maurice's point about needing a media that in future strikes the right balance between the uncritical past and the adversarial present is absolutely right, and echoes the argument of the Red Cross' World Disasters Report 2005.
While I agree with John that the public climate has changed, I'd like to suggest another, deeper cause behind the Newsnight incident. In this connected world, the revolution in global communication has changed everything - and, as a community, we have comprehensively failed to realise the potential that this brings for our work, or to identify the traps that this new world keeps in wait for us.
The Newsnight incident is another reminder that the patronising attitude that we adopt towards our public will always come back to haunt us. If the public does not have a clear understanding of the reality of relief operations, or of the improvements we have made in accountability in recent years, whose fault is that? Trapped in our 'traditional' fundraising strategies, we rely on the public to continue supporting us in emergencies without making much effort to engage them in the issues.
Fundraising for disasters still presents helpless victims waiting for international organisations to rescue them - even though we all acknowledge that the critical actors in emergency response are local people, communities and organisations. But the rush of communications that we have access to now - not just the mainstream media, but the 'citizen journalism' of blogging - means that our stakeholders on all sides have access to better information about our work than ever before - and more of a sense that their stake is an active one.
I felt that Sir Nicholas acquitted himself well in the face of persistent adversarial question. But Newsnight framed their coverage in a such a way that Sir Nicholas was unable to broaden the discussion beyond the very basic points that John identified in his post. Rather than seeing this as an example of how an antagonistic media doesn't play fair, perhaps we should take this as a wake-up call, a clear indicator that we need to improve our currently inadequate communications strategies?
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Wednesday, January 25, 2006 11:27 AM
Your interesting piece, and indeed the Newsnight interview, reminded me of the ballyhoo around an accurate but wholly inoffensive point made to a major newspaper by a DEC agency during the Tsunami response. The message was that it would probably be difficult to spend all the funds donated to that agency (ie many millions of pounds sterling) in the short term.
Now such a statement should hardly a shock to anyone with Homer Simpson's intelligence or above. Surely the media and British public can stomach this? Alas no - and its not all the media's fault.
The fact is that fundraising and communications departments of many major UK agencies have been spinning feelgood PR to the public and media for a decade or more. The gist has always been this - whilst the 'situation' and the needs of the vulnerable can be problematic, the agencies' performance and their ability to help (and spend) never is. Oh dear.
The cause of this was partly the imperative of corporate 'market share', and partly because many (but not all) agencies were just not bold enough to publicly raise, and engage with, complex but hugely relevant issues about the limitations of their work. These two factors were compounded by the fact that explaining the limits of an organisation's abilities to perform involves operational issues - and communications people didn't (and sometimes still don't) want to communicate problems, difficulties and complexities to supporters and donors.
So whilst I completely concur with your piece, I would also point to the historical spinelessness of the agencies themselves, for ducking hard issues such as operational capacities and over-funding, and doing little to counter the public impression that they were, indeed, angels of mercy. Like you, I was involved in many media interviews. I was explicitly told on several occasions to never talk about operational limitations or difficulties in the UK, for fear of "effecting confidence levels".
The reasonable thing to do when challenged by a pugnacious interviewer looking for a scandal might be to ask if anyone else out there would like to try and spend £50 million in 12 months along an often under-developed 15km strip over a thousand miles long. Let alone when there's another £1 billion also out there. Ask your average builder from Peckham or Barnsley. What would he say? "No bloody way, mate. It'll take years. And you'd better watch you don't get ripped off when you're out there." Exactly. But do we couch it in such reasonable and understandable terms? Rarely. We give the impression we can always 'do the job'. So the interviewer will, understandably, make hay on the (rare) occasions when its overwhelmingly obvious that we can't.
Or to put it another way - if DEC agencies had been historically more open and transparent about their usually very valid and unavoidable operational capacities and limitations, then Newsnight would have less of a public impression of infallibility to work with.
I hope and believe this is changing. I know that some DEC agencies have made very conscious attempts to address over-funding, such as calling time-limited appeals.
But given extreme cases such as the Tsunami, even the building of cohesion in appeal-making and liaison through bodies such as the DEC needs furthering. A more robust attempt to take their operational realities to the public would yield future dividends. It might not be slick or pretty in marketing terms, but it would be truthful. Which is the key to long term public support.
Such an approach would not cause mass donor panic and a fall in giving. What it probably would do is create a more educated donor base that is willing to adapt its giving in accordance with humanitarian needs globally as well as with operational realities on the ground. Which, to use the corporate thinking so prevalent in the UK charity sector, is only good 'investor sense'.
By the way, when the shock horror admission of the aforesaid DEC agency's difficulties in spending squillions of pounds by the weekend was published, all its big guns came in to man the barricades. But there was little need. Its public supporters, some of its trusts and corporate donors,and even other agencies,wrote in and expressed gratitude for this rare *** of realism, honesty and openness, and many happily offered to repledge their money to other appeals that well less well covered - such as that for Sudan. Perhaps the giving public aren't as undiscerning as we've been led to believe all these years.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Wednesday, January 25, 2006 2:02 PM
I read with interest this contribution and the response to it. Yes, publicly admitting/advertising deficiencies and shortcoming is potentially harming the ‘industry” in the short run but it is in the best interests of beneficiaries and responders alike in the long run. There are too many shoddy operators in the game. Re the comment from Maurice Herson, a cover-up on the fraud in cloning would have only compounded the negative impact on the public! Transparency is the best
and safest policy.
I believe that tough but honest criticism (in the opinion of the TEC evaluators is necessary in all TEC (Tsunami Evaluation Coalition) evaluations. We should not stick to a policy of pretending we were all doing the greatest humanitarian work and that the response to the tsunami was “effective” i.e. doing a lot of good. With the resources mobilized, it is the least we could expect! A consensus on this point needs to be reached in the next consultation among the TEC evaluation teams. Problem is also lying with the donors policies and pressures and should be reflected in relevant TEC evaluations.
> Claude de Ville de Goyet, MD. Consultant Humanitarian affairs and disaster reduction
> cdevill@attglobal.net
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Wednesday, January 25, 2006 2:08 PM
"Effective" is not the same as "doing a lot of good", it simply means "able to accomplish a purpose." In the case of the initial response to the Tsunami the purpose of the initial response was to limit further death (a low risk, but one that was greatly overplayed by the agencies, including WHO), and suffering. I believe that the response largely did this, and that, given the huge flow of resources it would have been difficult to have been ineffective..
The following reflects my view of the other standard evaluation criteria (both development and relief):
Efficiency - Not efficient as lots of duplication and waste.
Impact - Definitely had an impact in the short term - longer term impact is much more questionable.
Relevance - Probably broadly relevant (with some exceptions - see appropriate below). livelihood responses in particular were sometimes of questionable relevance.
Sustainability - The level of expenditure and agency focus on the tsunami affected area is clearly not sustainable.
Appropriateness - Many examples of inappropriate response (field hospitals, second-hand clothing, tinned fish etc.). Many, but not all of these were related to new entrants in the aid system. Lack of proportionality was inappropriate, but this is dealt with under coverage below.
Coverage - Big issues around proportionality (conflict IDPs vs Tsunami IDPs, allocations between countries, and within countries, tsunami affected countries versus other disasters). Better off got most assistance. Some groups were overlooked or got less than others.
Connectedness - Despite an effort by the aid agencies, the response is far less connected to the long term context than it should be (lack of conflict-awareness, gender sensitivity, attention to development issues).
Coherence - Difficult to say how coherent the response is with longer term interventions.
Coordination - Not at all coordinated.
You may have seen that Hilary Benn is due to make a speech critical of the humanitarian response system to the UN today:
http://politics.guardian.co.uk/development/story/0,,1693046,00.html
This seems to be part of a growing level of criticism of the overall system.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Friday, January 27, 2006 1:21 AM
The broader problem is that the humanitarian agencies want it both ways when it comes to press relations. They delegate needs assessment to the press in the early stages of a disaster, and then expect the press to ignore how the money they stimulate is spent.
Too often humanitarian organizations put public relations officers on the ground (and on television) even before a political or technical assessments are possible. And guess what, putting press officers in the field quickly works for getting money. Home governments and the publics they respond to support respond more generously to disasters that they are personally aware.
The problem is that this means the more systematic needs assessments are skipped as agencies rush to meet the demands of press deadlines. Implicitly the longer-term deadlines needed to deal most effectively with disaster victims are pushed to the side.
In this way, in the long run, agencies become accountable to the press in the long run, rather than the program, governments, or victims. The BBC report described here illustrates this principle. In effect, the current structure of humanitarian organizations delegates both needs assessment, and evaluation to reporters whose expertise is not (and should not be) humanitarian response. Improving press relations and educating the public helps a little with this, but does not ultimately address the core problem, which is reliance on the press response humanitarian funding. Things will remain this way as long as preference for funding of emergency response goes to those who shout loudest. Until then, this will be as good as things get!
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Friday, January 27, 2006 4:30 AM
For many of the humanitarians who worked thru the Tsunami response, this is not new and everybody is aware of it. How little agencies did (compared to the resources they mobilised and talked about principles) and what excuses they invented to justify this inaction is known to everyone. Endless discussions, countless number of drafts of 'straegies', 'frameworks', 'policies' was the norm. What is most ridiculous and outright arrogance on the part of these agencies is that they choose to fly the flags of so called principles and standards and yet do precious little to comply them in the first place. Many of the agencies immediately assume the moral high ground to criticise the Government policies and practices etc. Ridiculous.
Every disaster throws similar challenges and 'learnings' (sic) and yet there is very ittle that is actually retained and carried forward in spirit.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Friday, January 27, 2006 11:14 AM
Hi John (Mitchell),
My only reservation is that you seem to think it OK to publish the full evaluation. Would that not be a very retrograde step?
Tony Vaux, Independent consultant and evaluator.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Friday, January 27, 2006 11:21 AM
Hi Tony,
My point was that it is acceptable for an agency to publish their own response to the findings and recommendations from an independent evaluation. Our work in ALNAP shows that some evaluations are of poor quality and do not always represent a balanced view. It seems only right that there should be a right to reply. Sometimes findings from evaluations need to be challenged.
Having said that, my own personal view is that it is more transparent, and thus preferable, to publish the full report as well.
John Mitchell.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Saturday, January 28, 2006 11:47 PM
That interview had me worried. But shouldn’t we welcome Newsnight’s probing? It’s high time that serious media engaged critically with humanitarian action, beyond proxy fundraising and unquestioning support.
At best, the media helps to make powerful organizations accountable to the public. And aid-gone-wrong will usually make a good story: As new journalists learn, ‘dog bites man’ is not news, but ‘man bites dog’ (or ‘aid does harm’) – that’s news!
Remember also how the media helped to sustain the exceptional Tsunami response. Coverage was of high quality, an improvement on the standard clichés of disaster reporting; it showed how survivors coped and how aid workers struggled.
So what can aid agencies do if media attitudes are hardening? First, they could take their public accountability role seriously. Of course they address the public as potential supporters, but they must balance that with their responsibility to communicate the truth as they know it about the situation and challenges of responding. Learning and communications shouldn’t be going in different directions.
Second, humanitarian agencies could think about improving their media relations and communications function with that in mind. As it is, several studies have shown that many agencies don’t work very effectively with the media.
Third, agencies could learn from politicians. Credibility is hard to win and easy to lose: tell one lie and we’ll never believe you again. Openness and debate are ways to raise awareness and get people involved. Otherwise, shouldn't we expect greater manipulation of public humanitarian sentiment?
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Monday, January 30, 2006 12:22 PM
It's good to see the lively discussion triggered by the DEC leak.Hopefully the events will encourage humanitarian agencies not to take the risk of sitting on their reports, but I wouldn't bet on it.
I should declare an interest as a humanitarian evaluator. I was a peer reviewer on the DEC Tsunami evaluation and also led the DEC Kosovo evaluation.
The Newsnight report picked up the bad news on the Tsunami response, but did not clearly enough focus on the question of DEC accountability.
In particular, why is the DEC backing out of its earlier good practice of publishing all its evaluations in full? Apart from the Tsunami report, there are a dwindling number of earlier evaluations on the DEC website.
There is also a question about why fewer resources were committed to the DEC Tsunami evaluation than, say, to the Kosovo evaluation, the latter being the largest to date for the Tsunami. The DEC Kosovo evaluation had more human resources and more time and yet the Tsunami appeal is 6 or 7 times more in financial terms than the Kosovo appeal.
There is also the fact that the DEC Tsunami evaluation shared in common with many other Tsunami evaluations including the TEC ones, a ridiculously short timetable to meet the "deadline" of the first anniversary. Evaluations were planned late and evaluators had to rush to cover the ground, hardly good practice.
When the dust dies down on the Tsunami and the tsunami of evaluations, there will be need for reflection. Agency field staff were inundated with evaluation visits, in spite of the fact that the TEC evaluations were meant to reduce the overall number.It doesn't look at this stage as though the tsunami of evaluations is going to be a model of good practice, but I hope I'm wrong.
Why we need automatically published independent evaluations @ Tuesday, January 31, 2006 10:31 AM
In the “DEC sets the record straight” statement of 16 January, the DEC justifies the slow progress on shelter reconstruction with the following claim:
“The tsunami reconstruction and rehabilitation is the equivalent to rebuilding both Birmingham and Glasgow from scratch.”
(at
http://www.dec.org.uk/uploads/documents/Web_statement_16_January2.pdf)
While the combined populations of Glasgow and Birmingham is about 1.6million (for the council areas only - 2001 census – taking Greater Glasgow and Greater Birmingham would bring the total to over 2 million) is roughly on a par with the number displaced by the Tsunami (1.8 million), there is absolutely no comparison between the level of infrastructure between the affected countries and Birmingham or Glasgow.
The level of infrastructure in a country is related to the investment over the previous decades, and this depends on the income available for such investment. The current Gross National Income (GNI) provides some comparison of the relative economic might of different counties. The sum of the products of the GNI per capita by the number of the displaced for the affected counties is 1.6 trillion person dollars. This is less than 2% of the equivalent calculation for Birmingham and Glasgow, and even this understates the difference as the GNI of the affected counties is growing far faster than that of the UK, and parts of the affected areas were relatively undeveloped even in their national context.
Even if you think of housing alone, consider the cost of a house or flat in Birmingham against the £3,000 to £6,000 price sticker for a house in Aceh. An equivalent population in the UK needs a lot more houses because there are so many people living on their own rather than in families. Rebuilding Glasgow and Birmingham from scratch would take at least 100 times the cost of repairing the Tsunami damage.
It is grossly misleading to equate “tsunami reconstruction and rehabilitation” with the task of rebuilding both Birmingham and Glasgow. The fact that agencies peddle this sort of misleading twaddle to the public is the reason why independent evaluation is needed, and why such evaluations need to be published. The lack of transparency is illustrated by the fact that only the revised short report is available even now on the DEC website, even though the unrevised full report is available on the BBC website.
The revisions to the DEC evaluation report have led to suggestions that the DEC evaluations are not independent.
However, as someone who has carried out evaluations for the DEC, I would argue that in the past the evaluations have been independent as the evaluators had the assurance that their reports would be published. I understand that for the DEC Tsunami Evaluation the evaluation team were told that the decision to publish would rest with the DEC board. This does not promote independence as evaluation teams are then forced to consider how to balance objectivity with the desire to avoid being so critical that the report will not be published.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Thursday, February 02, 2006 10:44 PM
Dear All
Many thanks for your comments which are much fruit for thought, particularly John and Maurice for initiating this discussion process.
My humble thoughts following what has been topical and the various aspects and concerns that have been expressed are:
It is interesting that within the Good Humanitarian Donorship Practice framework as an initial strating point one is able to initiate a consultative process with donors, NGOs and local NGOs for teasing out ways to encourage various agencies all the way down the aid system to strive for institutionalizing greater Transparency as one compnent of Accountability. Indeed we live in world of multiple accountailities with an upward orientation in particular. So assessment exercises time and time again continue ot have a greater value for reporting rather than meaningful organizational learning.
One transparency specifically should we not strive for greater efforts to convince donors to provide a conducive policy environment that favours, indeed giving incentives rewards greater transparency. These incentives could be categorized into two forms: Enabling the aid recipient agency to realize and commit to Developmental goals and for Organizational Learning purposes;
Secondly, tangible incentives in forms of withdrawl of the unacknowledged latent threats and sensativity that aid recipient agencies are often observed feel reflected in their postures.
These are issues ofcourse require greater consideration of as to the range of processes and mechanisms through which an enabling culture of Humanitarian practice and assessment takes place.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Thursday, February 23, 2006 11:03 AM
Sorry, all
My comment of 27/1, posted on my behalf after an email correspondence, came out with the opposite sense to what I intended. My concern was that the outcome of the Newsnight interview may be to discourage the DEC from publishing its reports. I wanted to refer to this as a retrograde step. The DEC has up to now been a model of transparency; I fear that it would be a shame to lose that. Sorry not to have picked up on this earlier.
re: Have I Got Old News for you? The 'leaked' DEC evaluation of the tsunami response @ Saturday, March 04, 2006 4:18 PM
The evaluation of disaster relief, like that of international development efforts, has been a scandal waiting to break, and the leaked DEC report should be treated as potentially a blessing. The key question is whether the hum'n agencies react by increasing the extent, the externality, the frequency, and the public availability accorded to evaluation of their work; or by decreasing all four, as they show signs of doing, with too much sympathy from the discussants here. This is just a replay of the Red Cross and FEMA scandals, and a deplorable sign of the self-protection that is so common in areas where the current players think they own the moral high ground, and that critics are the enemies of righteousness. From the point of view of a well-trained evaluator, the standards currently employed in the disaster and development areas are so chummy that what they call an "external evaluation" is a bad joke to good evaluators, right in there with having your brother recommend you for a job. Time for a change, and good for the BBC (and DEC)!
Michael Scriven
Newsnight revisited @ Wednesday, December 20, 2006 2:05 PM
Did anyone see Newsnight's latest report on the tsunami reconstruction last night, available here:
http://news.bbc.co.uk/1/hi/programmes/newsnight/6193737.stm?
They're ending the year the same way they started it, criticising everyone in sight for the situation: "Two years on, billions of pounds available and people still living in tents".
Which would seem to suggest that the constraining factor here is not money. The report had an interesting comment from Gareth Thomas, a DFID minister, that it's up to national governments to hold aid agencies to account. Building tens or hundreds of thousands of houses anywhere would necessarily take a lot of careful political negotiation - about who owns land and who buys it, and where houses should be built, and who should build them, and how, and many other complicated and delicate questions.
A UN official commented that he was an architect, and he knows how long it takes to build a house. But building one house is surely a very different project to re-building many thousands of them. Particularly when the tsunami response is made much more complicated again by infrastructure problems.
Could agencies consider how to reinforce the message that money is not the only, or always the main, limiting factor in their work? Nobody can say with certainty "give us money and we will solve other people's problems" - we know that it always comes down to local politics of one sort or another. Is Newsnight justifiably hoisting agencies on their own petard?
