Global Policy Forum

By Owen Bennett-Jones

The British government was advised against publicly criticising a report estimating that 655,000 Iraqis had died due to the war, the BBC has learnt.

Iraqi Health Ministry figures put the toll at less than 10% of the total in the survey, published in the Lancet. But the Ministry of Defence's chief scientific adviser said the survey's methods were "close to best practice" and the study design was "robust". Another expert agreed the method was "tried and tested".

Mortality rates

The Iraq government asks the country's hospitals to report the number of victims of terrorism or military action. Critics say the system was not started until well after the invasion and requires over-pressed hospital staff not only to report daily, but also to distinguish between victims of terrorism and of crime.

The Lancet medical journal published its peer-reviewed survey last October. It was conducted by the John Hopkins School of Public Health and compared mortality rates before and after the invasion by surveying 47 randomly chosen areas across 16 provinces in Iraq. The researchers spoke to nearly 1,850 families, comprising more than 12,800 people. In nearly 92% of cases family members produced death certificates to support their answers. The survey estimated that 601,000 deaths were the result of violence, mostly gunfire.

Shortly after the publication of the survey in October last year Tony Blair's official spokesperson said the Lancet's figure was not anywhere near accurate. He said the survey had used an extrapolation technique, from a relatively small sample from an area of Iraq that was not representative of the country as a whole. President Bush said: "I don't consider it a credible report."

But a memo by the MoD's Chief Scientific Adviser, Sir Roy Anderson, on 13 October, states: "The study design is robust and employs methods that are regarded as close to "best practice" in this area, given the difficulties of data collection and verification in the present circumstances in Iraq."

'Cannot be rubbished'

One of the documents just released by the Foreign Office is an e-mail in which an official asks about the Lancet report: "Are we really sure the report is likely to be right? That is certainly what the brief implies." The reply from another official is: "We do not accept the figures quoted in the Lancet survey as accurate. " In the same e-mail the official later writes: "However, the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones."

Asked how the government can accept the Lancet's methodology but reject its findings, the government has issued a written statement in which it said: "The methodology has been used in other conflict situations, notably the Democratic republic of Congo. "However, the Lancet figures are much higher than statistics from other sources, which only goes to show how estimates can vary enormously according to the method of collection. "There is considerable debate amongst the scientific community over the accuracy of the figures."

'Mainstreet bias'

In fact some of the British government criticism of the Lancet report post-dated Sir Roy's comments. Speaking six days after Sir Roy praised the study's methods, British foreign office minister Lord Triesman said: "The way in which data are extrapolated from samples to a general outcome is a matter of deep concern...."

Some scientists have subsequently challenged the validity of the Lancet study. Questions have been asked about the survey techniques and the possibility of "mainstreet bias".

Dr Michael Spagat of Royal Holloway London University says that most of those questioned lived on streets more likely than average to witness attacks: "It would appear they were only able to sample a small sliver of the country," he said. Dr Spagat has previously conducted research with Iraq Body Count, an NGO that counts deaths on the basis of media reports and which has produced estimates far lower than those published in the Lancet.

If the Lancet survey is right, then 2.5% of the Iraqi population - an average of more than 500 people a day - have been killed since the start of the war. The BBC World Service made a Freedom of Information Request on 28 November 2006. The information was released on 14 March 2007.