What case for change?

"I am a passionate supporter of our NHS, and that is why I understand the passionate debate it arouses. It is also why I resent those Opposition Members who seek to misrepresent the NHS, its current achievements and its future needs"

- Andrew Lansley, Wednesday 28 February 2012

"For example, if our cancer survival rates were at the European average, we know we would save 5,000 extra lives a year"

- Andrew Lansley, Wednesday 1 June 2011

"The cost of new medicines alone, for example, has been rising by nearly £600m a year"

- Andrew Lansley, Sunday 10 April 2011

Sooooo, my dual approach of tweeting and emailing has paid dividends. Well, to be more accurate the countless tweets have so far yielded nothing (thanks very much to those who re-tweeted, by the way), but a couple of emails to healthandsocialcarebill@dh.gsi.gov.uk resulted in a response after eight days (not too shabby at all).

Highlights of the response I received are below:

"Many thanks for your emails dated 20 and 27 February in which you asked to be provided with the sources for two lines from the Overview of the Bill factsheet, published on the Department of Health website
Saving 5,000 additional cancer patients’ lives each year by bringing England in line with the European average is based on the analysis of number of deaths that could have been avoided if England’s five-year survival rates matched the average European five-year survival rates
Further information about the study can be found on page 33 of Improving Outcomes: A Strategy for Cancer- First Annual Report 2011 on the Department's website at 
http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_131690"

And yes, you guessed it, bottom of page 33 of that document says:

Abdel-Rahman et al, What if cancer survival in Britain were the same as in Europe: how many deaths are avoidable?, Br J Cancer. 2009 Dec 3; 101 Suppl 2:S115-24 www.ncbi.nlm.nih.gov/pubmed/19956155

I've since replied with the below:

"Thanks very much for your reply.

I'm incredibly interested as to why the 5,000 deaths figure is still being used, based on Abdul-Rahman et al's study, despite it being comprehensively discredited:

http://www.guardian.co.uk/commentisfree/2011/apr/16/bad-science-goldacre-nhs-statistics 

Please can you clarify whether you disagree with Dr Goldacre's analysis, and why it is deemed appropriate to continue using the figure a year on, as if it were an accurate, contemporary 'fact' (note the use of the phrase "we would save", rather than "a study based on data no later than the year 1999 estimated that")"

So far, so predictable. However, that was only one half of the response that I received. When I emailed on the 27th of February, I also requested the source for the claim that the cost of medicines is rising by over £600m per year. Here's that part of the response:

"The estimate of £600m is based on the average growth in total drugs expenditure over ten years. The table below shows the total NHS drugs expenditure from 2000-01 to 2009-10 and draws on information contained in the Department of Health Annual Accounts, Financial Returns, and Foundation Trust year-end accounts during that period. 

 

My reply concerning this was:

"And on the cost of medicines, please can you provide the relevant links to the information that forms the basis of that table. Apologies, but I can't find it in here:

http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_130154.pdf"

Right, now for the interesting (ahem) bit... let's have a closer look at that table above.

Well, it covers the first 10 years of the 21st century. And it states that the cost of medicine in 2000/01 was £6,690m. Now, if as has been claimed by Factsheet A1 the cost of medicines is growing by over £600m per year, then the cost of medicines in 2009/10 must be at the very least more than £12,090m (£6,690m + 9 x £600m)... agreed? (embarrassingly I had that initially as 10 x £600m, but hopefully now have it right)

Hmmm, how strange. The bottom line of the table states that the cost of medicine in 2009/10 was £11,920m. That's more than £170m less than the absolute minimum figure necessary for the 'fact' to be correct.

So it's wrong. The 'fact' on Factsheet A1 is simply incorrect. Now, anyone who read my previous post on this (my wife, possibly rightly, thinks I'm getting obsessed, but there is a reason for my persistence... I'll get on to that at the end of this post) will know that not only does the £600m per year 'fact' get an airing on Factsheet A1 ('Overview of the Bill'), it also makes a repeat appearance on Factsheet A2 ('Case for change') in a slightly different guise. Such is the importance placed on this 'fact'!

Factsheet A2 states that the cost of medicine has been growing on average by nearly £600m a year. 'Nearly' is nice and vague... I might have to concede that 'nearly' in the context of the figures involved could cover a shortfall in excess of £170m.

But now I'm wondering why the last 10 years has been included, and why go for the average over that decade?

Let's have a look at that table again...

From 2001/02 to 2004/05 inclusive, the annual cost of medicine did rise by over £600m - the highest rise was in 2003/04, when medicines cost £916m more than in 2002/03. In 2005/06, there is what looks like an anomaly... possibly a data error, but also possibly down to an external factor? I don't know, and I await the raw data so I can dig into it, but according to the data supplied, the cost of medicine in 2005/06 was only £28m higher than the cost of medicine in 2004/05.

Now, here's the crucial bit. From 2006/07 to 2009/10 inclusive, the annual cost of medicine rose by less than £600m - the highest rise was in 2006/07 (the year after the likely data anomaly), when medicines cost £563m more than in 2005/06.

Furthermore, the percentage year on year increase in the cost of medicine is consistently lower in the most recent years.

And if you take 2004/05 as the starting point, rather than 2000/01, the average annual increase in the cost of medicine suddenly becomes £390m - bit of a difference from £600m, right? And the £390m is using the most recent five years of available data.



Now, to try and explain why I keep banging on and on about all this... it's because these are the 'facts' being repeatedly used to justify a monumental shake-up to the NHS. Of all the things that could have been chosen, these are what the Government view as the best possible justifications.

A study of cancer patients from the last century, which pre-dates the NHS Cancer Plan, isn't a compelling case for change. Nor is the rising cost of medicine, which is rising consistently lower in recent years than it was.

Please don't accept these 'facts' as justification for the Health and Social Care Bill. They're both incorrect and misleading.

If you want to learn about the performance of the NHS, take a look at this. Highlights include:

- Levels of public confidence and satisfaction in the NHS higher than any other country studied
- UK has one of the least expensive health systems among countries studied

And please don't be fooled into thinking that dismissing the Government's flimsy and erroneous case for change is akin to saying that the NHS is perfect and never needs to improve or adapt.

What worries me most is that if we're being misled about the reasons for the Health and Social Care Bill being introduced, what does that say about the actual content of the Bill and how it will be implemented? For me, it's like cowboy builders turning up at your house and telling you that your roof needs urgent repairs. If you get someone else to go up on your roof, see that the type of work they claimed needed doing so urgently is actually a con job, then how confident would you be in letting them go ahead and do the work? Do you think they'd do a good job?

Chris



P.S. If I've stuffed up any of the medicine cost analysis, I do apologise, and would be more than happy to correct.

P.P.S. Quick final one for nostalgia's sake - anyone remember "Someone in this country is twice as likely to die from a heart attack as someone in France"? Compare and contrast that often quoted justification for changes to the NHS with this from the Information Centre today...

The NHS IC‏ @NHSIC

Following

 

Heart attack patients: emergency admissions drop while death rate nearly halves in a decade:tinyurl.com/7nzl7yl

• 11RETWEETS
• 

11:09 AM - 29 Feb 12 via web · Embed this Tweet

•  Reply 
•  Classic RT 
•  Retweet 
•  Favorite

How do you kill a zombie statistic?

Zombies, eh. They're all the rage. But there does at least seem to be pretty widespread consensus on how to kill them. Don't aim for the heart, you fool! As anyone who has played first person shooters knows, it's all about head shots.

Sadly, zombie statistics also seem to be all the rage. And I have absolutely no idea how to go about killing them. Oh, and if anyone thinks that zombie statistics aren't dangerous, think again! They'll eat your brain if you let them. And often, you won't even realise that they're doing it.

It's either a testament to my tedium or their resilience that I am going to write a fourth (count them - one, two, three and now number four!) blog post that includes the line "5,000 lives a year from cancer".

To borrow the terminology of the day, courtesy of The Walking Dead (you know, the one with the man-child Egg from This Life playing the husband of the wooden Dr Sara Tancredi from Prison Break), the cancer deaths 'walker' is still very much with us.

But first, a bit of wider context. Just this morning, BBC news tweeted the below:

BBC Radio 5 live poll suggests 80% of people in England don't think government's done enough to explain NHS changes http://bbc.in/xMspjN

First of all, I'm delighted to see the word 'changes' being used, rather than the usual 'reforms' or 'overhaul'. Orwellian language, innit. You reform or overhaul something because it's either plain bad or you're patently making it better. We are years away from knowing whether these changes will make anything better (and that's if we will ever know... don't laugh, it certainly wouldn't be the first time that the NHS train set had been played with by the Fat Controller of the day without any way of knowing whether it made things better or worse). For now they are just changes. Secondly, four out of five people - wow, that's a lot. The Government seem to be in a bit of a pickle, but is it fair to say that the Government hasn't done enough explaining?

Well, let's see. Yesterday we had the rapidly convened 'summit' meeting of... er, well, anyone who hasn't rocked the boat thus far really. Here is a relevant, ahem, infographic / data visualisation. And just to be crystal clear, this is definitely not what happened during the hour long meeting at Number 10. However, this did happen to the Secretary of State for Health on his way to the meeting.

So not such a good start to the week... ah well, can't win 'em all. And looky over here! What do people want? An explanation of the Health and Social Care Bill. Well, well, well, all ye doubters, look what the Department of Health bestowed upon us just last Friday - Health and Social Care Bill explained. You see, they are listening!

Not only are they listening, they've also produced fact sheets to help explain. Sheets... full of facts! Facts are good. Facts will reassure us. We love facts! Go to work on a fact. Get on your fact. We want facts, and we won't wait. In facts we trust. Keep facts and carry on.

Any Bill that requires 19 separate fact sheets has to probably have some sort of question mark hanging over it, but let's gloss over that for the moment.

Bring on the lovely facts on those lovely sheets!

First up - Factsheet 1. Nice one, just the one page. Maybe I am being unfair having a little moan about there being 19 of them. Short and no doubt sweet. Well, short anyway. On this first sheet of facts, there are in fact only two discernible 'facts' (quotes required, trust me).

The first 'fact' is that the cost of medicines is growing by over £600m per year (for information, over here are 10 facts on the cost of medicines, giving some very interesting comparisons and context). To date, and despite the lack of a primary source to check the figure against, I've just kind of accepted it when I've heard it in a kinda yeah, sounds about right, bound to be getting more expensive, grrr big pharma type way. But actually, I have no idea if it is right. SOURCE PLEASE! And I don't mean ketchup (although I do love ketchup). And this seems to cast a bit of doubt on it straight away. So is it the 'cost of medicines' or the 'cost of advances in treatments and medicines' that is growing by 'over £600m per year' or 'around £600m... every year'? COUGH UP THE SOURCE AND WE'LL CHECK IT!

The second 'fact' is our infamous walker. Somebody please aim for the head and take it down once and for all. Here are the immortal words: 'If we had cancer survival rates at the average in in Europe, we would save 5,000 lives a year'. I really don't need to say any more. This post is littered with links thoroughly debunking that claim. And the debunking started about a year ago!! And the problem now is that, faced with a Government that is happy to keep peddling a 'fact' that was outed as dodgy as Free Piece Sweet, I simply don't trust the other 'facts' that are being used in relation to the Health and Social Care Bill. Unfair? I don't think so. They've been Goldacre'd. They must know they've been Goldacre'd. And yet, they keep peddling and carry on. What they should do is apologise, of course. But that's about as unlikely as a politician answering a straight question with a yes or no answer. Failing an apology, they should at the very least stop peddling it. But they haven't. They have therefore lost my trust, and I am now skeptical of their other 'facts'.

Sooo, to sum up Factsheet 1. Until a source is supplied regarding the year on year increase in the cost of medicines, it does not contain any actual facts. That is a pretty special achievement for a fact sheet. Take a bow, Factsheet 1.

Skimming through what I have written so far, I realise that if I go through all 19 fact sheets, this post is going to get longer than either I want to write or you want to read. So let's stop at two.

Second up, and finally (for the time being at least) - Factsheet 2. This is entitled 'The case for change', so big, important stuff must be on it. It's another one pager - absolutely nothing wrong with brevity (note to self). And it opens up with a quote from Paul Corrigan.

Read it for yourself and make your own mind up, but to me it's a bit of an odd, obscure quote to choose. Don't get me wrong, Paul's often very quotable, and I often quote him, but it's just that this particular quote isn't the kind of punchy, hit you between the eyes, this is why we need this Health and Social Care Bill in a nutshell quote that you'd think the Government would be looking for. And don't forget, they could have chosen any quote from anyone over the last year and a half. Paul also happens to write a very interesting, very good, very regular (note to self) blog. I would strongly encourage you to read some of his posts, because he isn't always... how can I put this... hmmmm, the case for change for this particular Health and Social Care Bill personified. Paul is now aware that he has been quoted on the second sheet of facts, and will be responding on his blog tomorrow (Wednesday 22 February). I have absolutely no inkling what he might say (he might be both delighted and flattered), but I shall certainly be reading his response!

So on to the 'facts' on Factsheet 2. Well, we've got the cost of medicines. Again. Let's move on. Actually, hold on, more doubt cast! Factsheet 2 doesn't even agree with Factsheet 1. Engage pedantry, level 3.

'The cost of medicines is growing by over £600m per year' - Factsheet 1

'The cost of medicines has been growing on average by nearly £600 million a year' - Factsheet 2

Small beer? Yeah, probably. But I'm not filled with confidence in a suite of fact sheets that aren't even consistent between themselves.

The next 'fact' we have is another one that I've never thought to query before now. On the same basis again, that it sounds plausible, but I'm in full on skeptic mode now. Obesity costs the NHS £4bn per year, and will cost £6.3bn per year within the next four years. SOURCE PLEASE! Seriously, it is so, so, so, so easy to include the source. Just a little superscript number, and a list at the bottom of the page. Here's an article about obesity, which talks about £1.9-£2bn per year medical costs being added by 2030, and £5.5bn being added by 2050. But certainly no reference to £2.3bn being added by 2015/16. Here's the Department of Health quoting the £4.2bn per year figure, but again, no mention of it being £6.3bn within four years. Also, please do note the Department describing the £4.2bn figure as an estimate! COUGH UP THE SOURCE AND WE'LL CHECK IT!

The final 'fact' I'll look at is from a 2011 Royal College of Surgeons report. Of course, Factsheet 2 doesn't include a link to the source, but I will. It's here. It's a very interesting report with some very practical recommendations, but seriously, I can find no concrete reference for the 'thousands of lives could be saved if  patients had better access to facilities such as x-rays, scanners and operating rooms, and better post-operative care'. If I've missed it, apologies. And I would be delighted if someone could enlighten me. But I want a specific reference to thousands of lives being saved, remember.

Right, so as per usual, this post is both longer and less focused than it should be. The title suggests that it should be just about the cancer deaths myth being wielded out again (somewhat deliciously, our PM just yesterday said it's time to do some myth busting - lock and load, C-man. Aim for the head! Take that walker down), but I've wandered off, around and all over the shop.

My excuse - there's a lot going on with the Health and Social Care Bill right now. And a lot of what's going on ain't good. All I ask as a starting point is to play fair and straight with facts and figures. Some NHS statistics are very complex and can be employed by both sides in an argument - waiting lists are a classic example. See here. But making up 'facts' to justify huge changes to the NHS is just not on.

I would end by saying, now where's my double barrelled shotgun, but that might get me into Robin Hood airport type trouble.

Chris

Once, twice, three times a headline

Hello, is it an accurate headline you're looking for?

On the afternoon of Friday 12 August, the BBC tweeted the below:

BBCNews BBC News 

A&E waiting times nearly double http://bbc.in/qaBGhr

12 Aug 

I could pretend that it made me think, wow - that's shocking news! But because it is just way, way, way too far fetched to be remotely believable, it immediately made me think, oh dear, somebody's made a major gaffe.

The previous day (Thursday 11 August), the Department of Health had released the latest A&E performance data, covering Quarter 1 of the 2011/12 financial year (1 April 2011 to 30 June 2011).

The previous Government's target was for a four-hour maximum wait in A&E from arrival to admission, transfer or discharge. In practice, this used to be measured by using a 98% threshold - in other words, an NHS trust was deemed to have achieved the target if 98% or more of people attending A&E waited less than four hours.

With the change in Government, the 'target' remained. Ish. 'Target' is now a dirty word and a big no,no - it smacks of the previous regime. So the 'target' became a 'standard', and the threshold for achievement was relaxed from 98% to 95% in June 2010.

Now that we're in 2011, and a whole new financial year - the first full financial year since the change in Government - we have the Department of Health further distancing themselves from the culture of targets.... by.... well...... by using much more creative language.

We don't have targets any more.

We don't even have standards.

What we have is IPMfNOs.

Ok, so I made up the acronym, but I can only dream of coming up with the satirical masterpiece that is.... ready for it.

Integrated performance measures for national oversight.

So targets then? No, no, no - integrated performance measures for national oversight.

Standards? Nope, you're not listening. Pay attention at the back - integrated performance measures for national oversight.

Riiiiight. You say integrated performance measure for national oversight. I say target. And I'll be in Scotland afore ye.

The reason why the BBC's tweet is too far fetched to be remotely believable is that 3.6 million people attended major A&E units between April and June 2011. For the time that they had to wait to almost double would be a mind-blowingly catastrophic deterioration in service.

What the latest figures from the Department of Health actually show is that 95.5% of the 3.6 million people attending major A&E units between 1 April 2011 and 30 June 2011 waited less than four hours.

During the same period last year (1 April 2010 to 30 June 2010), 97.7% of the 3.6 million people attending major A&E units waited less than four hours.

A deterioration in performance? Yes.

A doubling in waiting times? NO.

In Quarter 1 last year, 84,439 people waited longer than four hours. In Quarter 1 this year, 161,422 people waited longer than four hours.

Ahhhh! So that's the almost doubling that the shockingly inaccurate headline is trying to address! We've got almost twice as many people waiting more than four hours this year than last year.

Interestingly, the current official figures (much more detailed data is now available, but it's very much still in its experimental infancy) don't capture how long individual people actually wait - they only look at the four hour barrier, and report those waiting less than four hours and those waiting more than four hours. So actually the 161,422 people waiting more than four hours this year might (incredibly unlikely, but might) all have only waited four and a half hours, while the 84,439 people waiting more than four hours last year might (again, incredibly unlikely, but might) have waited seven hours.

Unrealistic example, granted. But the point is that we just don't know. All we know is the volume under four hours and the volume over four hours.

I replied to the BBC after their tweet:

Do0g1e Chris Mason 

Seriously BBC that is a shockingly inaccurate headline @bengoldacre @undunc RT @BBCNews A&E waiting times nearly double bbc.in/qaBGhr

12 Aug 

They didn't acknowledge me, but the power of the Goldacre is great, particularly when he re-tweets you:

bengoldacre ben goldacre 

yup -> RT @Do0g1e: shockingly inaccurate headline RT @BBCNews A&E waiting times nearly double http://bbc.in/qaBGhr

12 Aug 

A dozen or so re-tweets later, the BBC seemed to get the message (although still no acknowledgement... and they still haven't deleted their original tweet).

Their story headline went from the diabolical:

To the rubbish and cryptic:

To the still misleading and far too generic:

And that's what's currently up there.

Three swings, three strikes.

And I used to take note of the 'Last updated' tag, believing it to be a trustworthy way of knowing whether and when stories had been amended. The fact that all three versions of the story purport to be 'Last updated at 15:11' is clearly misleading nonsense.

All in all, a pretty poor show, and a totally unfounded and easily avoidable slur on the performance of the NHS.

Always dangerous to to be sanctimonious, but you'd have to think that a 10 second reality check would have prevented the original error. You don't need to be a health expert. You just need to think for a second and say, "So does this mean that people are waiting twice as long as they used to?".

If the answer is a resounding no, then tailor your headline appropriately. And if you revise it (repeatedly), try to remember to mention A&E, and the crucial four hour barrier.

It, you know, helps.

Chris