Book 1 - Ben Goldacre "Bad Science"
Jan. 1st, 2022 10:59 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
jazzy_daveBen Goldacre "Bad Science" (Fourth Estate)
A light-hearted lay approach to deconstructing what awful messes journalists and 'humanities graduates' can make of basic statistics and scientific press releases - and actual science papers on the rare occasions that they look at them. The focus is mainly on medicine and health - partly because Dr. Goldacre is himself a medical doctor but also because that is where the media seem to find so many stories on the assumption that people like them.
Other key chapters focus on Homeopaths, Nutritionists, and in a rare break from health - children's education. The overall message is quite straightforward. Science isn't difficult, even if the details can be - you test your idea fairly and then faithfully report the outcomes, but exaggeration doesn't help, even if it means the story sounds better. And arranging a non-fair test is just stupid, it's as difficult and expensive to do as a fair test, but doesn't tell you anything and may even mean that you end up believing in something that is wrong.
Ben carefully and patiently takes you through the various ways in which the human brain is capable of fooling itself: much like optical illusions, humans are good at spotting patterns even when they aren't there; and how nature contrives to aid this process via the placebo effect and 'regression to the mean'. He then moves on to looking at what is a 'fair' test through the means of various counterexamples so readily provided by Complementary Alternative Medicine practitioners. It's not that they are lying or deliberate fraudsters - although this may be true too - it's that in the examples he chose it is clear that for whatever claim is made, no evidence exists to substantiate it. It is possible, in contravention of all currently known theory, that some of these practices and products may work - but until you test it with controls against the placebo effect, insufficient numbers of patients and duration, with 'blinded' and properly randomized procedures, to avoid regression to the mean and bias, you can't know if it is working. It's not the product per se that Ben has a problem with, it's the methods used to claim it works.
Conventional medicine isn't of course immune from this either, and Ben spends a while discussing how they can be even more creative in presenting artificially good news. But just because a pharmaceutical company has presented misleading data doesn't mean therefore that the CAM product is better or vice versa.
Occasionally he gets bogged down in details, and sometimes he skips a few steps that perhaps would be clearer if they were fully explained - a tricky line to draw. I'm not sure how much basic understanding is required to read this - some definitely, especially familiarity with logical arguments. So it isn't suitable for just anybody. There are numerous references at the back to the various papers and studies he’s quoting. But because he wants this read easily they are not numbered in the text – which makes them hard to check. 'Humanities graduates' get a few ad hominem attacks levelled against them. I don't know if a newspaper editor is more or less likely to be a humanities graduate, but the practice of assigning high-profile science stories to a general reporter rather than a specialist science correspondent does seem to bring out the worst in reporting standards.
What to do about it - this is perhaps the weakest point in Ben's book. Other than calling for a national trials database he makes a few suggestions. You should read the method and results in sections for the published paper that underlies any story - if you can't find a published paper then the story is probably rubbish to start with. But who has time to read and check such things? Ask searching questions. Or else treat almost everything you come across with a great deal of skepticism.
The light-hearted tone prevails though. Although many people are silly, truth and wisdom will out, especially if you read this worthwhile book and think about what underlies some of the assertions the media and CAM are trying to tell you. Besides in many places, it's also quite funny.
A light-hearted lay approach to deconstructing what awful messes journalists and 'humanities graduates' can make of basic statistics and scientific press releases - and actual science papers on the rare occasions that they look at them. The focus is mainly on medicine and health - partly because Dr. Goldacre is himself a medical doctor but also because that is where the media seem to find so many stories on the assumption that people like them.
Other key chapters focus on Homeopaths, Nutritionists, and in a rare break from health - children's education. The overall message is quite straightforward. Science isn't difficult, even if the details can be - you test your idea fairly and then faithfully report the outcomes, but exaggeration doesn't help, even if it means the story sounds better. And arranging a non-fair test is just stupid, it's as difficult and expensive to do as a fair test, but doesn't tell you anything and may even mean that you end up believing in something that is wrong.
Ben carefully and patiently takes you through the various ways in which the human brain is capable of fooling itself: much like optical illusions, humans are good at spotting patterns even when they aren't there; and how nature contrives to aid this process via the placebo effect and 'regression to the mean'. He then moves on to looking at what is a 'fair' test through the means of various counterexamples so readily provided by Complementary Alternative Medicine practitioners. It's not that they are lying or deliberate fraudsters - although this may be true too - it's that in the examples he chose it is clear that for whatever claim is made, no evidence exists to substantiate it. It is possible, in contravention of all currently known theory, that some of these practices and products may work - but until you test it with controls against the placebo effect, insufficient numbers of patients and duration, with 'blinded' and properly randomized procedures, to avoid regression to the mean and bias, you can't know if it is working. It's not the product per se that Ben has a problem with, it's the methods used to claim it works.
Conventional medicine isn't of course immune from this either, and Ben spends a while discussing how they can be even more creative in presenting artificially good news. But just because a pharmaceutical company has presented misleading data doesn't mean therefore that the CAM product is better or vice versa.
Occasionally he gets bogged down in details, and sometimes he skips a few steps that perhaps would be clearer if they were fully explained - a tricky line to draw. I'm not sure how much basic understanding is required to read this - some definitely, especially familiarity with logical arguments. So it isn't suitable for just anybody. There are numerous references at the back to the various papers and studies he’s quoting. But because he wants this read easily they are not numbered in the text – which makes them hard to check. 'Humanities graduates' get a few ad hominem attacks levelled against them. I don't know if a newspaper editor is more or less likely to be a humanities graduate, but the practice of assigning high-profile science stories to a general reporter rather than a specialist science correspondent does seem to bring out the worst in reporting standards.
What to do about it - this is perhaps the weakest point in Ben's book. Other than calling for a national trials database he makes a few suggestions. You should read the method and results in sections for the published paper that underlies any story - if you can't find a published paper then the story is probably rubbish to start with. But who has time to read and check such things? Ask searching questions. Or else treat almost everything you come across with a great deal of skepticism.
The light-hearted tone prevails though. Although many people are silly, truth and wisdom will out, especially if you read this worthwhile book and think about what underlies some of the assertions the media and CAM are trying to tell you. Besides in many places, it's also quite funny.
![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
no subject
Date: 2022-01-03 04:14 pm (UTC)no subject
Date: 2022-01-03 04:33 pm (UTC)