Scientific studies are generally either causation or correlation and are pretty easy to tell the differences between, as long as you are careful with them. Causation means we can show a direct link with physical evidence, and those studies can be smaller, but still be pretty accurate. Correlation studies are simply seeing that two things occur at the same time or circumstances, and for there to be any accuracy, you have to exclude anything that could cause a problem, and you need really large numbers involved.
I once explained it to my kids like this. Almost every night we have dinner at our house. Almost every night we have music playing during dinner. For exclusions - Music is not played during lunch or breakfast. A simple correlation might be that dinner causes music, or music causes dinner. A study might be designed to observe how often dinner and music happen together. A small sample of nights at our house, from say...the last several weeks would prove that dinner causes music.
But if you were to observe over several years, you would note that dinner did occur without music long ago. You would see that in other homes, dinner occurs with or without music, and that music is played even when no dinner is planned or eaten. A bigger sample gives different results.
Or a REALLY good scientist would carefully test Mr. or Mrs. Cotta by preventing them from flicking the on switch to the stereo prior to dinner, or maybe make them eat dinner in a lab without any source of sound.
Or he could just get his head out his ass, and realize that dinner doesn't cause music.
Which leads me to the ESHRE conference this year that has been turning out loads and loads of papers, presentations, studies. Only one problem...most of them seem designed to get headlines, not progress the cause of curing infertility.
I once mentioned to a friend that several of the REs, OBs & Peris we all see are very PR savvy & competitive, and I don't think she believed me. But really, most of these guys have loads of dough, so much that when I run my annual Google, Lexis-Nexis, & credit check on each of my Docs, they come up overflowing with cash. (No, I'm not kidding, when I was young and naive I dreamed of kind & competent Doctors, now I'm old and bitter I want to read the case files on all their lawsuits...)
So I know it's all about headlines.
How else can I explain the fact that this piece of trash got presented? There was no separation of methods of alternative therapies, no tracking of who saw a licensed Naturopath or Acupuncturist vs. who just bought a bunch of crap from their local health food store. They literally lumped every single type of treatment together, but of course, didn't take into account that patients LIE about alternative treatment to medical Doctors all the time. Why do they lie? Because so many medical Docs dismiss alternative therapy without ever actually studying it, and patients are so afraid of the white coats, they don't want to fess up. They can end up harming themselves because of dangerous interactions, or wasting their money on things that don't work. Or tragically, they could miss out on something that does work.
I remember one pharmacology lecture at our local big university, and the prof described a study they had done on Greens. The creator claimed it was wonderful for health and energy and the immune system, but without a study, no proof. So...they did a study. And in a large double-blind placebo vs. Greens randomized control trial, the people taking Greens had better functioning immune systems & better health, as measured by blood tests, compared to the beginning of the study.
But no-one wants to hear that, they only want the bad stories. So we have trashy poorly done studies, and GREAT BIG HEADLINES.
Like this next one, a discussion of PGD and IVF outcomes. I can't begin to describe all the problems I have with this study, but you know I'll try. Causation studies are 100% better than correlation studies at a level of 408 patients per group, I believe. That's far too small in my opinion when we don't even know for sure why the women were doing fertility treatment. Had everyone of them gone through a thorough diagnostic workup for thrombophilias? Had they all had laparoscopy to check for endometriosis? Did any of them have genetic or chromosomal issues that might've clouded the results? Was it simply blocked tubes? Male factor? If so, did they have ongoing adhesions elsewhere that might've harmed follicle development or implantation? Why were only women 35+ put in the study when we know that women of all ages are susceptible to chromosomal losses, which is why the SOGC now recommends offering prenatal screening to all women?
We don't know the answers to these important questions, because it was poorly designed.
A better example is the genetic study I was in. It had only 50 people in each group (patients & controls), but it was a causation study. Every woman, even the controls, had had a laparoscopy and either had proof of endometriosis by pathology, or proof they were endo free. Then they did a gene test looking for PAI-1 alleles. 48 of the women who had endometriosis had the PAI-1 4G/4G allele or the 4G/5G allele. The women without endo had the 5G/5G allele. This gene prevents excess fibrin from being swept away whenever a tear, even a micro one is caused. Endometriosis adhesions are fibrin. That's causation people.....so 100 total divvied into two groups is okay. More is better, but at some point we have statistical confidence. Would it be better to watch the fibrin as it grows? A little bird told me they have done it in a lab. Just a bit harder in a live woman because we don't know how to stick tiny cameras into the body for years on end yet.
For the PGD study, embryos were sampled very very early, which leads to the possibility of mosaicism, or that the scanning they used was not adequate. The lead researcher (and I'm using that label loosely....eyeroll) admitted that they had not scanned every chromosome that could possibly show up. WTF? There ARE methods that can scan for every chromosome, so why the hell wouldn't they have used the more advanced procedure? This is the equivalent of using substandard old-style radiation treatment for cancer, and declaring that radiation doesn't work! Now we have NO WAY of knowing why the women miscarried, because all chromosomal abnormalities like say Trisomy 3, 4, 5, 6 etc....cannot be excluded.
You know how they could've saved this study?
By finding out why each woman in the study lost their embryo. Think about it, for IVF at least we know that sperm met egg, we know that sperm and egg were growing and dividing and at least one did get into each uterus. No one checked if it was a receptive environment ahead of time, so no one could take low molecular weight heparin, or aspirin, or extended progesterone, or steroids, or hcg....but they could've looked afterwards, right?
For almost every single case in this study they could've done chromosomal analysis on the fetal tissue, or checked for calcifications, or inflammation, or infection, or thrombophilias. Many of the pregnancies in this study ended after 6 weeks and many after 12 weeks, so why couldn't they look?
For each pregnancy were there any cases of birth defects? Terminations for fetal anomalies? IUGR? Preterm deliveries? Pre-eclampsia? Stillbirths? - a serious cause of death which is regularly excluded from stats simply because it makes the study numbers look bad and no one wants to find the causes of stillbirths?
On the live babies who are born afterwards, were any chromosomal tests done? Any metabolic tests? Were there any children with mosaic results?
Without follow-up and outcomes what is the point of this, besides headlines?
This study, once again, is bullshit. 'Scuse me while I go pick music for tonight's dinner. It would be a better use of my time I think.
I'm with you on the "bad science" and "bad science reporting". I was very angry when this study came on the news last night because I personally know 3 women IRL that only seemed to conceive after they started seeing a homeopathic/acupunturist physician in conjunction with IVF. They had tried many cycles of IVF with no luck. So how's that correlation for ya?!
ReplyDeleteAmen.
ReplyDeleteScientists and science reporters are getting worse and worse. And the ones with real issues (think disappearing bees), can't get congress to understand diddly about their research. Oh, of course, let's not forget that they don't know what cloning is - but think they have outlawed only that.
I just reviewed one of the worst books I have ever read on IVF that is getting great press - Everything Conceivable - on my blog. I have nothing but loathing for the current state of affairs in both reporting and research. And it isn't just in medical research. Everyone, it seems, is out for the fame rather than the knowledge. And facts have no bearing on reality.
My DH has been known to turn off the radio or TV when some idiot reporter or scientist starts going on and on about a false fact. Oh, and let's not even go into the funding bias issues! Now, I suffer from corn allergy, but corn allergy doesn't exist according to the University of Nebraska which receives major funding from Monsanto. (The largest agribusiness concern that infests everything with corn.)
Oh, the easiest way to get away from the hype is to look at what the international studies have to say as well. After dealing with medline for allergies and immunology - which is poorly studied in some areas but gets accidental coverage through AIDS - I was shocked at the dearth of good research in infertility. It really is a baby science in soooo many ways.
Pax,
MLO
Abso-bloody-lutely. I'm a bit pissed at all the publicity since I've been diligently writing a summary of the conference all week by hacking into the website to read the actual abstracts, but they have so many disclaimers in I halted publishing it til the conference was over. Now all the sexy stuff is out there anyway, causing mayhem. Ah well.
ReplyDeleteThank you, Aurelia. I knew you'd tear it apart so we could see the filth underneath.
ReplyDeleteIt's just an extension of that "bubble headed bleach blonde comes on at five" syndrome. Reporting, it seems, is now thoroughly about what can get the outlet the most money in advertising dollars and not about truth - it was only a matter of time before "science" followed. Public good? What's that?
ReplyDeleteExcellent analysis. Thanks for sharing it.
yes...way to educate.
ReplyDeleteThe natural therapies one seems more a case of bad reporting. Compare the headline and bold-type first paragraph with the 4th/5th paragraphs. I don't know what statistical analyses had been done, but you have to take retrospective studies for what they are. I just love how the media reports these headlines, then about halfway down you start getting quotes fromm doctors saying, "Look, this is far from answering the question definitively..."
ReplyDeleteI'm yet to read that other one.
Bea
Hi
ReplyDeleteNew to your site. Interesting analysis of the conference. I've been disappointed with what's been reported this week. It's all about what's newsworthy...IVF is big bucks business.
Portia
Thanks for ripping the article. I didn't have the energy to even think about it.
ReplyDeleteI just love these studies that come to blaming conclusions about women.
very well said. THank you for breaking this down so well!
ReplyDeleteI agree that most studies do not use control groups, are not longtitudinal nor can they be replicated. Most are just correlations which does not get to causal factors. So my questions is: When are you going to write for a grant to do a real study...of course if you did, it may not get headlines right away.
ReplyDeleteGreat rant - thanks for the explication.
ReplyDelete