Episode Transcript
Interviewer: It's estimated that half of scientific studies are irreproducible. They can't be replicated and this is a problem. Today, we're talking about a case study in irreproducibility, up next on The Scope.
Announcer: Examining the latest research and telling you about the latest breakthroughs. "The Science and Research Show" is on The Scope.
Interviewer: I'm talking with Dr. Heidi Hanson from the Huntsman Cancer Institute in the Department of Family and Preventive Medicine at the University of Utah. It's been estimated that up to half of scientific studies are irreproducible. They can't be replicated and this is a big problem. Dr. Hanson, you've actually published a study that feeds right into this conversation. The study calls into question a correlation that has gotten a lot of attention in the past few years.
Alzheimer's Disease and Cancer
Dr. Hanson: It's previously been reported that cancer and Alzheimer's disease have an inverse association. So basically, what's been said up to this point is that if you have cancer, you're protected from getting Alzheimer's disease later in life. If you have Alzheimer's disease, it protects you from having cancer.
Interviewer: And this got a fair bit of attention. There was a report in USA Today, there were reviews and nature of reviews, neural science and several other publications. How did the authors of those studies come to that conclusion in the first place?
Dr. Hanson: There have been a couple of studies where they've looked at individuals that have had cancer, and followed them for a period of time, and look at their Alzheimer's disease risk. And then, they also look at patients with Alzheimer's disease and look at their cancer risk later on in life. It's been published using a couple of bigger studies. They did the normal statistical methods that you might be doing just to come to that conclusion.
Interviewer: So basically, for those people who have cancer, fewer of them are found to develop Alzheimer's disease?
Dr. Hanson: Yeah, that's correct.
Interviewer: And what about that result set alarm bells off for you?
Dr. Hanson: I'm trained to think a lot about selection, and in particular, mortality selection. So what that means is I think about how processes that lead to different rates of death can affect the results that we see. And part of my demographic training is to think through some of those things. So I'm constantly looking at a result and asking if I really think that that's what's going on or if there is something underlying the result that we're seeing. So yes, it may be what the data is telling you, but is what the data is telling you actually what's going on? Are we missing something bigger?
Interviewer: Keeping that in mind, what was it that you found in your study?
Dr. Hanson: Our study replicated some of the previously reported results. And then, we showed, once you start to think about these things, and think about how mortality is affecting the rates of Alzheimer's diagnosis in these patients, you actually see a different story. It's not that there is not that inverse association that exists, but it's that mortality is driving that inverse association. It's not because there is some underlying cellular genetic mechanism underpinning both diseases. It's because if you have cancer, you have higher mortality. You're not going to go on to live long enough to be diagnosed with Alzheimer's disease.
Age Related Diseases
Interviewer: It certainly makes sense. And that's actually really important, you've said, when you're thinking about aging-related disease and the aging population. Can you talk about that a little bit more?
Dr. Hanson: Yeah, absolutely. So when we're aging, there's a lot going on. You aren't usually suffering from a single chronic disease. There are multiple thing going on at the same time. And if you think of aging in a single context or aging with a single disease and you're ignoring all of those other things that are going on, you're missing the bigger story.
Interviewer: Do you think someone could come along a few years from now and find that maybe you didn't consider something in your analysis?
Scientific Studies
Dr. Hanson: Absolutely, and that's why I like science so much. We're not coming up with the best answers all of the time. It's an iterative process. We should all be considering each other's work, and we should all be critical of each other's work and figuring out how we can really understand what's going on. And to do that, it's necessary to be critical and to try to decide, okay maybe if we look at this a different way, we will be seeing something else. So maybe there is this underlying mechanism and if we're able to look at it this way, we can get more into what's going on. And that's what should be happening.
Interviewer: Yeah, that's a really good point. I think one of the issues that you had brought up is that you're really trained to really look at the data and consider all the factors that might go into some of these correlations or some of these results. What do you think can happen to make sure that some of these people who are trained in the life sciences might consider some of these other types of analysis or other types of questions?
Dr. Hanson: Yeah, one of the biggest things that I think can really help that is working interdisciplinary. If we are working across our own disciplines, naturally we are trained to think different ways, naturally we're going to approach problems from a different direction, and naturally we want to start to question different things. Things where I've been trained to somewhat ignore them through my training, someone else may look at the same problem and say, "Wait a second. You're not thinking about this. You need to be really critical of this."
And that's what's so fascinating and fun to work with individuals from different disciplines. It's how really good science is done, in my opinion. And really good science can't be done without that difference of thought. I think it's absolutely necessary. And I'm seeing a lot more of it, which is exciting.
Interviewer: So do you think this is a common problem that people aren't considering their questions carefully enough?
Publication Bias
Dr. Hanson: I do. I think it's a very common problem. I think that people find the results that they're looking for a lot of times, and I think that's unfortunate. And I think that publication bias leads into the kinds of problems that we are seeing where people are only reporting certain things or things are only getting published if they are of interest to the public. I think that causes problems. I also think the really big push to publish fast causes huge problems. And it's unfortunate.
People just aren't as thorough with their statistics, with their methods, with their thinking through the problem as they should be because there's such a push to get the publication out. It's this huge push. Everybody wants to move things quickly, do one analysis and send it off. And that's what you do. And I think it's unfortunate.
Announcer: Interesting, informative and all in the name of better health. This is The Scope Health Sciences Radio.