Larry Wall once noted that the scientificness of a field is inversely correlated to how much the word “science” appears in its name. Physics, of course, doesn’t have science in the name and is the most scientific of all sciences. Then comes biology and and ethology and so on. Then come the non-sciences like Computer Science and Poultry Science. And worst of all is Scientology.
In general, whenever someone tells you that “science” has decreed that you should do one thing or another that doesn’t seem reasonable, it’s probably because they’re trying to pull one over on you, whether it’s the “scientific” medicines you see on late-night TV or the science of the behaviorists who say you shouldn’t love your kids.
But nowhere is this more evident than when people try to tell you what science itself is. This field of meta-science seems to attract more charlatans and malintents than any other. If you control how the very notion of what’s scientific is defined — well, then that’s real power. Even if the very idea is patently absurd. (A real scientist would never tell you that doing X isn’t really science; their goal is to get the truth, not sit around making rules about who’s in and who’s out.)
For much of the outside world, the test for a real science is “falsifiability” — the possibility that there could be evidence proving the claim wrong. This notion was invented by Karl Popper, who was himself an enemy of science who tried to insist that science never actually made any progress, that we never learned anything more about the world.
But even if we put aside this noxious pedigree, Popper’s definition is still absurd. Take the distinction between astronomy and astrology. We would all agree, I think, that the first is a science but the second isn’t. But both of their predictions are equally falsifiable — astrology makes a dozen falsifiable predictions in the newspaper five times a week. Popper’s criteria isn’t of much help to us, even on such a basic case.
Sadly, like many American intellectuals, the Supreme Court assumed that falsifiability was a standard scientific test. In the Daubert case it, as Chris Mooney summarizes the view of the American Journal of Public Health, “blundered miserably” and set judges the task of using this “deeply confused philosophy of science” to act as gatekeepers in keeping scientific claims from juries. Actual scientists like DefendingScience.org is working to undo this these mistakes, but you wouldn’t know it from the rhetoric — after all, Daubert’s defenders claim their just trying to uphold sound scientific standards. (Chris Mooney’s book The Republican War on Science, among others, has a fascinating exposé of the junk science/sound science notions cooked up by the PR industry to trash actual science. But that’s another subject.)
What are the real effects, though? Daubert was a parent whose child was born with birth defects they believed were caused by the drug Benedictin which, in animal studies, appeared to cause the defects they were suffering. By making it harder for science to be presented in Court, these kinds of rulings make it easier for drug companies to claim there’s no “sound science” that they’re hurting anyone.
America isn’t alone, however. In Britain a group supporting what they call “evidence-based medicine” is trying to tighten restrictions on what experiments can be examined when approving drugs. Evidence-based medicine? Who could be against that! But again, they’re playing the same games. Behind evidence-based science are a bunch of very bizarre claims about what science is and isn’t, taken not from doctors or scientists, but from econometricians (the subspecialty of economics that has to do with calculating things), which have quite a few problems of their own when it comes to the subject of evidence.
Under “evidence-based medicine” rules, doctors aren’t allowed to prescribe drugs on the basis of case studies and other reports; instead, the only real evidence are large double-blind random controlled trials whose results have a less than 5% probability of being due to chance. (Why 5%? No good reason. But according to the EBM people anything more than that isn’t evidence.)
Again, you have the same negative effects: when someone tries to claim in court that a certain drug destroyed their life, the drug company can claim that there’s no “evidence” to support this if the studies just happen to be 94% likely instead of 95%, or if there’s only a series of case studies instead of a controlled trial.
This isn’t evidence, this isn’t rationality, this isn’t science. Science is about trying to get the truth about the world, using whatever mechanisms are most effective at the job, whether you’re studying the nature of planets in space or the nature of other cultures. When someone tells you otherwise, tries to insist that technique X or subject Y doesn’t deserve the name science, it’s probably because they’re trying to pull a fast one on you.
(Why 5%? No good reason. But according to the EBM people anything more than that isn’t evidence.)
The 5% cutoff for finding statistically significant results is a standard in statistics. So they aren’t just making that up. It goes all the way back to R.A. Fisher. Of course, he was just making it up. :-)
posted by crh
on October 23, 2006 #
crh: That’s what I meant.
posted by Aaron Swartz
on October 23, 2006 #
I’m amused by the use of statistics in determining exactly what’s true or not when the intro to statistics book I’m reading explains that statistics will never tell you that. They will only tell you things like to what degree the data fits a hypothesis. Statistics is such an abused and misunderstood area of mathematics that I would say that anyone who isn’t a professional statistician is 89% likely +/- 4% to be wrong about it. The book I’m reading also explains how sometimes you just can’t do tests like the ones they suggest. How do you do a double blind test of the effectiveness of birth control pills on human beings ethically?
posted by Michael Conlen
on October 23, 2006 #
Econometrics studies the measurement of things (in particular of random variables). Nothing to do with calculating, for calculating algebra is just fine.
posted by econgeek
on October 23, 2006 #
<
p>Aaron: can you provide a link or a reference where I can find a list of these ‘evidence-based medicine’ rules? I’d like to look
<
p>From my reading of what you paraphrase, it sounds like they’re only saying “experiments used to evaluate the efficacy of medicines must follow a given set of statisitcal best practices”. To me that seems a fairly reasonable thing to do. But before I continue to rant about this, I’d like to read the same rules you refer to, so that we’ll be discussing the same thing.
posted by crh
on October 23, 2006 #
<
p>Aaron: To respond to your two-word email,
<a href=”http://en.wikipedia.org/wiki/Evidence-Based_Medicine”
I did try wikipedia
. It doesn’t say what you’re saying about EBM.
<
p>In particular, it does not say that the only acceptable evidence under EBM are well-designed experiments.
<
p>Three separate sets of criteria for “Levels of Evidence” are present. Two of the three accept “expert opinion” (eg. what your doctor thinks) as a form of evidence, but say that it’s a weaker form than double-blind randomized trials. This seems reasonable. Nothing on wikipedia’s EBM page mentions the 5% cutoff for significance.
<
p>Is wikipedia’s EBM page more moderate than the general EBM population?
posted by crh
on October 24, 2006 #
“A real scientist would never tell you that doing X isn’t really science; their goal is to get the truth, not sit around making rules about who’s in and who’s out.”
This claim is absurd. Real scientists tell you that doing X is or is not science all the time. In the US, biologists fight creationism. Scientists write books about pseudoscience or junk science and why it’s not science. Richard Dawkins has a new bestseller out on why religion is incompatible with science.
There is no contradiction between getting the truth and making rules about what (and who) is in and what’s out as methodology goes for reaching that goal; the latter helps the former.
It’s also absurd to say that just because you can falsify an astrological prediction in a local paper, Popper’s falsifiability criterion is thereby refuted. Popper advanced the falsifiability criterion as a necessary condition for a scientific theory, not a sufficient one. I can make all sorts of falsifiable predictions (for example, which of my friends are going to marry or divorce over the next few years), and it wouldn’t mean that I’m basing them on a scientific theory.
posted by Anatoly
on October 24, 2006 #
Creationism, psuedoscience, and junk science are claims, not methodologies.
And can you think of anything useful that falsifiability does?
posted by Aaron Swartz
on October 24, 2006 #
I disagree with your point about astrology.
The whole reason people believe in astrology is not because they believe in the “theoretical underpinnings” of it.
They believe because the predictions are sufficiently vague that they apply to everybody all the time.
I’ve heard a story about a prof who hands out personalized astrology readings to everybody in their class, and then has them rate how accurate the prediction is. Most people rate the predictions fairly highly.
He then reveals that everybody has the same prediction.
posted by Eric Gunnerson
on October 24, 2006 #
Creationism isn’t simply a claim, it’s a movement. It has its own methodology, its own “researchers”, its own journals. Its methodology provides one of the main lines of attack scientists use when they argue that creationism isn’t science, and falsifiability - or lack thereof in creationalist methodology - happens to be one of the primary arguments they use. For example, the claim that God created Earth 6k years ago - but did it with all the exact evidence that seems to show it’s much older (with fossils already in the ground, etc.) - is internally consistent. It is, however, unfalsifiable and therefore arguably unscientific.
Falsifiability is used all the time. There’s a current debate about string theory in physics with two books out, by Peter Woit and Lee Smolin, arguing that string theory never made good on its promise to deliver testable predictions. You can’t have a testable prediction if the theory’s unfalsifiable.
Falsifiability is one of the arguments that were used to show that Marxism and Freudian psychoanalysis are pseudoscience and not science. Both claimed to be scienctific, both convinced many people (including real scientists). Popper, whom you glibly and absurdly condemn as “an enemy of science”, was the first to realise that psychoanalysis was flexible enough to explain any behaviour whatsoever, and therefore didn’t really explain anything. Marxism as a real economic theory could be and was falsified, but as actually practiced by Marxists it was unfalsifiable.
posted by Anatoly
on October 24, 2006 #
posted by
on October 24, 2006 #
“Larry Wall once noted that the scientificness of a field is inversely correlated to how much the word “science” appears in its name.”
This is ludicrous. Is Islam more scientific than computing science? And what about Chemistry and Christianity? Neither of them have ‘science’ in the name. Do we thus conclude that their status is equal? If Physics was renamed Phyiscal Science tomorrow would it lose its ‘scientificness’? If Scientology became Hubbardology would it increase in ‘scientificness’?
The reason why older sciences like Physics do not have the word science in their name is simple - when they were born, centuries ago, the notion of science had not yet been clearly articulated (Indeed the more modern meaning of the word science dates back to only the early 17th century).
The name ‘computing science’ is appropriate as the word ‘computing’ denotes an awful lot of activities not all of which can be described as scientific.
Whether or not a field is scientific can be determined only by the nature of its activity and not simply by its name.
posted by Eric
on October 24, 2006 #
Falsifiability is a necessary but by no means sufficient condition for something to be scientific. It seems to me that it is not only a question of whether a field states falsifiable proposition that makes it a science. It also depends on whether someone actually bothers to falsify these propositions which is probably not the case for astrology.
And I don’t get your point about scientists using whatever methods will get them to the truth at all. If the method is such that you will not be able to tell truth from falsehood, then what good does it do you?
posted by Tobias
on October 25, 2006 #
For a similar article that makes a run at double blind experiments and how testing takes place nowadays and has more compelling and coherent points to make, read this article. However, while that article also tries to drive in a similar direction, I’m not sure he ever gets there. Yes, there are huge problems with the way many experiments are done, but he’s largely arguing that double-blind experiments are not executed well, rather than arguing against the concept. I think there’s scope for other types of scientific experiments that take into account the ethical problems that human experiments can have. For example, once double-blind experiments are done and something has been released to the public, why aren’t all uses of the drug tracked and any adverse affects immediately noted? Why did more than 25,000 people have to have heart attacks before the drug Vioxx was finally pulled?
Your attack on falsifiability is ridiculous, Aaron. As Eric points out, astrology is actually not falsifiable most of the time because the predictions are designed to be vague, “Something good will happen to you this week.” And what does it matter if the number chosen for statistics is 5%? It’s like arguing that there should be no blood alcohol limit for drunk drivers because 0.08 is an arbitrary number. If you have a case for it being higher or lower or would like to use better statistical methods, fine, make your point. But to use anecdotal evidence of one person who has a problem and to point to genuinely murky scientific results like animal tests (animal tests sometimes predict what happens to humans and sometimes don’t) is not much of an argument. Double blind tests are the best methods we have come up with to tell when something is working or when it isn’t. When you argue against them, YOU are in fact arguing against science and rationality, especially if your only argument is that 5% is an arbitrary cutoff.
I do agree with you that these double blind experiments may be misused if people say that they prove that Benedictin doesn’t cause birth defects. However, they are right about a weaker but still relevant point: the experiments do show that we do not know what caused the problems and it is unlikely that it was this drug (assuming they ran their experiments properly, as detailed in the first link).
Finally, I agree that econometrics has many problems, though I haven’t looked into the matter very deeply. However, the same econometric mathematical models are used by the global warming crowd to reach their doomsday predictions (links:
1,
2,
3). Still skeptical about econometrics?
posted by Ajay
on October 26, 2006 #
And can you think of anything useful that falsifiability does?
Absolutely, it directs people to work towards areas that yield real scientific progress, where we can test models against reality, rather than complete speculation. It’s the reason that many may have wasted their lives (from a scientific perspective) if string theory doesn’t pan out. There are certainly legitimate reasons to create non-falsifiable claims, and discuss non-falsifiable issues, but they are not scientific claims/issues at that point.
posted by Jacob Rus
on October 27, 2006 #
Yeah, that must be it. The physicists involved in string theorists must all be idiots who never heard of Karl Popper or read the newspaper or watched a television special about themselves. If only they knew they weren’t doing science!
posted by Aaron Swartz
on October 27, 2006 #
String theorists do know they’re not doing “science” in the full sense. They know their theory isn’t falsifiable—it doesn’t make ANY predictions about reality that can be observed! That puts it closer to the unfortunate class that Intelligent Design is in. However, two things may set String Theory apart: the fact that some of the brightest minds on the planet are working on it, and that it does predict some things that may one day be testable/observable, but we don’t have the technology to be able to test them yet.
(This is just some thoughts based off a recent Slashdot posting.)
posted by Scott Teresi
on October 27, 2006 #
Aaron, in case you didn’t know, string theory is considered controversial for precisely the reasons that Jacob lays out. In fact, as Scott points out, the theorists would probably be the first to admit that they’re not actually doing science, just scouting out the possible solution space for likely answers. The scientific method will then involve testing these hypotheses to see what holds true. Instead of finding out more about this or asking questions, all you offer is ridicule, which is especially jarring considering your ignorance of this subject.
posted by Ajay
on October 28, 2006 #
I do think it’s ridiculous that Jacob presumes to tell prominent physicists how to spend their time.
Vague predictions are falsifiable; they’re just less likely to be wrong. And my argument isn’t that 5% is too high or too low, it’s that it’s arbitrary. If a study works only at 6% should we sentence people to death because we’re missing the other 1%. (Actually, what really happens is that people fake the data to get it over the hump, as a recent paper proved.) Evidence doesn’t have clean lines between true and false; we depend on the situation to decide how strong our standards should be.
Who’s arguing against double-blind tests? All I’m saying is that other stuff is evidence too.
An economist doesn’t believe global warming…so I should like economics more? Please.
posted by Aaron Swartz
on October 29, 2006 #
Aaron, where do you come up with these obtuse arguments?
I do think it’s ridiculous that Jacob presumes to tell prominent physicists how to spend their time.
Jacob didn’t tell anybody how to spend their time. He pointed out the common criticism that string theory might turn out to be a complete waste of time because they have yet to make predictions that are testable.
Vague predictions are falsifiable; they’re just less likely to be wrong.
It depends on who you’re trying to prove it to. Scientists and intelligent laymen know that astrology is falsifiable and that it mostly comes out wrong. However, the kind of people who believe in astrology are willing to seize on anything that might fit the prediction. “Something good will happen to you this week” = Monday was a bright and sunny day. Since you said falsifiability isn’t much help, we took it for granted you were talking about these dim people. For them, it is unfalsifiable because they are willing to make their definitions so broad as to be meaningless.
And my argument isn’t that 5% is too high or too low, it’s that it’s arbitrary. If a study works only at 6% should we sentence people to death because we’re missing the other 1%. (Actually, what really happens is that people fake the data to get it over the hump, as a recent paper proved.) Evidence doesn’t have clean lines between true and false; we depend on the situation to decide how strong our standards should be.
5% is not an arbitrary cutoff if comparison of many studies has shown it to be a good cutoff between studies that show repeatable results and those that don’t. I’m guessing that’s how it was arrived at. You setup a false situation by assuming that a whole bunch of studies cluster around that number and that any of them over 5% are then tossed away as a result. There are very likely few studies that even get close to that threshold. Besides, I doubt that doctors actually use a hard threshold like that. What you seem to be primarily talking about is a legal threshold, which seems to be required by the way the law works. I agree with you that if doctors are actually using a hard theshold like that, it doesn’t make much sense, but I doubt that they do.
Who’s arguing against double-blind tests? All I’m saying is that other stuff is evidence too.
Presumably you want to use other tests because you see some weakness or failing in double-blind tests that you would like to address. And you did say, “Behind evidence-based science are a bunch of very bizarre claims about what science is and isn’t,” which combined with your attack on falsifiability would appear to be an attack on the method they’re advocating. And I’m not (and I don’t think the others are) saying that there aren’t any problems with that way things are done. I previously pointed out that more effective follow-up studies should be done. What we’re doing is pointing out that your particular criticisms don’t hold much water. Falsifiability is a necessary condition for science, animal tests aren’t that helpful, anecdotal examples are not useful on their own, and case studies can be problematic.
An economist doesn’t believe global warming…so I should like economics more? Please.
Perhaps you didn’t read those posts properly, let me summarize. He is a former economist who points out that econometrics has failed miserably at what it was devised for, macroeconomics. He then goes on to point out that the same type of models, that you have claimed skepticism about, are used to make the dire predictions coming from the global warming crowd. Do you see how this points out a contradiction for you?
posted by Ajay
on October 29, 2006 #
Ajay now insists that theories can be true or not true depending on who reads them, that because so many people use the number 5 it simply must have been chosen through studies (although he hasn’t seen any), and that because economics once was wrong it’s a contradiction to believe that environmental studies are now right. I think Ajay should refrain from posting again until he learns basic principles of logic and evidence.
posted by Aaron Swartz
on October 29, 2006 #
Ajay now insists that theories can be true or not true depending on who reads them
No, what I said was that different people have different standards of evidence. The reason we call them theories and not laws is because new evidence may always come up that could overturn what we believe. Many people have very low standards of evidence; that is why they believe in astrology.
because so many people use the number 5 it simply must have been chosen through studies (although he hasn’t seen any)
No, I said that as long as it was only used as a legal standard, it’s probably okay. It’s true that I didn’t cite any studies, but then neither did you. Am I to be held to a higher standard?
because economics once was wrong it’s a contradiction to believe that environmental studies are now right.
Since you took the time to define econometrics above, you obviously know the difference between the two and that your conflation of the two is a misrepresentation of what I’m saying (your lies are Cheney-esque). Who’s talking about environmental studies? We’re talking about econometric models that purport to forecast future global warming based on current trends.
In your comments over the last several months, you repeatedly make bizarre statements or seize on irrelevant, minor points. Arguing with you is like arguing with a child, a style of discourse that Jon Stewart ripped in his famous Crossfire episode (though to be fair to you, those guys choose even more trivial points and only speak in repeated bromides).
posted by Ajay
on October 30, 2006 #
And arguing with you is like having an email debate with someone who can’t read.
posted by Aaron Swartz
on October 30, 2006 #
Dear Aharon, When I was at Faculty, I always thought Popper’s claims on “falsifiability” were “bogus”. The proper term must be “testability” although many aspects of nature are not testable because we lack adequate techniques to test them. However, those unverifiable aspects of nature can be deduced from careful and “repeatable” or “attainable” observations. Mathematics has many times helped us with solutions and explanations to several observed natural events, but it is spot on also that mathematics is a formal science (created by the human mind) that most times is not compatible with reality. For example, through equations theoreticians can deduce the existence of parallel universes, strings, branes, living bubbles, etc.; but those things do not exist in real nature and, if existed, they are not observable, so it is trivial even to consider it as knowledge. Science is a set of knowledge about the Universe ordered and systematized by means of observation, analysis and experimentation, which permits the deduction of principles (theories) and general laws.
posted by Nasif Nahle
on November 7, 2006 #
While the M-theory (based upon so much frustrating work with different models of the string theory) cannot be considered true science because of the falsifiability aspect, it is highly improbable that future falsifiable theories that will emerge will be entirely different from it. There are too many “happy coincidences” in this approach to be just junk! An “educated guess” will say that at least chunks of it will be part of any new theory!
In fact many alternative explanations are falsifiable and have been eliminated. I think this is a totally new domain of the human knowledge that does not have yet a name (a label), but it is the closest you can get to science. It is science “de jure” if not “de facto”.
posted by Anton Constantinescu
on November 14, 2006 #
You can also send comments by email.
Comments
The 5% cutoff for finding statistically significant results is a standard in statistics. So they aren’t just making that up. It goes all the way back to R.A. Fisher. Of course, he was just making it up. :-)
posted by crh on October 23, 2006 #
crh: That’s what I meant.
posted by Aaron Swartz on October 23, 2006 #
I’m amused by the use of statistics in determining exactly what’s true or not when the intro to statistics book I’m reading explains that statistics will never tell you that. They will only tell you things like to what degree the data fits a hypothesis. Statistics is such an abused and misunderstood area of mathematics that I would say that anyone who isn’t a professional statistician is 89% likely +/- 4% to be wrong about it. The book I’m reading also explains how sometimes you just can’t do tests like the ones they suggest. How do you do a double blind test of the effectiveness of birth control pills on human beings ethically?
posted by Michael Conlen on October 23, 2006 #
Econometrics studies the measurement of things (in particular of random variables). Nothing to do with calculating, for calculating algebra is just fine.
posted by econgeek on October 23, 2006 #
<
p>Aaron: can you provide a link or a reference where I can find a list of these ‘evidence-based medicine’ rules? I’d like to look
<
p>From my reading of what you paraphrase, it sounds like they’re only saying “experiments used to evaluate the efficacy of medicines must follow a given set of statisitcal best practices”. To me that seems a fairly reasonable thing to do. But before I continue to rant about this, I’d like to read the same rules you refer to, so that we’ll be discussing the same thing.
posted by crh on October 23, 2006 #
<
p>Aaron: To respond to your two-word email, <a href=”http://en.wikipedia.org/wiki/Evidence-Based_Medicine”
. It doesn’t say what you’re saying about EBM.<
p>In particular, it does not say that the only acceptable evidence under EBM are well-designed experiments.
<
p>Three separate sets of criteria for “Levels of Evidence” are present. Two of the three accept “expert opinion” (eg. what your doctor thinks) as a form of evidence, but say that it’s a weaker form than double-blind randomized trials. This seems reasonable. Nothing on wikipedia’s EBM page mentions the 5% cutoff for significance.
<
p>Is wikipedia’s EBM page more moderate than the general EBM population?
posted by crh on October 24, 2006 #
“A real scientist would never tell you that doing X isn’t really science; their goal is to get the truth, not sit around making rules about who’s in and who’s out.”
This claim is absurd. Real scientists tell you that doing X is or is not science all the time. In the US, biologists fight creationism. Scientists write books about pseudoscience or junk science and why it’s not science. Richard Dawkins has a new bestseller out on why religion is incompatible with science.
There is no contradiction between getting the truth and making rules about what (and who) is in and what’s out as methodology goes for reaching that goal; the latter helps the former.
It’s also absurd to say that just because you can falsify an astrological prediction in a local paper, Popper’s falsifiability criterion is thereby refuted. Popper advanced the falsifiability criterion as a necessary condition for a scientific theory, not a sufficient one. I can make all sorts of falsifiable predictions (for example, which of my friends are going to marry or divorce over the next few years), and it wouldn’t mean that I’m basing them on a scientific theory.
posted by Anatoly on October 24, 2006 #
Creationism, psuedoscience, and junk science are claims, not methodologies.
And can you think of anything useful that falsifiability does?
posted by Aaron Swartz on October 24, 2006 #
I disagree with your point about astrology.
The whole reason people believe in astrology is not because they believe in the “theoretical underpinnings” of it.
They believe because the predictions are sufficiently vague that they apply to everybody all the time.
I’ve heard a story about a prof who hands out personalized astrology readings to everybody in their class, and then has them rate how accurate the prediction is. Most people rate the predictions fairly highly.
He then reveals that everybody has the same prediction.
posted by Eric Gunnerson on October 24, 2006 #
Creationism isn’t simply a claim, it’s a movement. It has its own methodology, its own “researchers”, its own journals. Its methodology provides one of the main lines of attack scientists use when they argue that creationism isn’t science, and falsifiability - or lack thereof in creationalist methodology - happens to be one of the primary arguments they use. For example, the claim that God created Earth 6k years ago - but did it with all the exact evidence that seems to show it’s much older (with fossils already in the ground, etc.) - is internally consistent. It is, however, unfalsifiable and therefore arguably unscientific.
Falsifiability is used all the time. There’s a current debate about string theory in physics with two books out, by Peter Woit and Lee Smolin, arguing that string theory never made good on its promise to deliver testable predictions. You can’t have a testable prediction if the theory’s unfalsifiable.
Falsifiability is one of the arguments that were used to show that Marxism and Freudian psychoanalysis are pseudoscience and not science. Both claimed to be scienctific, both convinced many people (including real scientists). Popper, whom you glibly and absurdly condemn as “an enemy of science”, was the first to realise that psychoanalysis was flexible enough to explain any behaviour whatsoever, and therefore didn’t really explain anything. Marxism as a real economic theory could be and was falsified, but as actually practiced by Marxists it was unfalsifiable.
posted by Anatoly on October 24, 2006 #
posted by on October 24, 2006 #
“Larry Wall once noted that the scientificness of a field is inversely correlated to how much the word “science” appears in its name.”
This is ludicrous. Is Islam more scientific than computing science? And what about Chemistry and Christianity? Neither of them have ‘science’ in the name. Do we thus conclude that their status is equal? If Physics was renamed Phyiscal Science tomorrow would it lose its ‘scientificness’? If Scientology became Hubbardology would it increase in ‘scientificness’?
The reason why older sciences like Physics do not have the word science in their name is simple - when they were born, centuries ago, the notion of science had not yet been clearly articulated (Indeed the more modern meaning of the word science dates back to only the early 17th century).
The name ‘computing science’ is appropriate as the word ‘computing’ denotes an awful lot of activities not all of which can be described as scientific.
Whether or not a field is scientific can be determined only by the nature of its activity and not simply by its name.
posted by Eric on October 24, 2006 #
It’s called a joke.
posted by Aaron Swartz on October 25, 2006 #
Falsifiability is a necessary but by no means sufficient condition for something to be scientific. It seems to me that it is not only a question of whether a field states falsifiable proposition that makes it a science. It also depends on whether someone actually bothers to falsify these propositions which is probably not the case for astrology.
And I don’t get your point about scientists using whatever methods will get them to the truth at all. If the method is such that you will not be able to tell truth from falsehood, then what good does it do you?
posted by Tobias on October 25, 2006 #
For a similar article that makes a run at double blind experiments and how testing takes place nowadays and has more compelling and coherent points to make, read this article. However, while that article also tries to drive in a similar direction, I’m not sure he ever gets there. Yes, there are huge problems with the way many experiments are done, but he’s largely arguing that double-blind experiments are not executed well, rather than arguing against the concept. I think there’s scope for other types of scientific experiments that take into account the ethical problems that human experiments can have. For example, once double-blind experiments are done and something has been released to the public, why aren’t all uses of the drug tracked and any adverse affects immediately noted? Why did more than 25,000 people have to have heart attacks before the drug Vioxx was finally pulled?
Your attack on falsifiability is ridiculous, Aaron. As Eric points out, astrology is actually not falsifiable most of the time because the predictions are designed to be vague, “Something good will happen to you this week.” And what does it matter if the number chosen for statistics is 5%? It’s like arguing that there should be no blood alcohol limit for drunk drivers because 0.08 is an arbitrary number. If you have a case for it being higher or lower or would like to use better statistical methods, fine, make your point. But to use anecdotal evidence of one person who has a problem and to point to genuinely murky scientific results like animal tests (animal tests sometimes predict what happens to humans and sometimes don’t) is not much of an argument. Double blind tests are the best methods we have come up with to tell when something is working or when it isn’t. When you argue against them, YOU are in fact arguing against science and rationality, especially if your only argument is that 5% is an arbitrary cutoff.
I do agree with you that these double blind experiments may be misused if people say that they prove that Benedictin doesn’t cause birth defects. However, they are right about a weaker but still relevant point: the experiments do show that we do not know what caused the problems and it is unlikely that it was this drug (assuming they ran their experiments properly, as detailed in the first link).
Finally, I agree that econometrics has many problems, though I haven’t looked into the matter very deeply. However, the same econometric mathematical models are used by the global warming crowd to reach their doomsday predictions (links: 1, 2, 3). Still skeptical about econometrics?
posted by Ajay on October 26, 2006 #
Absolutely, it directs people to work towards areas that yield real scientific progress, where we can test models against reality, rather than complete speculation. It’s the reason that many may have wasted their lives (from a scientific perspective) if string theory doesn’t pan out. There are certainly legitimate reasons to create non-falsifiable claims, and discuss non-falsifiable issues, but they are not scientific claims/issues at that point.
posted by Jacob Rus on October 27, 2006 #
Yeah, that must be it. The physicists involved in string theorists must all be idiots who never heard of Karl Popper or read the newspaper or watched a television special about themselves. If only they knew they weren’t doing science!
posted by Aaron Swartz on October 27, 2006 #
String theorists do know they’re not doing “science” in the full sense. They know their theory isn’t falsifiable—it doesn’t make ANY predictions about reality that can be observed! That puts it closer to the unfortunate class that Intelligent Design is in. However, two things may set String Theory apart: the fact that some of the brightest minds on the planet are working on it, and that it does predict some things that may one day be testable/observable, but we don’t have the technology to be able to test them yet.
(This is just some thoughts based off a recent Slashdot posting.)
posted by Scott Teresi on October 27, 2006 #
Aaron, in case you didn’t know, string theory is considered controversial for precisely the reasons that Jacob lays out. In fact, as Scott points out, the theorists would probably be the first to admit that they’re not actually doing science, just scouting out the possible solution space for likely answers. The scientific method will then involve testing these hypotheses to see what holds true. Instead of finding out more about this or asking questions, all you offer is ridicule, which is especially jarring considering your ignorance of this subject.
posted by Ajay on October 28, 2006 #
I do think it’s ridiculous that Jacob presumes to tell prominent physicists how to spend their time.
Vague predictions are falsifiable; they’re just less likely to be wrong. And my argument isn’t that 5% is too high or too low, it’s that it’s arbitrary. If a study works only at 6% should we sentence people to death because we’re missing the other 1%. (Actually, what really happens is that people fake the data to get it over the hump, as a recent paper proved.) Evidence doesn’t have clean lines between true and false; we depend on the situation to decide how strong our standards should be.
Who’s arguing against double-blind tests? All I’m saying is that other stuff is evidence too.
An economist doesn’t believe global warming…so I should like economics more? Please.
posted by Aaron Swartz on October 29, 2006 #
Aaron, where do you come up with these obtuse arguments?
Jacob didn’t tell anybody how to spend their time. He pointed out the common criticism that string theory might turn out to be a complete waste of time because they have yet to make predictions that are testable.
It depends on who you’re trying to prove it to. Scientists and intelligent laymen know that astrology is falsifiable and that it mostly comes out wrong. However, the kind of people who believe in astrology are willing to seize on anything that might fit the prediction. “Something good will happen to you this week” = Monday was a bright and sunny day. Since you said falsifiability isn’t much help, we took it for granted you were talking about these dim people. For them, it is unfalsifiable because they are willing to make their definitions so broad as to be meaningless.
5% is not an arbitrary cutoff if comparison of many studies has shown it to be a good cutoff between studies that show repeatable results and those that don’t. I’m guessing that’s how it was arrived at. You setup a false situation by assuming that a whole bunch of studies cluster around that number and that any of them over 5% are then tossed away as a result. There are very likely few studies that even get close to that threshold. Besides, I doubt that doctors actually use a hard threshold like that. What you seem to be primarily talking about is a legal threshold, which seems to be required by the way the law works. I agree with you that if doctors are actually using a hard theshold like that, it doesn’t make much sense, but I doubt that they do.
Presumably you want to use other tests because you see some weakness or failing in double-blind tests that you would like to address. And you did say, “Behind evidence-based science are a bunch of very bizarre claims about what science is and isn’t,” which combined with your attack on falsifiability would appear to be an attack on the method they’re advocating. And I’m not (and I don’t think the others are) saying that there aren’t any problems with that way things are done. I previously pointed out that more effective follow-up studies should be done. What we’re doing is pointing out that your particular criticisms don’t hold much water. Falsifiability is a necessary condition for science, animal tests aren’t that helpful, anecdotal examples are not useful on their own, and case studies can be problematic.
Perhaps you didn’t read those posts properly, let me summarize. He is a former economist who points out that econometrics has failed miserably at what it was devised for, macroeconomics. He then goes on to point out that the same type of models, that you have claimed skepticism about, are used to make the dire predictions coming from the global warming crowd. Do you see how this points out a contradiction for you?
posted by Ajay on October 29, 2006 #
Ajay now insists that theories can be true or not true depending on who reads them, that because so many people use the number 5 it simply must have been chosen through studies (although he hasn’t seen any), and that because economics once was wrong it’s a contradiction to believe that environmental studies are now right. I think Ajay should refrain from posting again until he learns basic principles of logic and evidence.
posted by Aaron Swartz on October 29, 2006 #
In your comments over the last several months, you repeatedly make bizarre statements or seize on irrelevant, minor points. Arguing with you is like arguing with a child, a style of discourse that Jon Stewart ripped in his famous Crossfire episode (though to be fair to you, those guys choose even more trivial points and only speak in repeated bromides).
posted by Ajay on October 30, 2006 #
And arguing with you is like having an email debate with someone who can’t read.
posted by Aaron Swartz on October 30, 2006 #
Dear Aharon, When I was at Faculty, I always thought Popper’s claims on “falsifiability” were “bogus”. The proper term must be “testability” although many aspects of nature are not testable because we lack adequate techniques to test them. However, those unverifiable aspects of nature can be deduced from careful and “repeatable” or “attainable” observations. Mathematics has many times helped us with solutions and explanations to several observed natural events, but it is spot on also that mathematics is a formal science (created by the human mind) that most times is not compatible with reality. For example, through equations theoreticians can deduce the existence of parallel universes, strings, branes, living bubbles, etc.; but those things do not exist in real nature and, if existed, they are not observable, so it is trivial even to consider it as knowledge. Science is a set of knowledge about the Universe ordered and systematized by means of observation, analysis and experimentation, which permits the deduction of principles (theories) and general laws.
posted by Nasif Nahle on November 7, 2006 #
While the M-theory (based upon so much frustrating work with different models of the string theory) cannot be considered true science because of the falsifiability aspect, it is highly improbable that future falsifiable theories that will emerge will be entirely different from it. There are too many “happy coincidences” in this approach to be just junk! An “educated guess” will say that at least chunks of it will be part of any new theory! In fact many alternative explanations are falsifiable and have been eliminated. I think this is a totally new domain of the human knowledge that does not have yet a name (a label), but it is the closest you can get to science. It is science “de jure” if not “de facto”.
posted by Anton Constantinescu on November 14, 2006 #
You can also send comments by email.