Thursday, September 06, 2007

Reputations and Science

A comment from Tom B. on my last post has had me thinking about scientists' reputations and their impact on science. A similar topic was raised in a recent issue of Scientific American, when they published an article by a scientists who does not believe (despite overwhelming evidence) that the HIV virus causes AIDS; the article was on a different topic. In letters to the editor, people registered their opinions on this publication, including at least one letter which stated that the scientist's heretical ideas on AIDS should not affect how we view his work in other topics.

Let's start with a review of what I'm going to call idealized science. In an idealized science, a scientist (call him Alfred) develops a hypothesis (say that the mass of a certain type of star should be 1.5 times the mass of the sun). Alfred then devises an experiment to measure the mass of the star, and he measures that the star is indeed 1.5 times the mass of the sun. Alfred then writes a paper detailing his hypothesis, his experiment, and his results confirming his hypothesis. Now, scientists Betty, Charlie, and Dana read Alfred's paper. They check his work by repeating Alfred's experiment, and they get the same answer. Maybe Dana has another experiment that can also determine the mass of the star. She runs that experiment and gets 1.5 times the mass of the sun. Alfred's experiment is confirmed, and his idea gets credit. (Or, perhaps on repeating his experiment, Betty, Charlie and Dana all measure 1.1 times the mass of the sun; this would suggest Alfred made an error. Or maybe they all get Alfred's measurement using his experiment, but Dana's second experiment gets a different answer; this would suggest that further investigation needs to be done.)

In idealized science, it doesn't matter whether Alfred is famous or unknown, young or old, a student or a professor. And, if Alfred then goes to work on a different hypothesis, his success or failure on this first hypothesis doesn't matter -- the new result should stand on its own.

Now, in real life, this is not what happens; let's look at what I'll call real science. In real science, the same basic method as above is followed. I'll outline a few scenarios and mention what I suspect would happen.

  1. In the first scenario, Alfred makes a mistake (maybe he errs in a math calculation). Alfred's mistake is uncovered by Betty, Charlie and Dana. He publishes a retraction or an "erratum" admitting his mistake. When he does another experiment later, Alfred's second (and third and fourth) experiments are confirmed by further testing. In this scenario, most people will forgive Alfred's first error. When Alfred does a fifth experiment, most scientists will tend to believe the results, because Alfred has been right quite often. Yes, the results should be (and probably will be) checked, but many people will believe him before further testing, since Alfred is such a careful scientist.
  2. In a second scenario, Alfred is just a sloppy guy, and makes mistakes in several different experiments. Later experiments show Alfred made errors, Alfred admits to the errors, but he continues to be sloppy. When Alfred does later experiments, other scientists will be very sceptical at first. Yes, later testing may prove Alfred right, but until that testing is done, few will believe Alfred.
  3. In a third scenario, Alfred is a new scientist, and Dana is a well-respected scientist people tend to believe. If Dana contradicts Alfred, Alfred's results are in trouble, even if it is Dana who made the mistake. Alfred can still be vindicated, but he will have to do a lot of work to show why Dana is wrong to criticize Alfred. This is a very tricky scenario. If Alfred's work is actually wrong yet he insists that he is right, his reputation will suffer. If Alfred is right but, in proving Dana wrong makes her angry (maybe by a poor choice of words in a paper), Alfred is not helping himself, and his future work will be viewed with scepticism.

I can think of many other scenarios in real science -- maybe Charlie and Alfred had an argument long ago and Charlie tries to sabotage Alfred's career by spreading rumors that Alfred's work isn't reliable. Maybe Alfred is making up his data to bolster his points. In all of these scenarios, there are two major differences from idealized science. First, people are involved, meaning that human error and emotions play a role, even if unconsciously. Second, scientists in real science have memories. They use past performance to gauge future success.

And I think that real science has advantages and disadvantages when compared to idealized science. That memory factor is both an advantage and disadvantage. I I know that Alfred is constantly sloppy or occasionally invents data, then I have every reason to be wary of his new work. If Alfred is not acting in the good faith that idealized science requires, then it is wasteful of time and resources to be constantly following up his shoddy work. And if Alfred is a tremendously careful scientist, then it is wasteful of time and resources for many people to be checking on every detail of his every experiment (assuming, of course, that these are not situations of life and death). If the careful Alfred does make a mistake, it likely will be caught, maybe even by Alfred himself. And if the sloppy Alfred does happen to produce some good work, then the fact that others may disbelieve it is a result of Alfred's own actions, not necessarily bad science on the part of other people.

My opinion is that we cannot (and should not) remove the human element from real science. When awarding resources (such as telescope time), people who do careful research and important research should get those resources; sloppy researchers should not. Maybe the sloppy researchers can do good work if given resources, but if they don't have a good track record, then it is reasonable to expect that they will not suddenly reform. So, in short, I don't think we can and should neglect a person's record when considering new ideas from them.

Real science does have drawbacks. Like everyone, scientists have friends and enemies, hold grudges, and have things they feel passionate about and things they find boring. These human traits do work their way into science, into how resources and money are allocated, into whether somebody's work is believed or rejected. And that is very non-scientific and can be counter-productive. But then, competition spurs people to work harder, and friends help each other out on tricky problems. So, like in the non-scientific world, society has advantages and disadvantages.

In short, scientists do have memories and biases. These biases can be both good and bad in the cause of advancing science. It would be great to minimize the bad biases, and, in theory, to dispose of all biases. But as long as people are involved in science, these biases will undoubtedly be present.

No comments:

Post a Comment