Monday, December 10, 2007

The psychology of astronomy

Although we scientists take pride in our independence of thought and our ability to look at data without those pesky human emotions influencing our interpretation. Sometimes, though, we are reminded that even the best scientist is human, and some very unscientific things influence our work.

Today's example comes from astronomy's most popular preprint server. A preprint is a scientific article that is sent out to other astronomers before it is officially published in an astronomical journal. Sometimes these articles have been approved for publication and are just waiting their turn to be printed, and sometimes these articles have just been submitted for approval (and so may undergo serious revisions along the way, or, in rare cases, may never be accepted for publication). This isn't too different from rock bands that release singles before their album is available, only imagine if some of the singles had not yet been edited, and some of the un-edited songs were so bad that they don't make the final album. Anyway, in ye good olde days, preprints were produced by an astronomer and mailed to astronomy departments around the world. In the last decade or so, these paper preprints have been phased out in favor of "astro-ph", a website of electronic preprints.

Astro-ph is updated daily; papers submitted by 2pm Eastern Standard Time will appear the next day; papers submitted after 2pm will appear the next day. Since the papers are listed in the order they are received, papers that show up at 2:01pm on Monday are at the top of Wednesday's list, while papers submitted at 1:59pm on Tuesday will be at the bottom of Wednesday's list. This has led several people to time their submissions -- on a given day, three to five new papers will have been submitted within 30 seconds of 2pm; the remaining 30 or so papers are spread out over the next 24 hours.

Today, one of those first few preprints is called "The Importance of Being First" by astronomer Jörg Dietrich at the European Southern Observatory in Germany. In the paper, Dietrich presents evidence that the preprints listed at the top of each day's listings are cited more often than papers at the bottom. In other words, preprints put in at 15 seconds past 2pm get used more often than those preprints posted later in the day. Since the value of a paper is often measured in how many times a paper gets cited, this finding is bound to turn some heads.

The question then arises, why is this so? It may be that people who have a paper they consider important purposefully time that preprint's submission, but will post less-important papers at other times. Or it could be that astronomers tire of reading summaries of 30-plus preprints every day, and more astronomers stop reading after the first few. It could also be that those astronomers willing to put in the extra effort to time their submissions are generally self-promoters who try hard to increase the visibility of their work. Or maybe the finding is a statistical fluke. Dietrich put forward most of these suggestions in the paper, but has little evidence for or against any of these ideas.

Just as interesting to me is the reaction of people in our department to this article. Different astronomers believe different explanations, and often the explanation they support depends on their own personal attitude toward the preprint server. One postdoc believes the hypothesis that people try to put better papers first, because he himself has done that. A professor who admits to not paying attention to when he submits papers feels it is a statistical fluke. A third person who gets tired of reading the lists of papers likes the idea that not everyone reads through all 30 papers.

So, maybe there is some truth to all of the hypotheses. Now the question becomes, can we (or even should we) eliminate this bias? Dietrich's preprint argues for a Google-esque sorting scheme where preprints that fit a person's own interests are shown first. The order of the papers could be randomized. Or, if the papers at the top of the list are truly "better" on average than later papers, maybe the list order should be kept as is.

Regardless, I'm curious to see if, in the next few days or weeks, there is a big increase in the number of papers submitted within a few minutes of the daily deadline, hoping for an increase in the number of times their work gets cited.


  1. This comment has been removed by a blog administrator.

  2. This comment has been removed by a blog administrator.