Thursday, September 17, 2009

Data, Analysis, and Reality

As a scientist, I must always struggle to remember the difference between actual data and inferences based upon those data. This difference is often pretty clear. For example, if I take a picture of a star cluster, the actual data I've collected are counts of the number of photons arriving from different parts of the sky. I then infer that an excess of photons coming from a single point is a star, while an excess coming from an elongated smudge is likely a galaxy. My data are counts of photons, and my inferences are the nature of the object. Clear enough.

Sometimes the line is blurrier. For example, suppose I take a spectrum of the light from a star, where I split the light into its component colors. Typically, I see light from many colors of the rainbow, with perhaps a few specific colors missing (see some examples here). Those missing colors are due to individual elements, like hydrogen, helium, or oxygen, absorbing that light. Most often, that light is absorbed in the atmosphere of the star I'm looking at, and there are diagnostic tools that I can use that look at the lines and tell me the temperature of the star and how much of the given element exists in that star.

A few years ago, I wrote a paper on spectra of a group of white dwarf stars. I noticed that about one quarter of the white dwarf spectra showed the fingerprint of calcium. Calcium is an element that is very easy to see in stars, and it usually indicates the presence of many other elements, like iron and magnesium, that are not as easy to see. This is interesting, because the gravity of white dwarfs is so high that elements heavier than hydrogen and helium should sink out of sight below the stars' surfaces in a matter of years. Those white dwarfs with calcium and other heavier elements must therefore be swallowing this material from somewhere, and there's good reason to think that this material may be from asteroids or comets, the remains of solar systems around these dead stars. So, did 1/4 of my white dwarfs once have planets?

Sadly, the answer is no. I was seeing calcium, all right, but it wasn't in the star. It is probably mixed in with the interstellar gas that occupies the mostly (but not fully) empty space between the Earth and the white dwarf. In the meantime, I'd gone around proclaiming that the "data" showed there was calcium in the star. But my real data were just the number of photons of different colors of light, my spectra; the rest was just interpretation of what I was seeing. And my interpretation was not correct.

In the past couple of days, a couple of astronomy news stories have made the rounds that show that I'm not the only person who confuses data and interpretation. One story involves a cold spot in the Universe. The WMAP satellite has spent five years measuring the temperature of different parts of space using microwaves. Most of the Universe is about 3 degrees Kelvin, the "echo" of the Big Bang, but some parts are a little hotter and some a little cooler. Some of those hot and cool spots are relics of the earliest seeds of galaxies in the Universe, while others are caused by structures in space between us and the distant reaches of the Universe from whence these microwaves came. If the microwaves travel through a bit of space that has more matter than average, they tend to heat up a bit. If the microwaves travel through an empty part of space, they tend to cool off a bit. The technical term for this is the Integrated Sachs-Wolfe Effect.

So, WMAP measured the temperature of each part of the sky. In one part of the southern hemisphere sky, there is a cool spot. Initial indications were that this spot was much cooler than any spot we would expect in the Universe. In fact, to get a spot this cool would require an absolutely humongous empty void of space, a void so large that the Big Bang theory says shouldn't be possible. Had astronomers discovered evidence of a problem with the Big Bang theory? Over the past five years, at least 125 papers have been written that talk about this cold spot. Some just mention it, but many do detailed studies of its shape, temperature, and how it means that exotic physics, like cosmic membranes or superstrings or "textures" or "domain walls" or even portals to other Universes exist at that point in the Universe.

However, there are some people who claim this cold spot is not real. A new paper by Ray Zhang and Dragan Huterer, physicists at the University of Michigan, claims that the spot is, in fact, an artifact of the way the data were analyzed. WMAP indeed found a cold spot in the Universe, but that doesn't mean that the cold spot is a giant void. The huge void is an interpretation of the data when the data are analyzed in a certain way. Analyzed another way, the cold spot could just be a normal-sized void, of which there are many in the Universe, and all of which are expected by the Big Bang theory. I don't think that this paper is the final word; we observational astronomers just need to go and look at the cold spot with other wavelengths of light and see if there's anything there. That will settle the issue. But, again, before we go tossing the Big Bang theory and searching for cosmic textures, perhaps we need to remember what is data (a cold spot in the microwave sky) and what is inference (superhuge voids).

The other story making the rounds where data and inference are being confused by some is yesterday's announcement that European astronomers had determined that one planet around another star, CoRoT-7b, is the first rocky planet found outside our own Solar System.
First, let's look at what we know for certain (or at least can infer with a high degree of certainty). The parent star, with the horrible name of TYC 4799-1733-1, is a pretty normal star. Last year, a satellite called CoRoT was observing this star, and saw the star dim slightly every 20.4 hours. This dimming is consistent with a planet that has twice the diameter of the Earth orbiting the star once every 20.4 hours! (For comparison, the Earth goes around the sun once every 365 days, and the hot little planet Mercury goes around the sun once every 88 days.) Such a planet is very, very close to its parent star, would be blasted by the star's light, and would be a horrible place to visit.

Since its discovery, scientists have been measuring the spectrum of the parent star with a very sensitive spectrograph at the European Southern Observatory in Chile. The spectrum shows that the star is moving back and forth very slightly every 20.4 hours, indicating that the star is being pulled ever so slightly by the planet's gravity. The astronomers have made an amazingly precise measurement, finding that the star moves back and forth at a speed of 12 kilometers an hour ( 7.5 miles per hour). I can run faster than this star moves, and yet we humans can measure that speed from 500 light years away. Just amazing.

Anyway, from that motion, we can use Newton's theory of gravity to calculate the mass of the planet, and it is about 5 times the mass of the Earth. And, since we know the planet's mass, and since we know its diameter, we can calculate its average density. That density is 5.8 grams per cubic centimeter, almost exactly the same as the Earth.

Let's review: the data we have are the number of photons from the parent star we detect on Earth over time, and the spectra of the star taken over time. Very careful data analysis allows us to infer (though with a very high degree of confidence) that the periodic dimming of the star is caused by a planet with a diameter twice that of the Earth, that the star is moving in response to the planet's gravity, that the planet must be five times the mass of the Earth, and that the planet's average density is similar to that of the Earth.

Next, the astronomers claim that this means that the planet must be made out of rocky material, just like the Earth. I think this is probably right, but it isn't as certain. We don't have pictures of the planet, showing that it is made out of rock. It is possible to dream up weird mixes of materials such as iron, rock, water, and/or gases that could give you a planet with the same diameter and mass as CoRoT-7b, but these are pretty contrived. In our Solar System, all the planets with density similar to that of the Earth all are made out of rocks. And, when we look at newly-forming planetary systems, we see signatures of rocks and rock-like dust. So, it seems very likely that CoRoT-7b is made out of rock. But this is not a proven fact; it's a supposition, an inference. We don't know for certain that this planet is made out of rock; that is just the most reasonable explanation we can think of right now.

Further, many of the news stories (notably not the original press release) imply that this means that the planet is solid, or has a solid surface. Again, the planet may well be solid. But we really have no clue. The part of the planet that faces its parent star could be as hot as 2000 degrees Celcius (3600 degrees Fahrenheit) -- hot enough that Earth-like rocks would be molten. The back side of the planet could be cold enough for water ice to form. Could there be a planet with a surface that is half lava, half ice? Or the planet could have some sort of thick atmosphere that gets denser until it just sort of merges into a solid or liquid state, though it is hard to imagine how a planet could hold on to an atmosphere at those temperatures. We don't know. We don't really have a clue. We just have educated guesses of what sort of surface such a planet could have.

So, again, let's remember to separate what we know (a planet about twice the diameter of the Earth, five times the mass of the Earth, and a density similar to that of the Earth) from that which we can only infer very indirectly, such as what the planet may be made out of and what its surface may be like. As we find more and more Earth-sized planets (and I am confident that we will find Earth-sized planets very soon), and as we find Earth-sized planets in orbits far enough from their parent star to be similar to Earth's orbit, we have to remember that we will know very little about what those planets are actually like. Do they have atmospheres? Do they have liquid water? Do they have life? Answers to those questions are likely still a decade or more away. In the next few years, all we will know are the diameters and masses of any Earth-sized planets. The rest will come. But we'll need real data, not inferences.


  1. Great post! The blurry lines between inference, correlation, conjectures, etc are a big part of science, especially (or so it seems) in the world of astronomy. Thanks!

  2. Thanks for noticing the difference between what we know, what we believe with very high certainty, and what we conjecture, or believe with less certainty. These distinctions are important to statisticians (like myself), and what we spend time trying to stress to our students (like physics, chem, bio, psychology, etc. majors) so that they don't look foolish someday when the lines get blurred in their minds. Of course, sometimes that line is blurry for us too, since we tend to have a great deal of faith in the methods we use.

  3. By the way ... I'm Sam Wilcock. Thought it might display my name, rather than my blog nickname. In the future I will have to remember to sign my posts.

  4. Thanks for the post, This article will be used in the October 2009 issue of SACnews, the Offficial newsletter of the Saguaro Astronomy Club of Phoenix, AZ:

    Rick Tejera
    Editor SACnews