*[or: what keeps me up at night] *

As you will see in the next post, I think Planet Nine is really
out there. But that doesn’t mean you should think it is out there. You might be
skeptical. In fact, I would prefer that you were skeptical. I would prefer that
you read the scientific paper, looking for potential flaws, caveats, and places
where we might have been led astray. But, OK, I understand that the actual
scientific paper is on the weighty side, so, rather than make you wade through
it finding the potential piutfalls, instead, I will give you my top list of
things that might be wrong.

First, though: what is *not*
wrong.

*If *there is an
approximately 10 Earth mass planet on an extremely elliptical orbit in the
outer solar system, it would definitely line up the orbits of Kuiper belt
objects with similarly elongated orbits, it would create Kuiper belt objects with
orbits twisted by 90 degrees to the planets of the solar system, and it would
make objects, like Sedna, which have elongated orbits which don’t ever come
close to the rest of the Kuiper belt. These effects we now know from a general
mathematical analysis and from detailed computer simulations to double-check
the mathematical analysis. This analysis, I am confident to say, is iron-clad.
Astronomers will try to reproduce it (I hope), and they will get the same
results (I know). There truly is no wiggle room here. A 10 Earth mass planet
does exactly all of the things that we are trying to explain.

If I am so confident of this, how could Planet Nine possibly
not exist? Just because Planet Nine can explain all of these effects, it
doesn’t not mean that there is no other possible explanation. We tried to think
of everything that we could, and systematically ruled out alternatives, but
that doesn’t mean that someone else won’t come up with an idea that works.
Again, I hope that there are skeptical astronomers working *right now* to come up with alternatives. I am confident that they
will not come up with them (because I do actually think we considered
everything that could possibly be out there), but, unlike my statement above, I
will definitely not say that this one is iron-clad. Aluminum-clad, maybe.
Stainless steel, perhaps. I’d be willing to bet a lot of money against the idea
that someone will find an alternative explanation for all of the effects that
we are seeing. But it is possible I could lose.

There is one insidious way in which we may have been fooled
into thinking that Planet Nine exists, however, and it is a problem that
permeates all of experimental science. My single biggest worry is that perhaps –
just perhaps -- we have been fooled into seeing a pattern where none exists.
Humans excel at recognizing patterns, even when they are not there (see:
everything single face-on-Mars claim ever, for example). Could we have been
similarly fooled? Absolutely (again: I don’t think we have been, for reasons
detailed in that next post, but is it possible? Of course). Here’s how:

In our analysis, we show that the six most distant objects
that have orbits extending outward from the Kuiper belt all line up within a
100 degree quadrant and all have orbital planes which are tilted away from the plane
of the planets by about 20 degrees (and within 6 degrees of each other). From
some very simple calculations we can show that the probability of these
alignments happening due to chance is only about 0.007%. You could also say
that there is a 99.993% chance that the alignments we are seeing in the outer
solar system are real, and that we are not simply being fooled into seeing a
pattern where none exists.

But, really, if you said that, you’d be wrong. Real
statistics don’t work that way. You can’t, for example, flip 100 coins, realize
that 10 of them in the far upper right corner all turned up heads and then say
“wow; the chances of 10 heads in the far upper right corner is only one in 1024
so something must be happening up there.” And if you flipped all of the coins
again, chances are you wouldn’t get 10 heads in the far upper right corner (in
fact, chances are 1 in 1024). The real statistical question that you should be
asking at the first coin flip is more like “what is the probability that
something that seems anomalous will appear just due to chance?” That question
is essentially impossible to answer, because it relies on knowing what a human
who is looking for anomalous patterns would call anomalous.

There are two good ways to combat these sorts of flawed
statistics. The first I just mentioned above: replicate the experiment and look
for the same result. Your eye and brain might pick out a random pattern from
the noise one time, but the same pattern will not occur again. You might see
different patterns, but that just shows you how easy it is to find patterns in
data.

How do we replicate the finding that the most distant
objects in the outer solar system are unusually aligned? We find more of them.
If the alignment was just random pattern finding by easily fooled humans, it
will quickly go away when the next half dozen objects are discovered. And while
it took 12 years to discover the first 6 aligned objects, the next few should
be much faster, as telescopes and surveys continue to get bigger and more
powerful. One caveat: our computer simulations do not predict that 100% of the
most distant objects will be clustered. Just the vast majority. So finding one
or two outside of the cluster is not the end of Planet Nine. But finding the
next six objects randomly distributed around the sky would be a pretty clear
indication that we fell into the pattern-matching trap and that Planet Nine is
a fantasy. The Planet Nine hypothesis makes strong predictions, and these can
be used to show that the hypothesis is wrong, if it is.

The second way to combat the flawed statistics of pattern
matching is to use your explanation for the pattern that you see to predict
something entirely unrelated to the pattern. In our example of the coins above,
you could hypothesize that the explanation for all of the coins coming up heads
in one spot on the table is that there is a powerful magnetic field in that one
location of the table (ok, I’m not sure how that could make things come up all
heads, but work with me on this one). You could then make predictions. Perhaps
you would predict that a set of ball bearings placed on the table would
systematically roll towards that location. Or something. The key is that the
prediction is something that you don’t know the answer to ahead of time, is not
directly related to your original observation, and has a low probability of
occurring on its own. Here you are not replicating the experiment but are
instead performing a different experiment and predicting a very specific
answer.

Our hypothesis passed this test with flying colors – in my
opinion – with the prediction and subsequent realization of the existence of
what we call the distant twisted orbits (maybe we need a better term for these;
definitely we need a better term for these). But maybe it’s all just a bigger
case of pattern matching? Such an explanation begins to get unlikely, but now
we have a second set of objects that can be replicated, and we’ll all be
watching the results come in.

There is one other more mundane in which we can be “wrong.”
Given the small number of objects and our, currently, limited number of
computer simulations (“limited” here still means ~6 months of super computer
time, but we haven’t had time to precisely explore all of the possible
parameters of Planet Nine), it is possible that our estimates are not precisely
correct. Maybe Planet Nine js 20 times the mass of the Earth instead of 10.
Maybe it is actually a giant terrestrial planet instead of a small gas giant.
Maybe it is slightly further away or tilted into a slightly different plane.
These tweaks are OK. We would still say
that the Planet Nine hypothesis is correct.

If, however, a planet is found beyond Neptune and it is
totally different from what we have described, if it exists, but it fails to
cause the basic gravitational interactions that we have discussed, then, quite
simply, we are wrong. We are not predicting that there is *some* planet beyond Neptune, we are predicting that there is *this* planet beyond Neptune and it is
causing *these* effects on the outer
solar sytem.

Now, finally, is what you can tell your friends and family
to impress them by your informed skepticism of the Planet Nine hypothesis:

“I worry that they have underestimated the likelihood of
finding an intriguing pattern in the orbital data and that they have just been
fooled into finding a pattern where there is none. I am waiting for the next
few discoveries of distant object to see if they, too, have aligned and twisted
orbits like the theory demands. And, for now, I am also running some computer
simulations to check some ideas I have about other ways that the patterns can develop.
Ask me again in six months.”