I’m sure that you are tired of revisiting your original “hockey stick” paper of a decade ago, but I do have a question (well, actually a few questions).
According to Steve McIntyre, the “short centered” PCA method you implemented can “mine” random noise for “hockey stick” shapes.
Well, I was able to replicate McIntye’s results with a SciLab script that I wrote. When I generated a number of ensembles of “red noise” time-series and computed “short centered” principal components, the probability of obtaining a “hockey-stick-shaped” leading principal component was quite high.
However, when I looked at the eigenvalue magnitudes of my “red noise hockey-sticks”, I noted that they were *much* smaller than the eigenvalue associated with your “hockey stick” leading principal component. I was able to generate “hockey-stick” leading principal components with associated eigenvalue magnitudes of about 0.03 or so, an order of magnitude smaller than the eigenvalue associated with your “hockey stick” leading principal component.
In fact, in order to capture a large fraction of the variance of the red noise, I would have had to include 30 or more “red noise” principal components in the regression step. This is in contrast to the 3 or 4 (IIRC) eigenvalues needed to capture most of the variance in your own data.
So it appears to me that McIntyre’s exercise was more “red herring” than “red noise”.
So Dr. Mann, is my line of thinking correct here? Did McIntyre effectively accuse you of using the PCA method to “cherry-pick” a tiny fraction of your data to fish out a “hockey stick”?
And wouldn’t a quick look at the eigenvalue spectrum of an ensemble of time-series pretty much guarantee that any competent analyst wouldn’t commit the blunder that McIntyre implied that you committed? I mean, wouldn’t a nearly flat eigenvalue spectrum tend to indicate that there isn’t much of a “common temperature signal” to work with?
And if I am correct here and were trying to explain all this to a nontechnical person, would it be fair to say that while the leading principal component may define the “shape” of the hockey stick, its associated eigenvalue defines its “size”?
And would it also be fair to say that McIntyre generated a “tiny ” hockey stick from red noise that he then tried to equate with your “big” hockey stick?