---

AD HOC COMMITTEE REPORT ON THE ‘HOCKEY STICK’ GLOBAL CLIMATE RECONSTRUCTION, AKA "The Wegman Report", PDF (2006)
(Background from the Wall Street Journal)

The Chairman of the Committee on Energy and Commerce as well as the Chairman of the Subcommittee on Oversight and Investigations have been interested in an independent verification of the critiques of Mann et al. (1998, 1999) [MBH98, MBH99] by McIntyre and McKitrick (2003, 2005a, 2005b) [MM03, MM05a, MM05b] as well as the related implications in the assessment. The conclusions from MBH98, MBH99 were featured in the Intergovernmental Panel on Climate Change report entitled Climate Change 20013: The Scientific Basis. This report concerns the rise in global temperatures, specifically during the 1990s. The MBH98 and MBH99 papers are focused on paleoclimate temperature reconstruction and conclusions therein focus on what appear to be a rapid rise in global temperature during the 1990s when compared with temperatures of the previous millennium. These conclusions generated a highly polarized debate over the policy implications of MBH98, MBH99 for the nature of global climate change, and whether or not anthropogenic actions are the source...

...In general, we found MBH98 and MBH99 to be somewhat obscure and incomplete and the criticisms of MM03/05a/05b to be valid and compelling. We also comment that they were attempting to draw attention to the discrepancies in MBH98 and MBH99, and not to do paleoclimatic temperature reconstruction. Normally, one would try to select a calibration dataset that is representative of the entire dataset. The 1902-1995 data is not fully appropriate for calibration and leads to a misuse in principal component analysis. However, the reasons for setting 1902-1995 as the calibration point presented in the narrative of MBH98 sounds reasonable, and the error may be easily overlooked by someone not trained in statistical methodology. We note that there is no evidence that Dr. Mann or any of the other authors in paleoclimatology studies have had significant interactions with mainstream statisticians.

In our further exploration of the social network of authorships in temperature reconstruction, we found that at least 43 authors have direct ties to Dr. Mann by virtue of coauthored papers with him. Our findings from this analysis suggest that authors in the area of paleoclimate studies are closely connected and thus ‘independent studies’ may not be as independent as they might appear on the surface. This committee does not believe that web logs are an appropriate forum for the scientific debate on this issue.

It is important to note the isolation of the paleoclimate community; even though they rely heavily on statistical methods they do not seem to be interacting with the statistical community. Additionally, we judge that the sharing of research materials, data and results was haphazardly and grudgingly done. In this case we judge that there was too much reliance on peer review, which was not necessarily independent. Moreover, the work has been sufficiently politicized that this community can hardly reassess their public positions without losing credibility. Overall, our committee believes that Mann’s assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis....

...Recommendation 1. Especially when massive amounts of public monies and human lives are at stake, academic work should have a more intense level of scrutiny and review. It is especially the case that authors of policy-related documents like the IPCC report, Climate Change 2001: The Scientific Basis, should not be the same people as those that constructed the academic papers.

Recommendation 2. We believe that federally funded research agencies should develop a more comprehensive and concise policy on disclosure. All of us writing this report have been federally funded. Our experience with funding agencies has been that they do not in general articulate clear guidelines to the investigators as to what must be disclosed. Federally funded work including code should be made available to other researchers upon reasonable request, especially if the intellectual property has no commercial value. Some consideration should be granted to data collectors to have exclusive use of their data for one or two years, prior to publication. But data collected under federal support should be made publicly available. (As federal agencies such as NASA do routinely.)

Recommendation 3. With clinical trials for drugs and devices to be approved for human use by the FDA, review and consultation with statisticians is expected. Indeed, it is standard practice to include statisticians in the application-for-approval process. We judge this to be a good policy when public health and also when substantial amounts of monies are involved, for example, when there are major policy decisions to be made based on statistical assessments. In such cases, evaluation by statisticians should be standard practice. This evaluation phase should be a mandatory part of all grant applications and funded accordingly....

5. SOCIAL NETWORK ANALYSIS OF AUTHORSHIPS IN TEMPERATURE RECONSTRUCTIONS
One of the interesting questions associated with the ‘hockey stick controversy’ are the relationships among the authors and consequently how confident one can be in the peer review process. In particular, if there is a tight relationship among the authors and there are not a large number of individuals engaged in a particular topic area, then one may suspect that the peer review process does not fully vet papers before they are published. Indeed, a common practice among associate editors for scholarly journals is to look in the list of references for a submitted paper to see who else is writing in a given area and thus who might legitimately be called on to provide knowledgeable peer review. Of course, if a given discipline area is small and the authors in the area are tightly coupled, then this process is likely to turn up very sympathetic referees. These referees may have coauthored other papers with a given author. They may believe they know that author’s other writings well enough that errors can continue to propagate and indeed be reinforced.

In order to answer such questions about the relationships among authors in the area of temperature reconstructions, we developed two datasets. The first specifically focusing on Dr. Mann was developed by first considering all of his co-authors and then examining the abstracts produced by the co-authors. We focus on Dr. Mann because he is the lead author of MBH98/99 and because he is extremely influential in this area as can be seen by his high degree of centrality. Drs. Bradley and Hughes also appear in the social network, but do not exhibit the centrality that Dr. Mann exhibits. We used the Engineering Compendex database, which is available on the web, to develop the abstract database of his coauthors. Based on the expanded database we examined the co-authors of his co-authors. This first database is Dr. Mann centric with the idea of understanding the relationships among his reasonably close associates. This first database consisted of 43 individuals all of whom have close ties to Dr. Mann. The second database was developed by looking for abstracts in the Engineering Compendex that treated aspects of temperature reconstruction. This second more expanded database contained more authors. In our analysis, we considered only the top 50 and top 75 in terms of numbers of papers published. There were more authors who wrote only one paper in the area and are thus not consistent researchers in the area. We report here the analysis with the top 75 authors, i.e. the 75 most frequently published authors.

...Michael Mann is a co-author with every one of the other 42. The black squares on the diagonal indicate that the investigators work closely within their group, but not so extensively outside of their group. The occasional off diagonal boxes indicate that some investigators have joint papers with investigators outside of their immediate group. The order of the authors on the vertical and horizontal axes is the same. Unfortunately, there is overprinting on the horizontal so that individual authors are not readable. However, it is immediately clear that the Mann, Rutherford, Jones, Osborn, Briffa, Bradley and Hughes form a clique, each interacting with all of the others. A clique is a fully connected subgraph, meaning everyone in the clique interacts with every one else in the clique.

...As mentioned before, Michael Mann is his own group since he is a co-author with each of the other 42. The cliques are very clear in this layout. In addition to the Mann-Rutherford-Jones-Osborn-Briffa-Bradley-Hughes clique there are several others that are readily apparent. They are Rind-Shindell-Schmidt-Miller, Cook-D’Arrigo- Jacoby-Wilson, Folland-Vellinga-Allan-Knight, Stahle-Shugart-Therrell-Druckenbrod- Cleveland, Sangoyomi-Moon-Lall-Abarbanel, and Clement-Zebiak-Cane. The last cluster is somewhat of the miscellaneous cluster of people who had published with Michael Mann, but not much if at all with each other.

...Other authors of the remaining 42 that have some degree of centrality are Cook, Rutherford, and Lall.

...There are some interesting features. Although Michael Mann remains an author with high centrality, Tett, Briffa and Cook emerge as belonging to their own cluster and they also exhibit high centrality. Schweingruber and Collins also appear to have relatively high centrality. One interesting observation is that although Tett is fairly central, he has no direct linkage to Mann. Similarly the Gareth Jones-Allen-Parker- Davies-Stott clique also has no direct linkage to Mann. There are two Joneses Gareth Jones is not the same person as the person previously labeled as Jones.

...Several other interesting details emerge. The clique Claussen-Ganopolski- Brovkin-Kubatzki-Bauer is completely isolated from the other researchers in the area. Similarly, the Ribe-Gimeno-Garcia-Herrera-Gallego clique and the Arbarbanel- Rabinovich-Tsimning-Huerta-Gibb clique are nearly isolated with only linkages to Mann in the first case and linkages to Mann and Lall in the second case.

...The social network analysis of authors’ relations suggests that the “independent reconstructions” are not as independent as one might guess. Indeed, the matrix outlined in Figure 5.8 illustrates the proxies that are used more than one time in twelve major temperature reconstruction papers. The black boxes indicate that the proxy was used in a given paper. It is clear that many of the proxies are re-used in most of the papers. It is not surprising that the papers would obtain similar results and so cannot really claim to be independent verifications....