ClimateGate: Free The Data
November 25, 2009
I wanted to get this out because I’m quickly becoming consumed with other things. But I’ve been following the ClimateGate scandal for coming up on a week now. And every time I turn around it looks worse for anthropogenic global warming.
For those of you who don’t know what I’m talking about, here’s a quick summary:
Someone stole (or possibly leaked) a ton of files and e-mails from the Climate Research Unit
My position on climate change has heretofore been: “I’m not a climate scientist, but there seems to be a pretty significant agreement among those who are that the main points of climate change are solid. The earth is warming and humans are causing it to some degree. The extent to which humans are causing it (do we account for 90% of the change? 50%? 30%?) and what to do about it seems to still be a matter of debate. ”
I’ve read a number of the journal articles on the matter just because I’m interested enough in what is going on and my inclination is to get as close to the data as I can.
Because that’s my thing. Data.
Everything about data is vital to the scientific process. How we collect it, how we analyze it, how we compare different sets… these things are desperately important to good scientific work. When data gets too big, we use statistical analysis to understand it and models to predict what will happen next.
Most importantly, for science to work we need people to check our work. The next scientist down the line should be able to work his way to the same conclusion in order to be able to rely on moving toward the next conclusion. Verification is the heart and soul of the scientific process.
And the process is more important than the result. If you don’t believe me, go read up on Fermat’s last theorem. Pierre de Fermat made a conjecture in 1637 that turned out to be true, but mathematicians couldn’t prove it for over 300 years. That the conjecture was true is important, but how we know it is true is the key part.
That is why I am so pissed off at the scientists at CRU. If you read their e-mails (a good collection of what they say has been collected by Bishop Hill), they spend a ton of energy making sure other people can’t do independent verification of their data. They attack people who disagree with them, not because those people have bad data or use poor process, but because the results are not consistent with the message the CRU scientists are trying to propagate.
Add to that the fact that the CRU e-mails reveal an almost violent disregard for proper scientific peer review in favor of bullying journals into accepting only appropriate papers. And they make no bones about it: Appropriate is defined in relation to the desired result. If the result is different from what they want to hear, they worked tirelessly to politically punish people who found those results.
And we haven’t even started talking about the code.
I have a solution to this, one that I believe is non-partisan and vital to future work:
- If a paper is going to be referenced in an IPCC report, they need to post their all the data, an explanation of the process and the code for the paper where anyone can look at it and verify it.
- Any grants that are offered with federal money should require public access to the data, the process and the modeling code. If “the people” bought the research, we should be able to look at it, not just at some 10 page summary report.
- Any paper used for public policy purposes should hold the same requirement.
In short, this is a call to free the data. We can’t make decisions in the dark. If these guys have done good science, anyone with an appropriate expertise will be able to verify it.
Is this unfair to climate scientists? A violation of intellectual property?
Forgive me if I don’t give a sh**. These guys have crapped all over the scientific method and made a mockery of objective science. This kind of bad PR will take years, possibly decades, to overcome. If they want to keep their data to themselves, they can get a private firm to support their research and stop using their findings to push public policy.
Take note: This does not mean that the conclusions the CRU scientists have come to are wrong. They could be 100% right and still be huge assholes who want to hide their data from everyone else. But we have no reason to believe that they are 100% right because we can’t see the data and we don’t know their process. Just because you cheer the deaths of your opponents doesn’t make you wrong. In the future it’s going to take more to convince me than “But the scientists SAID SO!”
Also, given the blatant and horrific way in which these people have manipulated the peer review process, the “But the skeptics aren’t published in peer reviewed journals” argument is a pretty sh***y line of attack from here on out. Just from reading the e-mails, we can see that:
- That isn’t even remotely true
- Manipulation of the peer review process has been a top priority for these scientists, to the point of intentionally ruining careers and lives.
From here on out, they can have my confidence in their results when I see their data.
Political Math 
November 25, 2009 at 8:13 pm
Brilliant. Libertarian, too. Political Math is the best blog on the web. Seriously. Waiting two or three weeks for a new entry is maddening, but always worth the wait.
Figures don’t lie, but liars figure. Thanks for pointing out statistical and logical fallacies. Making fun of Paul Krugman is just an added bonus.
November 25, 2009 at 8:26 pm
Sorry for the time between posts… I’m slammed at my “real job”. I’ll try to post more after Dec 1 (big deadline).
November 26, 2009 at 5:02 am
Awesome post, as always! I was waiting to see if you were going to tackle this stuff.
Would love to see you discuss the types of computer modeling used in climate change research (ie. general circulation models which use complex partial differential equations to describe the ocean-atmosphere climate system mathematically versus computer projections of climate constructed using data on past climate change which identify mathematical patterns within them and project these patterns into the future). I never do an adequate job of explaining to friends why the latter method is better than the former.
November 26, 2009 at 5:36 pm
Here’s part of my thing with this. Peer review is evil. Peer review is the method by which “soft sciences” get around the scientific method requirement of “reproducibility”. In theory, peer review should require the reviewers to replicate the experimant and verify that the results they got are the same, It shouold weight “sceptics” very strongly and should be rigorous. In practice it amounts to a cult of personality wherein the reviewers make no attempt to reproduce the results and in fact simply accept or refuse a paper based on whether it agrees with their pre-concieved notions.
This has resulted in abominations like Intelligent Design having a peer review process that is comparable in strength to JAMA. It means that all you need to get a “peer reviewed” paper that carries the full force of a scientific paper is to find someone with letters after your name to agree that you got it right.
There clearly needs to be some method by which ecdonomics papers can be published with credibility, but holding the physical sciences to the same standard as psychology is absurd on its face and worse in practise.
November 27, 2009 at 2:07 pm
Sadly, Greenfyre was unavailable for comment.
I think this pic might solve the mystery of our missing global warming alarmist:
November 27, 2009 at 8:26 pm
Cut him some slack… he’s really busy running defense on the matter.
I do have to say that I find it fascinating that in light of new evidence, he has changed his mind on the matter.
Oh… wait. No. In light of new evidence he is holding onto his old belief with the same religious zeal he had before. If you read any of his new stuff, it is about 60% misdirection (along the lines of “but hacking files is illegal!”) and 40% statements of belief (along the lines of “This doesn’t change anything because a scientist told me that it doesn’t”).
November 28, 2009 at 1:30 pm
Hehehe.
Yeah, I’ll give him that hacked e-mails are ethically questionable. You know what else is ethically questionable? Making stuff up and pawning it off as scientific fact.
The reaction of global warming alarmists like Greenfyre tells me that these guys are members of a religious community and not engaged in actual science. I have no problem with people attending whatever church service they want, but it’s another thing when they want to use the power of big government to forcibly convert the rest of us. Which is basically what the climate change hand-wringers want to do.
November 28, 2009 at 11:34 pm
For years I’ve wondered how any scientist could claim objectivity while referring to those whose work has led to differing conclusions as “deniers”. I’m not at all surprised that they rigged the field.
Excellent post, as always. I, too, hate waiting for new ones but they’re always worth waiting for. We understand your real job comes first.
November 29, 2009 at 1:17 pm
[…] Congress uses to justify their position came from the Climate Research Unit. Well, according to Fox News most, if not all, of the data that has come from them may now be found as worthless and […]
December 4, 2009 at 10:35 pm
But we don’t need to find any proof of global warming, we just need to WANT to find it. That’s the way it works!
December 11, 2009 at 5:03 pm
[…] favorite things to talk about for a variety of reasons. The most prominent reason is given in the following blog. (This guy is one of my favorite bloggers. I do not know him personally but my love for math and […]
December 14, 2009 at 9:39 pm
Hiya … I stumbled on your oage by mistake. I was looking in Bing for Antivirus software that I had already bought when I found your site, I must say your site is really cool I just love the theme, its amazing!. I don’t have the time at the moment to totally read your entire site but I bookmarked it and also signed up for your RSS feeds. I’ll back around in a day or two. thanks for a great site.
December 24, 2009 at 4:29 am
The CRU emails only served a purpose to wake up those people which never took a hard look at climatology and how it has been politically perverted and manipulated. You did not need the CRU incident, all you had to do was use some common sense and read the original research papers as so many skeptics have done for years. I began investigating about 5 years ago, and become so disgusted what was presented as “science” when it was more akin to voodoo, manipulation, and blatant lies by the climate alarmists. Seriously, if climatology is science then we need to let astrology into science because its track record at predictions is superior than that of climatologists. Once you get over the fraud that is “climate science” I propose you and others to begin looking at other corrupted areas of science, you will be surprised how far the disease infects the field that is supposed to be above question. Take a look at the work in cold fusion which was suppose to be this enormous fraud and bogus trickery, yet look at all the work done quitely by so many prominent institutions such as Los Alamos Labs, but of course they no longer call it cold fusion but LENR so as not to be stained with the discredited name of “cold fusion”. Next, look at the big bang theory which is just a pile of unproven, fudge factors like Dark Matter and Dark Energy whose sole purpose is to try and make the broken model work. No one listens to the plasma physicists who have put up alternatives that agree more with experimental results whereas the big bang is just mathematical folklore devoted to big bang religon. I hope others begin waking up and start questioning the perversion of science. It reminds me of the state of science a little over a century ago when “science was finally complete”, then shortly after this arrogant nonsense relativity and quantum mechanics made these arrogant people eat crow.
December 24, 2009 at 6:40 am
Derick,
My perspective is that physicists generally do engage in open and honest debate, continue to consider the alternatives, and even admit (albeit grudgingly) when they’ve been wrong. As it stands today, the Big Bang Theory is the best they have and they freely discuss where it falls apart. They claim it to be the accepted history of our universe (multiverse?)- unless some other facts come along to knock it off its perch in favour of a different theory.
So the Big Bang is not a parallel to what’s going on with AGW. Most non-scientists aren’t equipped to reasonably consider the scientific method, peer review, or data integrity. These people make judgements based upon the wisdom of scientists who do this for a living, along with what they can perceive as being plausible.
The idea that the Earth is unnaturally warming because hydrocarbons that have been dug up from deep below the surface and re-introduced into the atmosphere on a large scale over a short time frame does not seem an unreasonable one. That some very smart people who study these things say that’s what’s happening is convincing. So the release of those emails alerted all of us to a potential scam and now others, such as this blog’s author, are now questioning additional factors relevant to following the true science of the matter.
December 30, 2009 at 5:40 am
We will just have to agree to disagree.
On the Big Bang issue I will refer you to the frustration of plasma physicists like Eric Lerner who has vented his frustrations on this issue many times (but silenced).
You used the phrase “knocked of its perch”, how can you say it has not been knocked off its perch? What is your definition of “being knocked of its perch”? Does that mean when your favorite scientist says so? Like a bunch of religous priests in a temple which declare to us what to believe? Even the red shift assumption of Big Bang has been proven wrong as scientists like Halton Arp has shown for years, yet this is silenced in academia. And it is very similar to AGW as the scientific academia is not acting in good faith. Your argument that we accept a seriously flawed theory until something better comes along demonstrates this unbelievable diminishment of science and that we mix religon, politics, and superstition along with science. Let me put forth the purely scientific viewpoint on the origin of our universe: we don’t know yet. And in science, it is totally acceptable to say: we don’t know yet. Why can’t people accept this??? We have some facts but not yet a complete picture. Only in religon, politics, and superstition do we fill in the blanks with nonsense, to make some people feel good or boost egos or plunder taxpayers. I suggest you start seeing how much taxpayer money has been spent looking for dark matter (supposedly far more abundant than regular matter) as an example if you think this is not a serious issue and source of corruption.
You did not address the cold fusion issue which has shown to be a real phenomenom, yet there has been no academic exoneration of this scientific phenomenom. The political issues surrounding cold fusion are immense as billions upon billions have been poured into its cousin “hot fusion” which has not panned out at all. I am assuming you have not studied this issue yet.
Then you state, its not unreasonable to believe that hydrocarbons dug up and re-introduced could cause warming. I am dumbfounded by this statement. I can make similar claims, let me try. It does not seem unreasonable that hydrocarbons dug up an re-introduced could cause more volcanic activity. Geez sounds resonable, we should look into it as a possibility. Really, why? I can provide a crackpot theory as to why, does that mean its reasonable to assume its true and anyone who doesn’t believe it is required to prove me wrong?
By the way, what are a few immediate major problems of the AGW theory. 1.) CO2 the supposed culprit has increased by man yet the 1940s-1970s cooled, further, the highest amount of manmade CO2 has been release in the last few decades but since the late 90s we have cooled again. 2.) The models taking in all the assumptions which the alarmist propose as the CO2 mechanism have been 100% wrong. 3.) The medieval warm period was hotter than it is now, how was that caused by CO2? 4.) CO2 lags temperature because the fluctuations are coming from CO2 contained in the oceans, that is the only way to explain the CO2/temperature relationship seen over thousands of years unless you believe cavemen and further back dinosaurs drove polluting SUVs.
If someone makes any claim, they need to support it, we dont assume its true and its up to skeptics to disprove it, when did that become part of science? The alarmist have made claims and they turned out not true, its not even evident to me as to whether CO2 has any effect at all on earth’s temperature since at 300-400 ppm it is near 3/100th of a percent of the lower atmosphere.
Why do you take this assumption that scientists are not humans, full of errors, full of bias, full of bologny, full of agendas, easily corruptable, and should not be beyond reproach. Your first statement in your reply is an anethema to me, I don’t assume scientists engage in open and honest debate because I have known too many of them over my lifetime. That is why we have the scientific method and we repeat experiments independently. What happened to healthy skepticism, I guess it died in the name of “progress”.
December 30, 2009 at 7:22 am
-AGW-
In the first place, Derek, you seem to make the assumption that I believe AGW is a reasonably proven fact. I do not. Point is, though, what I believe is irrelevant; The science, over time, will filter it out. Politics and human emotions of scientists may delay this, but it will happen. Scepticism is good, but we must also be sceptical of the arguments against.
I did not address cold fusion because the reasons for its failure to achieve acceptance are not a parallel to the AGW fiasco. It’s over on the little back burner, but it hasn’t been dumped.
Two researchers, both following scientific method and both with experimental results that can be duplicated, can still legitimately arrive at differing conclusions. Each can be fervent supporters of their own interpretation and sometimes, many years later, it turns out the stubborn fool was right. “Accepted” is what works experimentally, or even in theory if that’s all there is.
Newton’s theory on gravitation worked so well and its predictive qualities so accurate that it became a “law”…until it got knocked of its perch by Einstein’s theory of general relativity.
December 31, 2009 at 1:01 pm
I could not reply to your last post because the indention seems to have met its limit so I am responding here.
The parallels I have tried to assert are the corruption of key areas of science. The Newton-Einstein response does not fit the argument. I do not see any evidence that Newton was corrupted, that someone directly contradicted his theory saying that at relativistic speeds his model would break down and then Newton responding by telling that person he was a nutjob skeptic.
We are told to buy into the mainstream views on AGW, big bang, and coldfusion yet there is serious flaws with the mainstream views. In fact, there is strong scientific evidence to refute the mainstream view in each case. This is not an evolution from Newtonian to Einstein models, in contrast, the evidence to support the mainstream views is ridiculously weak yet skeptics to the mainstream view in each of these cases are smeared disgustingly. Why? When did smearing people which on honest and good faith demonstrate serious defects in the proposed models become part of the scientific method? I don’t see how this parallels Newton-Einstein at all. Instead, I see coercion to accept a particular view on faith even though the evidence does not support it.
You can call me wrong, misguided, or crazy, but I challenge you to look into these issues. For example, you say cold fusion is over on the back burner but not dumped. I find this perplexing….they have demonstrated fusion at near room temperature, and examples of self-sustaining heat production where the electrical inputs were removed. By your response, I must assume you do not see much importance in these results and that it is appropriate to put this on the back burner, yet we continue to spend billions on “hot fusion” research for decades with nothing to show for it. That is why I stated before we are just going to have to agree to disagree.
December 27, 2009 at 11:49 am
Dear Political Math,
“Because that’s my thing. Data.”
Excellent! I like it.
Since data is your thing, I probably don’t have to warn you about the dangers of “naive empiricism” (hat tip: Nassim Nicholas Taleb). In this context, naive empiricism means gathering data with a view towards measuring something, without having a clear conceptual framework concerning the entities and quantities you are measuring.
In gathering data about the climate, there is an assumption made by the IPCC/AGW believers crowd that natural climate variation is well understood and well quantified, and therefore, measuring artificial (anthropogenic) variation is a simple matter of measuring the amount of deviation from well-established norms. I have come to believe that this assumption is not justified, that the climate has far more natural variation than is generally thought to be the case.
For an outstanding, peer reviewed scientific summary of this problem, and many other problems with the IPCC consensus, I suggest you take a look at the Nongovernmental International Panel on Climate Change (NIPCC) report, which can be found at:
http://www.nipccreport.org/aboutReport.html
The Climategate revelations caused you to conclude that the data supporting the AGW hypothesis was much weaker and sloppier than you thought; you speculate that the AGW hypothesis might still be validated if the climategate problems are fixed. I hope the NIPCC report will convince you that the AGW hypothesis is almost certainly wrong.
February 24, 2010 at 2:10 pm
Hmmz, your blog looks kind of weird on my pocket pc. (HP iPAQ Pocket PC Hx2495b)
February 16, 2011 at 4:27 am
>>
“They attack people who disagree with them, not because those people have bad data or use poor process, but because the results are not consistent with the message the CRU scientists are trying to propagate.”
>>
Reading the emails I got the impression that these guys felt very strongly that the papers and the journal in question were seriously flawed. I’m not saying they were right, but it seemed pretty clear that they viewed the offending material as bad science. Where do you get the impression that this wasn’t the case?
>>
“they spend a ton of energy making sure other people can’t do independent verification of their data.”
“They could be 100% right and still be huge assholes who want to hide their data from everyone else.”
>>
While it is true that they express a clear unwillingness to cooperate with a number of very specific people, can you really draw the conclusion that they are trying to avoid having *anyone* independently scrutinize their work? There is another possible hypothesis here: that they are perfectly fine with genuine criticism or skepticism of their work, but have a problem with a few particular people who’s criticism they perceive insincere, unjustified, or just plain unscientific. They may be completely wrong in that perception. But, it would still lead to a different conclusion than the one you are drawing.
Individual scientists are rarely pure paragons of objectivity. And, sometimes they can act like complete a**holes. In particular, nasty emails are no less common in my circles than they are in any other community of professionals. But, the existence of a large community of scientists, competing individuals and institutions, and multiple journals keep most of that stuff in check (although sometimes things slip through the cracks). What people say in emails and what they do are two different things. The individuals who wrote these emails are definitely important figures in the climate science community. But, they do not exist in a vacuum. Whether or not they feel entitled to change a review process that they see as flawed, doesn’t mean that they will act on it or that they *can* act on it. There is a much larger community of climatologists with different expertise, different affiliations, different political views, and different personalities who will not automatically listen to them, unless there is a science case to support their position.
>>
“Most importantly, for science to work we need people to check our work. The next scientist down the line should be able to work his way to the same conclusion in order to be able to rely on moving toward the next conclusion. Verification is the heart and soul of the scientific process.”
>>
My biggest problem with your article is the insinuation that there is no cross checking in the climate science community and no access to the data. As far as I can tell: there are multiple land, sea, and satellite-based temperature measurements all available online and downloadable. The techniques for building reconstructions are published by many different authors representing many different institutions. Mann’s infamous hock stick stands among a large number independent proxy-based reconstructions. Proxy-data and paleo-climatological reconstructions are readily available. The methods are all published (including the “nature” trick). The system needs to be even better. But, much of these data are readily available. Am I missing something?
August 11, 2014 at 3:04 am
I was wondering if you ever considered changing the structure of your blog?
Its very well written; I love what youve got to say.
But maybe you could a little more in the way of content so people could connect with it better.
Youve got an awful lot of text for only having one or two images.
Maybe you could space it out better?