Nov 24 2009
Where Is CRU’s Raw Temp Data?
Update: More on the crappy CRU code at Hot Air (maybe I won’t have to dig into it after all!) – end update.
One question still remains after all the email and document dumps – where is the raw climate data CRU used to create the global warming false alarm? We know that Phil Jones claims he ‘accidentally’ deleted it (but in emails planned to do so if he was forced to comply with FOI laws). And I find it interesting that no real data was contained in the dump – just reports, proposal, code and data products.
I ran across an interesting “readme” file in the documents. A “readme” file usually contains information that users of programs need to run the programs and listing known bugs, written by the programmers. But in the case of CRU the “readme” files are sometimes detailed logs by the developers (the Harrry_READ_ME file provides a wonderful view inside the crappy code upon which the future of humanity apparently rests). One such file I discovered under the original zip path was /documents/EMULATE-corrections (which you can read here).
In this file we find the sources for the Mean Sea Level Pressure (MLSP) data, which I presume comes from weather stations that also collect temperature values. So the sources for the MSLP are probably the sources for much of the land data CRU had (or has).
Excellent analyzing and reporting on this AJ! Thank you.
Frogg1,
Many thanks – more to come!
Thank you for pounding this, AJ!
On another note, I found it interesting that in Alabama, I could only find ONE story on Sarah Palin’s visit to a Birmingham mall yesterday at al.com. And get this: NO NUMBERS were mentioned. Just a sentence that the mall was packed and they were chanting her name. On Sunday, however, they wrote that 1800 people stood in line to get one of the 1000 wristbands for a guaranteed signature. 1800, the only number they mentioned. Had pictures from SUNDAY, NOT MONDAY! And NEITHER of these stories was on the front page of the internet version-I had to search to find the story! the comments on the paper’s site were foul.
Also, did you see Robin of Berkeley’s excellent commentary at AmericanThinker.com on the Wilding of Sarah Palin? It’s a keeper.
Once again, we’re up against the Democrat voting media. CBS has that good report on their website, but will we see it on the evening news? Seems doubtful:
http://tinyurl.com/ydqe5v2
Fox has it, but we can’t follow Fox, the White House said so.
But if we ask these guys for their data in congressional investigations, and they can’t produce it, that might just resonate and demand scrutiny.
The obstruction that was going on was just insane. You can read what I think is the best timeline I have seen to date in this article and I know at least one side of the story is accurate because I was “watching” it unfold on the Internet on McIntyre’s blog as it happened.
But that article linked above is the most complete timeline of an actual case of evasion of FOIA I have seen to date with information from both sides presented along with what was going on in the background now that we have the Jones emails.
Just amazing. What is also amazing is the discovery of simple facts like grid cells that contain no recording stations being given an anomaly from surrounding regions and the grid cells get assigned a HIGHER anomaly than the surrounding cells from which the anomaly is derived. So imagine you have a cell with no data but the cells surrounding it are +1C over “average”. You don’t then give the cell with missing data a +2C anomaly! But this is what was found in the HADCRUT data.
In order to find the reason why that happened, a researcher asked for the raw data and the program code that created that “fill” value so he could possibly discover why and discover if other such errors existed in the data output from CRU.
He never got it.
A great read.
CP,
Knowing you I suspect you have the files. Did you look at opnormals.f90? If so did you look at the subroutine “MergeTwo”. If you don’t have the file send me an email and I will send you a copy.
If I read this subroutine right it ensures any New Station (C) required to fill holes always takes the greater of the two estimates from any nearby real stations (Station A and Station B)
Yes, I have the files. I have not looked at the opnormals.f90 file but will have a look.
My programming background is more in C and Python with a few other more obscure languages tossed in but I never picked up fortran (never had to) so it might take me a bit to get through it.
But in any case, Jones told Hughes that it should never happen that a “fill” value should be greater than any of the surrounding cell values. It can be the same as one of them (the greater of two as you say above) but it can not be greater than any of them.
thanks to ivehad it for directing us to American Thinker to read Robin of Berkeley http://www.americanthinker.com/2009/11/the_wilding_of_sarah_palin.html It’s a great blog post and be sure to click on “Comments” at the bottom and read the also great comments. thanks, ivehadit.
AJ this is an interesting read.
Iowahawk Geographic: The Secret Life of Climate Researchers
That code gives me a headache. I guess there is a reason I don’t write in Fortran.
But that score comparison is interesting though I don’t understand what it is doing well enough to figure out if I should care. I would have to go all the way back to the start of the program to make sure I understand what all the arrays are, etc. in order to understand what it is comparing and multiplying to come up with the score to decide what it is actually writing out.
Robin of the Americanthinker was terrific.
CP,
Uncle! I will do the analysis (sort of boring not all that complicated)
You’re welcome, Whatever.
Sorry, Whomever!
No, isn’t all that complicated, just need to spend time on it and I don’t have it. Grandma’s in town for the holidays, kids out of school starting tomorrow … I won’t have any time until next week when things return to “normal” around here.
Happy Thanksgiving to you and yours.
I am not a computer programmer. So excuse my ignorance if this is dumb question.
Why did they use a program based in Fortran and not C++? Isn’t that a better language for this typ of project?
While not a computer programer, I have had some experience entering data into a statistical program. I am not an expert by any means but am at least a competant layman and have a fair understanding of basic statistical method
This data is garbage. A 15 year-old could do better than this.
If I’d turned this in as a project in grad school I’d have probably been thrown out of the class.
These people need to not only be removed from their jobs they need to be laughed at. Nothing is more humiliating to a professor, especially a leftist professor from the UK than to be laughed at.
Fai,
The answer is the PhDs. They use a tool called IDL to process and graph data (it is so antiquated). There are few older scientists who use modern programming languages, and that code looks likes some nightmare out of the 1980s.
CP, Enjoy your Thanksgiving!