Streetwise Professor

November 29, 2009

Code Breaking–or, Broken Code

Filed under: Climate Change — The Professor @ 9:41 am

In a couple of excellent posts, Shannon Love at Chicago Boyz notes that one of the most disturbing revelations resulting from the ripping open of the Hadley CRU’s kimono is the shockingly bad, ad hoc, sloppy, and (fill in own pejoratives here) nature of the computer code underlying the quantitative work that is such an important prop for the entire climate change policy edifice.  Love points out that software is not peer reviewed, and that scientists are for the most part self-taught programmers who do not follow the strict protocols associated with commercial software development.  For an endeavor like that undertaken at Hadley, incremental changes are made on the fly with little documentation, and soon the code resembles a rat’s nest, or an overgrown, weed-choked garden.

The code of the Hadley folks and their confreres (or should it be co-conspirators?) is mainly related to data preparation and analysis.  Many of the tasks it performs are relatively pedestrian in concept; the difficulties arise from dealing with the messiness of the underlying data (and, arguably, the perceived necessity of fitting the data to the theory).

But it does raise questions in my mind about the other major prop of the climate change policy edifice: dynamic climate change models.  These are huge and complex.  I know from much personal experience on simpler but related problems in finance that the kinds of equations they are intended to solve are extremely touchy.  Solution techniques can be very brittle.  Errors can be subtle and hard to catch.

It is my understanding that this code, like the Hadley programs, is written by scientists.  So, my questions: what is the quality of the climate model code?  Is it documented properly?  Has it been tested?  Has it been audited?  By whom?  What confidence can we have in its reliability? (Reliability in the relatively simple sense that it is bug-free, and properly performs the calculations implied by the underlying theories it is intended to implement.  The reliability and completeness of the underlying theories–relying, as they do, on “fudge factor” parameterizations and incomplete characterizations of potentially first order phenomena like clouds–are other issues altogether.)

Print Friendly, PDF & Email


  1. 1) The raw data should be available so other people can run it though their own software.
    2) The software should be open source, and if it is used to produce results presented in a peer reviewed paper the review must include a review of the software.

    Comment by Graeme — November 29, 2009 @ 10:33 am

  2. […] Streetwise Professor: √ In a couple of excellent posts, Shannon Love at Chicago Boyz notes that one of the most disturbing […]

    Pingback by Global Warming, Global Cooling or Global Taxing? - Page 101 - PPRuNe Forums — November 29, 2009 @ 12:19 pm

  3. It is beyond poor science, and has manifested as outright criminal activity. Destroying (let alone preventing release) of Data is punishable by prison. Gifting Public Funds to a known or even suspected criminal is likewise violative of long standing Code. Grants are forms of Public Assistance, and abuse of same by scientist or Professor is no less criminal than the common “Welfare Cheat”. It is too late, and too bad, the deed is done; as these phonies are outed, a new kind of politician may not be reluctant to enforce the people’s Laws.

    Comment by Will Fraser — November 29, 2009 @ 12:46 pm

  4. What do you make of the fact that Svante Arrhenius solved a series of differential equations to forecast a temperature rise of 5-6C for a doubling of CO2 levels, thus almost mirroring the IPCC’s high-end scenarios, back in the 19th century?

    Face it, your attempts to discredit a whole science based on one (minor) branch of it is not going to work.

    Comment by Sublime Oblivion — November 29, 2009 @ 3:42 pm

  5. Let me guess, instead of Arrhenius ‘mirroring IPCC’ isn’t it the other way round?

    Comment by Will Fraser — November 29, 2009 @ 4:38 pm

  6. So by latching on to a minor semantic mistake I presume you admit to having nothing of substance to add?

    Comment by Sublime Oblivion — November 29, 2009 @ 5:13 pm

  7. The bristlecone proxies are crap. Of dozens of growth factors involved in a very short growth period, temperature has assumed a position of ‘basis’ for GMT. I have a problem with the varving foundation of extrapolation. Of eight tenths of one degree (C) rise in the last 157 years (IPCC), if even one tenth is ascribed to CO2 (puhlease), the variations laugh at us all.

    Of all the actual discrepancies and lunacies in the warming agenda, at this point no one should be expected to trust anything from anybody, let alone professional students receiving big money from politicians who can control their “Income”. The b’tards have mooted anything we could possibly disagree about Sublime. (I like the NDE, btw)

    Comment by Will Fraser — November 29, 2009 @ 5:29 pm

  8. Gee, if Arrhenius solved everything in the 19th century, why do we need coupled climate models running on supercomputers? Why hasn’t the prediction been irrefutably supported by the data?

    Just invoking Arrhenius and ignoring all the potential feedbacks, other factors omitted from his models, and the myriad other factors that can affect climate (e.g., land use, ENSO) doesn’t cut it.

    Moreover, your attempt to characterize paleoclimatology as minor certainly wasn’t the norm before all this sh*t hit the fan. It was a central piece of evidence in the various IPCC reports, and was also widely used to build popular support for the view that recent temperature fluctuations were unprecedented and hence unlikely to have resulted from natural forces.

    Finally, it goes well beyond paleoclimatology, inasmuch as the CRU “data” purportedly on “observed” temperatures was utilized in myriad empirical studies. The dubious nature of the processing of that data now calls that research into question. GIGO.

    The ProfessorComment by The Professor — November 29, 2009 @ 7:08 pm

  9. One of the above made comments makes an impression on me that because some forecasts have been made by solving series of differential equations, the outcome should be credible. If yes, it wouldn’t be a credible argument.

    Differential equations, themselves, or any other equations for that matter, don’t forecast phenomenon. The models underlying equations do. Differential equations are just calculation tools. Now, if the underlying model is built to forecast growing temperatures, of course the equations will quantify it. And when it comes to modeling, what you model is what you get.

    If you want to model that the markets always go up, per se, and use the last decade’s historic data to support the conjecture, for example, of course, we can successfully calculate by how many percentage points the markets would go up. But be assured, they will always go up, if that is what we want to believe in. Now, if you believe in it, put your money were your mouth is… 😉

    Now, as far as the conjecture that the earth is “globally warming” is concerned, even if we assume that there is ample data to support such “global” conjecture, it would be useful to know which Law of Thermodynamics claims that over a period of time (let’s say 100 years period of time), earth’s [average?] temperature should remain constant?

    Comment by MJ — November 29, 2009 @ 7:55 pm

  10. MJ–absolutely. Re the value of ODEs (or PDEs), just think of how much money was lost during the last two years based on trades/hedges dictated by PDE solutions. If the underlying model that produces the PDE (as an equilibrium or no-arbitrage outcome) is junk, the output is junk too.

    Models are made to be challenged and tested until they are broken. That’s how you learn things. Instead, people who develop and use them have a tendency to fall in love with them.

    S/O–you should pay attention to MJ. If my surmise is correct 🙂 he is a mathematician who knows very much of what he speaks. I’ll let you guess as to where/when he was trained.

    The ProfessorComment by The Professor — November 29, 2009 @ 8:41 pm

  11. Serious climate scientists generally accept the imperfection of their modeling, and are open to corrections. Ideology and fitting data to models happens in all the sciences, of course — but that’s the whole point of the scientific method and the deployment of organized skepticism. So, on that level, your objections are part of the process.

    But the tone of your posting reveals an epistemological problem of yours: you’re ALSO seeking a relationship between data and model, just a different one than the vast majority of climate scientists currently find plausible. Nothing wrong with challenging their model — but simply griping at problems with their data, or their programs, or their models (no matter how substantively justified the griping) is not a contribution unless you also propose an alternative model.

    The ideologization of climate change models is real — but this post also demonstrates exactly the same (with sign reversed) ideologization. Me, I’d say that we don’t really know what’s going to happen in the climate, and that we don’t understand very well at all the interactions of multiple feedback loops that shape climate change.

    However, basic physics has not been repealed. Carbon dioxide, and other greenhouse gasses, really do cause heat trapping, and trapping heat will increase the world’s average temperature, one way or another: that’s not in dispute except at various irrational fringes. The debate is about how strong such an effect is, what feedback loops accelerate or modulate it, etc. Unless you’re willing to challenge physics, not just climate science, prudence suggests that letting smart people do their best to understand what might be happening is a good policy. And most scientists genuinely care about getting the empirical part right, whatever their positions on policy, and it seems sensible to (cautiously) listen to what a rather substantial majority of those involved in such research have to say, no?

    Comment by PQuincy — November 29, 2009 @ 8:41 pm

  12. PQuincy- Permit me a small disagreement, it is not incumbent upon the challengers to present a different theory, that is only true in politics, as you know. It is abjectly true, so please, no straw men. There is a conflation of purpose, method, and posture that is inescapable to the average critic of AGW, let alone those who spend time and money in pursuit of the research.
    That is one thing, and passion aside, the only thing. Notoriety, fame and money have always been part of the scientific landscape, it is true, all the more reason to engage reason. That the research, the ‘conclusions’, the personalities have lent so much to the ‘buzz’ is so discouraging, and a handicap to science and response to science to be deplored.
    Does one deny the attachment of the charlatan to the ‘debate’? I do not, and to the extent that it clouds one’s understanding, it is criminal. I have serious disagreements with the research, the methods and the ‘projections’. The scurrilous connection between the political and the ‘responsibles’ harms the selling of the proposition, of that I am appreciative, but rue the necessity of a scandal to hinder the scam.
    It is problematice to calculate the received energy of the Sun in toto, yet the AGW claim the ability to calculate a trace gas’ effect on its propagation and dispersion. Everyone seems to want a payoff before the discovery. “Trust me” is for the car lot, not the ‘selling’ of science. In other words ‘consensus’, one of the most dangerous words known to man.

    Comment by Will Fraser — November 29, 2009 @ 10:57 pm

  13. OK, really, my point is pretty simple. There are many models from respected scientific establishments indicating that a doubling of CO2 levels will lead to 5-6C global temperature rise (which as I understand it, are essentially based on solving a massive grid of heat equations in 3D – i.e., just physics, albeit with a large degree of uncertainty when applied to RL because we don’t know the coefficients with a high enough degree of precision). However, paleoclimate evidence concurs with the higher end predictions (5-6C) – our current atmospheric CO2 level of 384+ppm was last observed during the Pliocene 3mn years ago, when global temperatures soared by 3C. Since solar radiation was not substantially different from today then, this means that the kinds of projections made in the higher-end IPCC forecasts (or Arrhenius, for that matter) are likely valid.

    Secondly, you point out that there may be powerful negative feedbacks and moderating factors, which are left out of the models due to their complexity. Yes, noone disputes that. But the same holds for the opposite, e.g. a melting Arctic will reduce albedo and accelerate warming in the High North, which will in turn release more methane from defrosting permafrost. Why do you think the negative feedbacks will be stronger than the positive ones? And what about the problem of global dimming which suggests that the real magnitude of global warming has been underestimated in the past decades because of the artificial cooling effect of increased aerosol emissions like soot and SO2 particulates?

    Comment by Sublime Oblivion — November 30, 2009 @ 1:06 am

  14. I assume PQuincy’s criticism is directed at my comments. Just a few thoughts in response… First, I think my comments are misinterpreted or perhaps I didn’t express them clearly.

    I am trained to be very skeptical when I hear phrases like “Serious … scientists generally accept …” – especially when it refers to none or no criteria to understand who is serious and who is not. I am taught to dismiss such arguments and look for error right there… Science does not need to be “accepted”. It is evident. There cannot be argument about scientific facts. Valid arguments are about hypothesis.

    When I was in training and used to use phrases like “…it is obvious that…” my professors would stop me and say something like “why is it obvious.” If it was obvious, you would state the facts rather than their qualifications.

    In that spirit, while not pretending to be a teacher but merely a better student, perhaps, I am really skeptical to the phrases like, “… the vast majority of climate scientists currently find plausible.” Such phrases raise red flag with me on multiple levels. Two immediate levels are “the vast majority” and “currently find plausible.” The first one is not quantified with any information of who is on either side of the argument and what really the argument is about. The second one is very vague and already has a built-in hedge in it so that it could sound “plausibly” accurate whether the underlying statement is correct or not. While it makes no clear statement, it really makes an impression of a statement. Such statements are good in articles which pay honorarium per the number of words used but other than that explanation, I find their usage unfortunate.

    Scientific paradigms are not settled or should not be settled by the principles of democracy. Science is not politics or should not be treated as such.

    When something is right, everyone sees it. And in order for someone to be open to corrections one needs to first disclose sufficient information on what his/her methodology stands on and what it is that is open for corrections. When the methodology and underlying data are secretive, and the validating argument is “the vastness” of those who agree on that argument, it is ridiculous to contemplate on the “openness for corrections.”

    Furthermore, extrapolation is not a model. It is a tool to produce missing data according to one’s preferences or based on one’s incompetence. Models are validated not against the past data which they are calibrated against but against the future observations.

    Fluid dynamics is governed by Navier-Stokes equations, for example. There are no arguments about this model, though there is an understanding that it has limitations when it comes to turbulence, for example. There is no argument about the underlying model, because the outcome of the equations accurately models a wide spectrum of phenomena and leads us to rather significant interactions with nature which prove the model to be beyond skepticism – airplanes fly from one destination to another according to our controls and submarines merge and also move from one destination to the desired.

    When a certain model breaks down in certain circumstances, we also accept it without politicizing them. History of science has ample examples for it as well. An immediate example that comes to my mind is the breakdown of classic mechanics beyond its applicable limits and the rise of quantum mechanics. The phenomenon the later describes is no less complex than climate. There is a normal argument in the scientific community about some assertions of quantum physics. But the correctness or wrongness is not measured by the number of scientists in each camp.

    Now, if I understand correctly, I either should offer a better model or shut up. But the onus of disproving the validity of someone’s model is not on the readers or listeners. The onus of proving it is on its authors.

    Besides, it is my right to reject to be injected an H1N1 vaccine, if I feel it is not tested and there is no evidence not only that it prevents the decease, but it could also cause other harm. And if the Health Secretary publicly takes the vaccine it is no basis for me to test it on my children, is it? You wouldn’t expect me to take the vaccine or offer a better one, would you? Then why to apply such a methodology when it comes to climate? And why is it that everyone and their aunt has an opinion on climate change and how to cure it? Do they also have such opinions on cancer and its cures?

    I think my epistemology should be clearer now.

    Comment by MJ — November 30, 2009 @ 6:46 am

  15. I appreciate the seriousness of the responses.

    My post suggested something that many years of thinking — not in the natural sciences — has taught me: almost all theories, models, and other human constructs are quickly and cripplingly subject to total skepticism. For this reason, most of the so-called social sciences would be better off, in my judgment, giving up the illusion that they are hypothesis-testing Popperian sciences in the first place. People in descriptive and interpretive fields like history should simply recognize that they do something valuable — interpretation is real and useful — but not ‘scientific’ in the epistemological sense it has in English.

    However, the model of skepticism, empiricism, and rigorous hypothesis-testing we call science is also vulnerable to extreme skepticism when it deals with complex phenomena involving systems, rather than isolable observations. This is because the very process of questioning every single step of every single result for complex models ends with no hypothesis ever reaching a test, in effect. Anyone doubting this analysis should look at the history of those who insist that the ‘theory of evolution’ has been disproven, or not proven. At a sufficient level of skepticism, they’re right — yet I have not even residual doubt that evolution is an powerful model for understanding change in complex biological systems (such as organisms).

    The problem, here, then, is that AGW, like evolution, is likely to remain unprovable and vulnerable to intense skeptical inquiry until the case we have (N=1, after all, for the global climate) tells us the answer. In such circumstances — which are quite different from the circumstances of research that can be modeled in a controlled environment and carried out with controls — controls are impossible, of course, for testing AGW — it is not unreasonable for the non-specialist to consider the consensus of those who are specialists.

    This requires awareness, as several posters say, that specialists can be led astray, and that some of them may be wilfully fraudulent, others too trusting in their own first conclusions. Their claims should therefore be approached with caution. But so should the claims of those who clearly have some motivation outside the phenomena to criticize the specialists. That’s the ‘epistemological stance’ that I detected in the original post above: there’s an unseemly eagerness, frankly, to see error in every deviation from proper behavior, and conclusive proof that an entire intellectual edifice is illusory because a few bricks are defective. Seeing such responses justifies, for me, the same caution that is appropriate for the models being debated in the first place. That’s why a counter-hypothesis is relevant here, where it would not be if dealing with critiques of a laboratory analysis.

    Moreover, if it’s a matter of weighing my caution about several generations of scientists who have specialized in the relevant disciplines, who have gone through our best effort at selection and quality control (for all those efforts’ admitted weaknesses), and whose results trend in one direction despite many open questions, on the one hand, and my caution about another group that energetically criticizes most results in the field without being engaged themselves, that demands “proof” of propositions about complex systems of a level their implicit counter-hypotheses fail to provide, and who often either fail to provide alternatives at all, then the choice becomes for me becomes a matter not of “proof”, but of prudence.

    MJ makes the claim that “Science does not need to be “accepted”. It is evident. There cannot be argument about scientific facts,” and continues with “When something is right, everyone sees it.” Would that it were so!

    I’d prefer greater epistemological humility. I don’t think humans reach that level of certainty negatively or positively in this world. What is ‘evident’ to you is not always evident to me, and vice versa, even at a surprisingly elementary level. True, the world corrects us sometimes, when misplaced confidence turns out to be wrong. Planes crash, bridges fail (I just reread the history of Galloping Gertie, with interest). When we come to complex systems analysis, it seems to me, we are condemned to generally muddle along and continue to disagree, especially in cases with N=1. There is only one case of biological life we can study, so “everyone sees it” is not likely to work soon regarding the way life forms change over long periods of time. AGC is similar: the global climate has an N of 1, is admittedly complex, and that means that theories are likely to remain divergent, not ‘evident.'”

    Thus proof fails — we cannot build theories invulnerable to disproof, yet the skeptics cannot undermine the preponderance of analyses. In such a situation, if we have to act (and being humans, we act one way or the other), the alternative to demanding proof is prudence. Prudence is not science, and being human, is inevitably political — but it’s not pure foolishness, either, and can respond in a risk-weighted way to a cautiously weighted consensus of evidence and analysis. No rushing ahead — but no stopping because one lab or one result turns out to have flaws…if they do.

    Comment by PQuincy — December 5, 2009 @ 10:29 pm

  16. Despite PQuincy’s assertion that issues raised by him are only of epistemological character (if I understand him correctly), he is also addressing issues associated with the actual phenomenon subject to discussion.
    First I would like to address the issue of plains crashing or bridges falling. To the best of my knowledge this happens not because we misunderstand the foundations fluid dynamics or structural mechanics, per se, but because of human error in implementing of what has been well understood in terms of the underlying models. Or, plains fall, because they enter into the regime of turbulence, where we know that the underlying model (of Navier-Stokes) stops being descriptive. Obviously, I am emphasizing Navier-Stokes in order to make a larger point about the epistemological validation of models rather than to narrow down on PQuincy’s particular example.
    The assertion of “global warming” and its presentation as an “established fact” is suspect both from the factual and epistemological perspectives.
    First, coincidentally, I read the following in an article [UK Met Office to Publish Climate Records – ] in today’s CNN online:
    The emails were seized upon by climate skeptics who say they prove scientists are manipulating global warming data to strengthen the argument for man-made climate change.

    But Professor Mitchell told CNN that he didn’t see “any issue whatsoever with the soundness of global mean temperature records.
    “If you look at the land data, the sea surface data temperatures and mean air temperature data, those three records independently show a 0.7 degree warming trend over the past 100 years. That’s all published by the IPCC.””

    I am not in the position to verify or deny the assertion of the 0.7 degree warming. Hence I am not arguing the facts here. Perhaps someone else would be in a position to do it.
    However, I have a reason to believe that it is not physical observation but a statistical. I bet, at some point, due to the scarcity of data, the authors are using extrapolation techniques, which to an unaware person might sound an innocent thing to do.
    Anyone who does such things with proper discipline knows that while extrapolated data, especially on an uneven greed, might look ok, when it comes to the derivatives of the observation (i.e. the rate of the change of the data), it can produce totally irrelevant results. And if I understand correctly, the temporal and spatial derivatives of the temperature of the earth on the grid are most relevant for the prediction of the temperatures on the globe and over time. It might sound that here I am making a technical argument, but since I don’t have the command of the facts, I am merely making an epistemological argument about why such predictions would be perceived with great skepticism.
    Furthermore, I am astonished at its usage of the above referenced argument (from the CNN article) for “global warming” exactly based on epistemological categories – thus I use the term in quotation marks.
    As I have mentioned earlier, I want someone to point me to the Law of Thermodynamics, which claims that over a 100 years period of time the average temperature of the earth should stay the same or decrease. After all, this argument is the fundamental fact, presumably, which the army of the proponents of the theory of “global warming” stands on.
    For all I know, the average temperature of the earth has swung significantly over the several thousands of years (I use the term average here not in its mathematical sense) for which we have direct (from the ice deposits in the Arctic, for example) or indirect evidence.
    This is also related to the epistemology of the issue at hand, since what I am trying to emphasize is how one should not think about things.
    A couple of words about the N=1 argument… I have two legs. If for some reason one of them hearts and a doctor claims that it has to be amputated without making the case and demonstrating that I have a spreading gangrene, it doesn’t mean that I should give him a go ahead. Even though this is an N=1 even in the sense of the argument of PQuincy, I would still expect demonstration of a spreading gangrene beyond my toe with a significant degree of confidence. This is also an epistemological issue.
    Or otherwise, if the average temperature of the earth has increased by 0.7 degrees over a 100 years period of time, it doesn’t mean that in the next 100 years period of time it will increase by another 0.7 degrees and not by 0.1 degrees, for example. Why cannot be some kind of saturation over time? Furthermore, what does mean the term average temperature? Why they don’t claim that the temperature of the entire globe has reason by a certain quantity? The term average might mean that the colder corners of the earth are becoming warmer, for example – much of it happening in the Poles, Greenland and Siberia, for example, of which I have come across some reports. And if it is so, isn’t there another epistemological issue here which would raise skepticism regarding the “globality” of “global warming?”
    Another epistemological issue triggering skepticism is associated with the paradigm of “what to do,” if the apocalyptic predictions are indeed true.
    Over the last 40-50 years, the human population of the earth has doubled. I do not have a good estimate on the rate of increase of the livestock, but would conjecture that by more than twice, since not only it should have kept up with the growth of human population but, furthermore, grown faster, since over the last 50 years the phenomenon of world hunger has been greatly mitigated compared to the first half of the last 100 years. As we all know, we inhale oxygen and exhale carbon dioxide. Therefore the growth of the population of the world (and livestock) is a significant contributor for “global warming,” if we take the argument of the carbon dioxide trapping heat (which I am not arguing, btw). Why not to advocate genocide or wholesale vegetarianism? It might seem that I am making a rhetoric argument, but in reality I am trying to introduce another epistemological argument.
    A few other epistemological arguments can be made (I will leave the physical arguments to the specialists), but I think this is enough for the epistemology.
    And btw, as a citizen, when I hear some scientist making a televised statement in favor of “global warming,” I would like the other scientist, the one that doesn’t belong to the “overwhelming majority” also make his/her case. Somehow, I never see those in the “overwhelming minority” get access to TV or even mainstream printed press. After all, many of them are also tenure professors in leading universities, such as MIT, etc. Btw, PQuincy, denying access to the “overwhelming minority” also raises epistemological skepticism 😉

    Finally, epistemological humility goes both ways… if the skeptics have to demonstrate it, it is only fair to demand that the apocalyptic prophets do it too … 🙂

    Comment by MJ — December 6, 2009 @ 8:56 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress