Is Bigger Better?
No, it’s not what you’re thinking. And it’s not about the CME-CBOT merger either. It’s about models–climate models, in fact.
I have been reading a lot about these models in recent months. They embody a convergence of academic interests–in energy (because global warming and the possible policy responses thereto will have a major impact on energy markets, and are leading to the creation of new commodity markets for carbon), and in numerical methods (I research and teach on numerical methods in finance).
I must say that the models are incredible achievements. They are extremely complex. Indeed, they are far more complex than even the most advanced derivatives pricing models. The typical derivatives valuation model is a 1 dimensional or 2 dimenional single-equation linear PDE. Climate models involve systems of non-linear PDEs. Moreover, climate models must characterize the behavior of complex weather phenomena (such as clouds and precipitation) that cannot be resolved due to their small scale (relative to the scale of the discretized grid on which the equations are solved). Due to their complexity, and the numerical constraints that must be satisfied to ensure stability of the solution, these models must be solved on state-of-the-art supercomputers (whereas even fairly sophisticated derivatives models can be solved on desktop machines.)
That said, there is room for skepticism about these models. The characterizations of small scale phenomena–the “parameterizations”–are often problematic. Important forcings (e.g., solar forcings not related to the intensity of the sun, aerosols) are poorly understood, or missing altogether. Similarly, important feedbacks may be missing or mischaracterized. If there are non-linearities in climate processes (a proposition for which there is considerable theoretical support) observational errors in initial conditions can lead to forecasting errors. Coupled ocean and atmospheric models require computational “fudges” to permit solution, and to give even passably realistic results.
These problems manifest themselves in some serious, but too seldom acknowledged, problems. These models have very little skill in predicting regional climate variations. Their predictions about tropical climate are particularly bad. Moreover, their predictions about tropospheric and surface warming differ substantially from observed data (which either indicates a problem in the models, or in the surface observations, or both). Predictions about precipitation and soil moisture are especially dubious. These empirical rejections of the models’ predictions should give everyone pause when interpreting the dire model-based forecasts of impending climate doom–but this is too seldom the case. Day after day I read in the press of studies predicting dire changes in regional climate–notably regional precipitation–based on models alone. The models’ poor track record in predicting regional climate variations and precipitation is seldom noted in these reports. The modelers have become like the Wizard of Oz–imposing, disembodied authorities speaking down to the little people, shrouded in smoke, illuminated by flashing lights, raging at those who dare question them. Pay no attention to the little man behind the curtain.
The analogy is not exact, but the situation reminds me of macroeconomics in the 1970s. That was the era of the big model with hundreds of equations. Those models were also tours de force, but it eventually became evident that bigger wasn’t better. The big models did not perform well. The real economy was too complex. The Lucas critique also showed that these models could not capture important feedback effects–notably, the maximizing responses of economic agents. The models became progressively more complex, but their performance did not improve dramatically, and as Lucas noted, they faced inherent limitations; you could make them bigger and bigger, but that didn’t mean they got that much better. There are still big models around, but their day has passed.
I wonder if the same might not happen in climate science.
One alternative to these structural models is reduced form time series models, like those described in Dobrovolski’s book on stochastic climate models. These models clearly have their own deficiencies. Notably, they do not specify the causal relations and feedback mechanisms in the way that structural climate models do. However, they do allow parsimonious characterizations of the data. Moreover, they exhibit different behaviors at different levels of temporal and spatial aggregation. These can provide benchmarks against which the structural models can be tested: can these models generate the same time series patterns as observed in the data, and the diversity of behaviors at different levels of aggregation? Through use of Kalman filtering and MCMC methods, these time series models can also handle the observational errors that are inherent in climate data. Thus, although the structural and time series approaches have largely evolved along separate paths, there seems to be considerable benefit to integrating them in some fashion.
