A model‐data intercomparison of CO2 exchange across North America: Results from the North American Carbon Program site synthesis Christopher R. Schwalm,1 Christopher A. Williams,1 Kevin Schaefer,2 Ryan Anderson,3 M. Altaf Arain,4 Ian Baker,5 Alan Barr,6 T. Andrew Black,7 Guangsheng Chen,8 Jing Ming Chen,9 Philippe Ciais,10 Kenneth J. Davis,11 Ankur Desai,12 Michael Dietze,13 Danilo Dragoni,14 Marc L. Fischer,15 Lawrence B. Flanagan,16 Robert Grant,17 Lianhong Gu,18 David Hollinger,19 R. César Izaurralde,20 Chris Kucharik,21 Peter Lafleur,22 Beverly E. Law,23 Longhui Li,10 Zhengpeng Li,24 Shuguang Liu,25 Erandathie Lokupitiya,5 Yiqi Luo,26 Siyan Ma,27 Hank Margolis,28 Roser Matamala,29 Harry McCaughey,30 Russell K. Monson,31 Walter C. Oechel,32 Changhui Peng,33 Benjamin Poulter,34 David T. Price,35 Dan M. Riciutto,18 William Riley,36 Alok Kumar Sahoo,37 Michael Sprintsin,9 Jianfeng Sun,33 Hanqin Tian,8 Christina Tonitto,38 Hans Verbeeck,39 and Shashi B. Verma40 Received 23 November 2009; revised 23 July 2010; accepted 29 July 2010; published 9 December 2010. 1Graduate School of Geography, Clark University, Worcester, Massachusetts, USA. 2National Snow and Ice Data Center, University of Colorado at Boulder, Boulder, Colorado, USA. 3Numerical Terradynamic Simulation Group, University of Montana, Missoula, Montana, USA. 4School of Geography and Earth Sciences, McMaster University, Hamilton, Ontario, Canada. 5Atmospheric Science Department, Colorado State University, Fort Collins, Colorado, USA. 6Climate Research Division, Atmospheric Science and Technology Directorate, Saskatoon, Saskatchewan, Canada. 7Faculty of Land and Food Systems, University of British Columbia, Vancouver, B. C., Canada. 8School of Forestry and Wildlife Sciences, Auburn University, Auburn, Alabama, USA. 9Department of Geography and Program in Planning, University of Toronto, Toronto, Ontario, Canada. 10Laboratoire des Sciences du Climat et de l’Environnement, CE Orme des Merisiers, Gif sur Yvette, France. 11Department of Meteorology, Pennsylvania State University, University Park, Pennsylvania, USA. 12Center for Climatic Research, University of Wisconsin‐Madison, Madison, Wisconsin, USA. 13Department of Plant Biology, University of Illinois‐Urbana Champaign, Urbana, Illinois, USA. 14Department of Geography, Indiana University, Bloomington, Indiana, USA. 15Atmospheric Science Department, Lawrence Berkeley National Laboratory, Berkeley, California, USA. 16Department of Biological Sciences, University of Lethbridge, Lethbridge, Alberta, Canada. 17Department of Renewable Resources, University of Alberta, Edmonton, Alberta, Canada. 18Environmental Sciences Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA. 19Northern Research Station, USDA Forest Service, Durham, New Hampshire, USA. 20Joint Global Change Research Institute, Pacific Northwest National Laboratory and University of Maryland, College Park, Maryland, USA. Copyright 2010 by the American Geophysical Union. 0148‐0227/10/2009JG001229 21Department of Agronomy and Nelson Institute Center for Sustainability and the Global Environment, University of Wisconsin‐ Madison, Madison, Wisconsin, USA. 22Department of Geography, Trent University, Peterborough, Ontario, Canada. 23College of Forestry, Oregon State University, Corvallis, Oregon, USA. 24ASRC Research and Technology Solutions, Sioux Falls, South Dakota, USA. 25Earth Resources Observation and Science, Sioux Falls, South Dakota, USA. 26Department of Botany and Microbiology, University of Oklahoma, Norman, Oklahoma, USA. 27Department of Environmental Science, Policy and Management and Berkeley Atmospheric Science Center, University of California, Berkeley, Berkeley, California, USA. 28Centre d’études de la forêt, Faculté de foresterie, de géographie et de géomatique, Université Laval, Québec, Quebec, Canada. 29Argonne National Laboratory, Biosciences Division, Argonne, Illinois, USA. 30Department of Geography, Queen’s University, Kingston, Ontario, Canada. 31Department of Ecology and Evolutionary Biology, University of Colorado at Boulder, Boulder, Colorado, USA. 32Department of Biology, San Diego State University, San Diego, California, USA. 33Department of Biology Sciences, University of Quebec at Montreal, Montreal, Quebec, Canada. 34Swiss Federal Research Institute WSL, Birmensdorf, Switzerland. 35Northern Forestry Centre, Canadian Forest Service, Edmonton, Alberta, Canada. 36Climate and Carbon Sciences, Earth Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California, USA. 37Department of Civil and Environmental Engineering, Princeton University, Princeton, New Jersey, USA. 38Department of Ecology and Evolutionary Biology, Cornell University, Ithaca, New York, USA. 39Laboratory of Plant Ecology, Ghent University, Ghent, Belgium. 40School of Natural Resources, University of Nebraska‐Lincoln, Lincoln, Nebraska, USA. JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 115, G00H05, doi:10.1029/2009JG001229, 2010 G00H05 1 of 22 [1] Our current understanding of terrestrial carbon processes is represented in various models used to integrate and scale measurements of CO2 exchange from remote sensing and other spatiotemporal data. Yet assessments are rarely conducted to determine how well models simulate carbon processes across vegetation types and environmental conditions. Using standardized data from the North American Carbon Program we compare observed and simulated monthly CO2 exchange from 44 eddy covariance flux towers in North America and 22 terrestrial biosphere models. The analysis period spans ∼220 site‐years, 10 biomes, and includes two large‐scale drought events, providing a natural experiment to evaluate model skill as a function of drought and seasonality. We evaluate models’ ability to simulate the seasonal cycle of CO2 exchange using multiple model skill metrics and analyze links between model characteristics, site history, and model skill. Overall model performance was poor; the difference between observations and simulations was ∼10 times observational uncertainty, with forested ecosystems better predicted than nonforested. Model‐data agreement was highest in summer and in temperate evergreen forests. In contrast, model performance declined in spring and fall, especially in ecosystems with large deciduous components, and in dry periods during the growing season. Models used across multiple biomes and sites, the mean model ensemble, and a model using assimilated parameter values showed high consistency with observations. Models with the highest skill across all biomes all used prescribed canopy phenology, calculated NEE as the difference between GPP and ecosystem respiration, and did not use a daily time step. Citation: Schwalm, C. R., et al. (2010), A model‐data intercomparison of CO2 exchange across North America: Results from the North American Carbon Program site synthesis, J. Geophys. Res., 115, G00H05, doi:10.1029/2009JG001229. 1. Introduction [2] There is a continued need for models to improve con- sistency and agreement with observations [Friedlingstein et al., 2006], both overall and under more frequent extreme climatic events related to global environmental change such as drought [Trenberth et al., 2007]. Past validation studies of terrestrial biosphere models have focused only on few models and sites, typically in close proximity and primarily in forested biomes [e.g., Amthor et al., 2001; Delpierre et al., 2009; Grant et al., 2005; Hanson et al., 2004; Granier et al., 2007; Ichii et al., 2009; Ito, 2008; Siqueira et al., 2006; Zhou et al., 2008]. Furthermore, assessing model‐data agreement relative to drought requires, in addition to high‐quality observedCO2 exchange data, a reliable droughtmetric as well as a natural experiment across sites and drought conditions. [3] Drought is a reoccurring phenomenon in all climates [Larcher, 1995] and is characterized by a partial loss in plant function due to water limitation and heat stress. For terrestrial CO2 exchange, drought typically reduces photo- synthesis more than respiration [Baldocchi, 2008;Ciais et al., 2005; Schwalm et al., 2010], resulting in decreased net carbon uptake from the atmosphere. In the recent past drought conditions have become more prevalent globally [Dai et al., 2004] and in North America [Cook et al., 2004b]. Both incidence and severity of drought [Seager et al., 2007] as well as heatwaves [Meehl and Tebaldi, 2004] are expected to further increase in conjunction with global warming [Houghton et al., 2001; Huntington, 2006; Sheffield and Wood, 2008; Trenberth et al., 2007]. [4] In this study, we evaluate model performance using terrestrial CO2 flux data and simulated fluxes collected from 1991 to 2007. This timeframe included two widespread droughts in North America: (1) the turn‐of‐the‐century drought from 1998 to 2004 that was centered in the western interior of North America [Seager, 2007] and (2) a smaller‐ scale drought event in the southern continental Untied States from winter of 2005/2006 through October 2007 [Seager et al., 2009]. During these events Palmer Drought Severity Index values [Cook et al., 2007; Dai et al., 2004] and pre- cipitation anomalies [Seager, 2007; Seager et al., 2009] were highly negative over broad geographic areas. Ongoing eddy covariance measurements [Baldocchi et al., 2001], active throughout the aforementioned drought periods, provided flux data across gradients of time, space, season- ality, and drought. We use these data to examine model skill relative to site‐specific drought severity, climatic season, and time. We also link model behavior to model architecture and site‐specific attributes. Specifically, we address the following questions: Are current state‐of‐the‐art terrestrial biosphere models capable of simulating CO2 exchange subject to gradients in dryness and seasonality? Are these models able to reproduce the seasonal variation of observed CO2 exchange across sites? Are certain characteristics of model structure coincident with better model‐data agree- ment? Which biomes are simulated poorly/well? 2. Methods 2.1. Observed and Simulated CO2 Exchange [5] Modeled and observed net ecosystem exchange (NEE, net carbon balance including soils where positive values indicate outgassing of CO2 to the atmosphere) data were analyzed from 21 terrestrial biosphere models (Table 1) and 44 eddy covariance (EC) sites spanning ∼220 site‐years and 10 biomes in North America (Table 2). All terrestrial bio- sphere models analyzed simulated carbon cycling with process based formulations of varying detail for component carbon fluxes. Simulated NEE was based on model‐specific SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 2 of 22 T ab le 1. S um m ar y of M od el C ha ra ct er is tic s M od el A ttr ib ut e M od el A gr oI B IS B E P S B io m e‐ B G C C an ‐I B IS C N ‐C L A S S D L E M D N D C E co sy s E D 2 E D C M T em po ra l R es ol ut io n H al f‐ ho ur ly D ai ly D ai ly H al f‐ ho ur ly H al f‐ ho ur ly D ai ly D ai ly H ou rl y H al f‐ ho ur ly M on th ly V eg et at io n P oo ls 4 4 7 3 4 6 3 9 9 8 S oi l P oo ls 7 9 4 7 3 3 9 9 4 5 S oi l L ay er s 11 3 1 7 3 2 10 15 9 10 C an op y P he no lo gy P ro gn os tic S em ip ro gn os tic P ro gn os tic P ro gn os tic P ro gn os tic S em ip ro gn os tic P ro gn os tic P ro gn os tic P ro gn os tic P ro gn os tic N itr og en C yc le Y es Y es Y es Y es Y es Y es Y es Y es Y es Y es G ro ss P ri m ar y P ro du ct iv ity (G P P ) E nz ym e K in et ic M od el E nz ym e K in et ic M od el S to m at al C on du ct an ce M od el E nz ym e K in et ic M od el E nz ym e K in et ic M od el S to m at al C on du ct an ce M od el L ig ht U se E ff ic ie nc y M od el E nz ym e K in et ic M od el E nz ym e K in et ic M od el L ig ht U se E ff ic ie nc y M od el H et er ot ro ph ic R es pi ra tio n (H R ) F ir st or G re at er O rd er M od el A ir T em pe ra tu re S oi l T em pe ra tu re P re ci pi ta tio n S oi l M oi st ur e E va po ra tio n S oi l C ar bo n S oi l N itr og en S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n F ir st or G re at er O rd er M od el F ir st or G re at er O rd er M od el D ec ay M et ha ne A ir T em pe ra tu re S oi l T em pe ra tu re L itt er an d S oi l C ar bo n S oi l N itr og en S oi l M oi st ur e D ec ay M et ha ne S oi l T em pe ra tu re P re ci pi ta tio n S oi l M oi st ur e S oi l C ar bo n V eg et at io n C ar bo n S oi l N itr og en D ec ay M et ha ne C O 2 D if fu si on D is so lv ed C ar bo n L os s S oi l T em pe ra tu re S oi l M oi st ur e S ur fa ce In ci de nt S ho rt w av e R ad ia tio n S ur fa ce In ci de nt L on gw av e R ad ia tio n S oi l C ar bo n V eg et at io n C ar bo n S oi l N itr og en L ea f N itr og en S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n S oi l N itr og en S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n D is so lv ed C ar bo n L os s V eg et at io n C ar bo n S oi l N itr og en A ut ot ro ph ic R es pi ra tio n (A R ) A ir T em pe ra tu re S oi l T em pe ra tu re P re ci pi ta tio n S oi l M oi st ur e S ur fa ce In ci de nt S ho rt w av e R ad ia tio n S ur fa ce In ci de nt L on gw av e R ad ia tio n V eg et at io n C ar bo n A ir T em pe ra tu re G P P A ir T em pe ra tu re V eg et at io n C ar bo n L ea f N itr og en A ir T em pe ra tu re S oi l T em pe ra tu re P re ci pi ta tio n S oi l M oi st ur e S ur fa ce In ci de nt S ho rt w av e R ad ia tio n S ur fa ce In ci de nt L on gw av e R ad ia tio n V eg et at io n C ar bo n F ra ct io n of In st an ta ne ou s G P P A ir T em pe ra tu re V eg et at io n C ar bo n L ea f N itr og en G P P S oi l T em pe ra tu re A ir T em pe ra tu re S oi l T em pe ra tu re V eg et at io n C ar bo n L ea f N itr og en A ir T em pe ra tu re S oi l T em pe ra tu re V eg et at io n C ar bo n L ea f N itr og en G P P P ro po rt io na l to G ro w th E co sy st em R es pi ra tio n (R ) A R + H R A R + H R A ir T em pe ra tu re S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n V eg et at io n C ar bo n L A I A R + H R A R + H R A R + H R A R + H R A R + H R A R + H R A R + H R N et P ri m ar y P ro du ct io n (N P P ) G P P ‐ A R G P P ‐ A R S ur fa ce In ci de nt S ho rt w av e R ad ia tio n V ap or P re ss ur e D ef ic it C O 2 V eg et at io n C ar bo n L ea f N itr og en L A I G P P ‐ A R F ra ct io n of In st an ta ne ou s G P P G P P ‐ A R A ir T em pe ra tu re P re ci pi ta tio n S oi l M oi st ur e P ot en tia l E va po ra tio n V eg et at io n C ar bo n S oi l N itr og en L ea f N itr og en fP A R G P P ‐ A R G P P ‐ A R A ir T em pe ra tu re P re ci pi ta tio n S oi l C ar bo n S oi l N itr og en S oi l M oi st ur e V eg et at io n C ar bo n L ea f N itr og en L A I N et E co sy st em E xc ha ng e (N E E ) N P P ‐ H R N P P ‐ H R S oi l T em pe ra tu re S oi l M oi st ur e S ur fa ce In ci de nt S ho rt w av e R ad ia tio n V ap or P re ss ur e D ef ic it N P P ‐ H R G P P ‐ R N P P ‐ H R N P P ‐ H R G P P ‐ R N P P ‐ H R N P P ‐ H R SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 3 of 22 T ab le 1. (c on tin ue d) M od el A ttr ib ut e M od el A gr oI B IS B E P S B io m e‐ B G C C an ‐I B IS C N ‐C L A S S D L E M D N D C E co sy s E D 2 E D C M B io m es S im ul at ed C ro pl an ds 6 8 10 9 10 C ro pl an ds 10 6 6 S ite s S im ul at ed 5 10 36 27 31 33 5 39 25 10 M on th s S im ul at ed 19 2 94 5 20 01 19 78 20 82 22 46 19 2 24 50 16 84 65 8 S ou rc e K uc ha ri k an d T w in e [2 00 7] L iu et al . [1 99 9] T ho rn to n et al . [2 00 5] W ill ia m so n et al . [2 00 8] A ra in et al . [2 00 6] T ia n et al . [2 01 0] L i et al . [2 01 0] G ra nt et al . [2 00 5] M ed vi gy et al . [2 00 9] L iu et al . [2 00 3] M od el A ttr ib ut e M od el E P IC IS O L S M L oT E C L P J O R C H ID E E S iB 3 S iB C A S A S iB cr op S S iB 2 T E C O T ri pl ex ‐F L U X T em po ra l R es ol ut io n D ai ly H al f‐ ho ur ly H al f‐ ho ur ly D ai ly H al f‐ ho ur ly H al f‐ ho ur ly 10 m in H al f‐ ho ur ly H al f‐ ho ur ly H ou rl y H al f‐ ho ur ly V eg et at io n P oo ls 3 0 4 3 8 0 8 4 0 3 0 S oi l P oo ls 0 1 5 2 8 0 5 1 0 5 0 S oi l L ay er s 15 0 14 2 0 10 15 10 3 10 0 C an op y P he no lo gy P ro gn os tic P re sc ri be d P ro gn os tic P ro gn os tic P ro gn os tic P re sc ri be d P re sc ri be d P ro gn os tic P re sc ri be d P ro gn os tic P re sc ri be d N itr og en C yc le Y es N o N o N o N o Y es N o Y es N o N o N o G ro ss P ri m ar y P ro du ct iv ity (G P P ) N il S to m at al C on du ct an ce M od el E nz ym e K in et ic M od el S to m at al C on du ct an ce M od el E nz ym e K in et ic M od el E nz ym e K in et ic M od el E nz ym e K in et ic M od el E nz ym e K in et ic M od el S to m at al C on du ct an ce M od el S to m at al C on du ct an ce M od el S to m at al C on du ct an ce M od el H et er ot ro ph ic R es pi ra tio n (H R ) C O 2 D if fu si on D is so lv ed C ar bo n L os s A ir T em pe ra tu re S oi l T em pe ra tu re P re ci pi ta tio n S oi l M oi st ur e F ir st or G re at er O rd er M od el S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n Z er o‐ or de r M od el S oi l T em pe ra tu re S oi l M oi st ur e S oi l C ar bo n S oi l T em pe ra tu re S oi l C ar bo n Z er o‐ or de r M od el F ir st or G re at er O rd er M od el F ir st or G re at er O rd er M od el A ut ot ro ph ic R es pi ra tio n (A R ) N il F ra ct io n of In st an ta ne ou s G P P A ir T em pe ra tu re S oi l T em pe ra tu re S oi l M oi st ur e V eg et at io n C ar bo n G P P A ir T em pe ra tu re S oi l M oi st ur e V eg et at io n C ar bo n A ir T em pe ra tu re V eg et at io n C ar bo n F ra ct io n of In st an ta ne ou s G P P A ir T em pe ra tu re S oi l M oi st ur e V eg et at io n C ar bo n A ir T em pe ra tu re V eg et at io n C ar bo n G P P A ir T em pe ra tu re S oi l M oi st ur e S ur fa ce In ci de nt S ho rt w av e R ad ia tio n R el at iv e H um id ity L A I fP A R C O 2 A ir T em pe ra tu re V eg et at io n C ar bo n F ra ct io n of A nn ua l G P P E co sy st em R es pi ra tio n (R ) A R + H R A R + H R A R + H R A R + H R A R + H R F or ce d A nn ua l B al an ce A R + H R F or ce d A nn ua l B al an ce F or ce d A nn ua l B al an ce A R + H R A R + H R N et P ri m ar y P ro du ct io n (N P P ) L ig ht U se E ff ic ie nc y M od el N il G P P ‐ A R G P P ‐ A R G P P ‐ A R G P P ‐ A R A ir T em pe ra tu re S oi l M oi st ur e C O 2 R el at iv e H um id ity G P P ‐ A R G P P ‐ A R G P P ‐ A R F ra ct io n of In st an ta ne ou s G P P N et E co sy st em E xc ha ng e (N E E ) N P P ‐ H R G P P ‐ R N P P ‐ H R N P P ‐ H R G P P ‐ R G P P ‐ R G P P ‐ R G P P ‐ R G P P ‐ R G P P ‐ R G P P ‐ R B io m es S im ul at ed C ro pl an ds 5 6 9 10 10 10 C ro pl an ds 10 10 3 S ite s S im ul at ed U .S .‐ N e3 9 10 29 35 31 35 5 44 35 7 M on th s S im ul at ed 48 90 9 82 5 21 26 23 32 22 58 24 02 19 2 28 00 24 14 29 1 S ou rc e C au sa ra no et al . [2 00 7] R ile y et al . [2 00 2] H an so n et al . [2 00 4] Si tc h et al . [2 00 3] K ri nn er et al . [2 00 5] B ak er et al . [2 00 8] Sc ha ef er et al . [2 00 9] L ok up iti ya et al . [2 00 9] Z ha n et al . [2 00 3] W en g an d L uo [2 00 8] Z ho u et al . [2 00 8] SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 4 of 22 T ab le 2. S um m ar y of S ite C ha ra ct er is tic sa S ite ID N am e P ri or ity C ou nt ry L at itu de L on gi tu de E le va tio n (m a. s. l.) IG B P C la ss K öp pe n‐ G ei ge r C lim at e C la ss if ic at io n C A ‐C a1 B ri tis h C ol um bi a ‐ C am pb el l R iv er ‐ M at ur e F or es t S ite 1 C an ad a 49 .8 7 −1 25 .3 3 30 0 E N F M ar iti m e te m pe ra te C A ‐C a2 B ri tis h C ol um bi a ‐ C am pb el l R iv er ‐ C le ar cu t S ite 2 C an ad a 49 .8 7 −1 25 .2 9 18 0 E N F M ar iti m e te m pe ra te C A ‐C a3 B ri tis h C ol um bi a ‐ C am pb el l R iv er ‐ Y ou ng P la nt at io n S ite 2 C an ad a 49 .5 3 −1 24 .9 0 16 5 E N F M ar iti m e te m pe ra te C A ‐G ro O nt ar io ‐ G ro un dh og R iv er ‐ M at ur e B or ea l M ix ed W oo d 1 C an ad a 48 .2 2 −8 2. 16 30 0 M F W ar m su m m er co nt in en ta l C A ‐L et L et hb ri dg e 1 C an ad a 49 .7 1 −1 12 .9 4 96 0 G R A W ar m su m m er co nt in en ta l C A ‐M er E as te rn P ea tla nd ‐ M er B le ue 1 C an ad a 45 .4 1 −7 5. 52 70 W E T W ar m su m m er co nt in en ta l C A ‐O as S as k. ‐ S S A O ld A sp en 1 C an ad a 53 .6 3 −1 06 .2 0 53 0 D B F C on tin en ta l su ba rc tic C A ‐O bs S as k. ‐ S S A O ld B la ck S pr uc e 1 C an ad a 53 .9 9 −1 05 .1 2 62 9 E N F C on tin en ta l su ba rc tic C A ‐O jp S as k. ‐ S S A O ld Ja ck P in e 1 C an ad a 53 .9 2 −1 04 .6 9 57 9 E N F C on tin en ta l su ba rc tic C A ‐Q fo Q ue be c M at ur e B or ea l F or es t S ite 1 C an ad a 49 .6 9 −7 4. 34 38 2 E N F C on tin en ta l su ba rc tic C A ‐S J1 S as k. ‐ 19 94 H ar ve st ed Ja ck P in e 2 C an ad a 53 .9 1 −1 04 .6 6 58 0 E N F C on tin en ta l su ba rc tic C A ‐S J2 S as k. ‐ 20 02 H ar ve st ed Ja ck P in e 2 C an ad a 53 .9 4 −1 04 .6 5 51 8 E N F C on tin en ta l su ba rc tic C A ‐S J3 S as k. ‐ S S A 19 75 H ar ve st ed Y ou ng Ja ck P in e 2 C an ad a 53 .8 8 −1 04 .6 4 51 1 E N F C on tin en ta l su ba rc tic C A ‐T P 3 O nt ar io ‐ T ur ke y P oi nt M id dl e‐ ag ed W hi te P in e 2 C an ad a 42 .7 1 −8 0. 35 21 9 E N F W ar m su m m er co nt in en ta l C A ‐T P 4 O nt ar io ‐ T ur ke y P oi nt M at ur e W hi te P in e 1 C an ad a 42 .7 1 −8 0. 36 21 9 E N F W ar m su m m er co nt in en ta l C A ‐W P 1 W es te rn P ea tla nd ‐ L aB ic he ‐B la ck S pr uc e/ L ar ch F en 1 C an ad a 54 .9 5 −1 12 .4 7 54 0 M F C on tin en ta l su ba rc tic U .S .‐ A R M O K ‐ A R M S ou th er n G re at P la in s S ite ‐ L am on t 1 U S A 36 .6 1 −9 7. 49 31 0 C R O H um id su bt ro pi ca l U .S .‐ A tq A K ‐ A tq as uk 1 U S A 70 .4 7 −1 57 .4 1 16 W E T T un dr a U .S .‐ B rw A K ‐ B ar ro w 1 U S A 71 .3 2 −1 56 .6 3 1 W E T T un dr a U .S .‐ D k2 N C ‐ D uk e F or es t ‐ H ar dw oo ds 1 U S A 35 .9 7 −7 9. 10 16 0 D B F H um id su bt ro pi ca l U .S .‐ D k3 N C ‐ D uk e F or es t ‐ L ob lo lly P in e 1 U S A 35 .9 8 −7 9. 09 16 3 E N F H um id su bt ro pi ca l U .S .‐ H a1 M A ‐ H ar va rd F or es t E M S T ow er (H F R 1) 1 U S A 42 .5 4 −7 2. 17 30 3 D B F W ar m su m m er co nt in en ta l U .S .‐ H o1 M E ‐ H ow la nd F or es t (M ai n T ow er ) 1 U S A 45 .2 0 −6 8. 74 60 E N F W ar m su m m er co nt in en ta l U .S .‐ IB 1 IL ‐ F er m i N at io na l A cc el er at or L ab or at or y ‐ B at av ia (A gr ic ul tu ra l S ite ) 1 U S A 41 .8 6 −8 8. 22 22 7 C R O H ot su m m er co nt in en ta l U .S .‐ IB 2 IL ‐ F er m i N at io na l A cc el er at or L ab or at or y ‐ B at av ia (P ra ir ie S ite ) 1 U S A 41 .8 4 −8 8. 24 22 7 G R A H ot su m m er co nt in en ta l U .S .‐ L os W I ‐ L os t C re ek 1 U S A 46 .0 8 −8 9. 98 48 0 C S H W ar m su m m er co nt in en ta l U .S .‐ M M S IN ‐ M or ga n M on ro e S ta te F or es t 1 U S A 39 .3 2 −8 6. 41 27 5 D B F H um id su bt ro pi ca l U .S .‐ M O z M O ‐ M is so ur i O za rk S ite 1 U S A 38 .7 4 −9 2. 20 21 9 D B F H um id su bt ro pi ca l U .S .‐ M e2 O R ‐ M et ol iu s ‐ In te rm ed ia te A ge d P on de ro sa P in e 1 U S A 44 .4 5 −1 21 .5 6 12 53 E N F D ry ‐s um m er su bt ro pi ca l U .S .‐ M e3 2 U S A 44 .3 2 −1 21 .6 1 10 05 E N F SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 5 of 22 T ab le 2. (c on tin ue d) S ite ID N am e P ri or ity C ou nt ry L at itu de L on gi tu de E le va tio n (m a. s. l.) IG B P C la ss K öp pe n‐ G ei ge r C lim at e C la ss if ic at io n O R ‐ M et ol iu s ‐ S ec on d Y ou ng A ge d P in e D ry ‐s um m er su bt ro pi ca l U .S .‐ M e4 O R ‐ M et ol iu s ‐ O ld A ge d P on de ro sa P in e 2 U S A 44 .5 0 −1 21 .6 2 91 5 E N F D ry ‐s um m er su bt ro pi ca l U .S .‐ M e5 O R ‐ M et ol iu s ‐ F ir st Y ou ng A ge d P in e 2 U S A 44 .4 4 −1 21 .5 7 11 83 E N F D ry ‐s um m er su bt ro pi ca l U .S .‐ N R 1 C O ‐ N iw ot R id ge F or es t (L T E R N W T 1) 1 U S A 40 .0 3 −1 05 .5 5 30 50 E N F C on tin en ta l su ba rc tic U .S .‐ N e1 N E ‐ M ea d ‐ Ir ri ga te d C on tin uo us M ai ze S ite 1 U S A 41 .1 7 −9 6. 48 36 1 C R O H ot su m m er co nt in en ta l U .S .‐ N e2 N E ‐ M ea d ‐ Ir ri ga te d M ai ze ‐ S oy be an R ot at io n S ite 1 U S A 41 .1 6 −9 6. 47 36 1 C R O H ot su m m er co nt in en ta l U .S .‐ N e3 N E ‐ M ea d ‐ R ai nf ed M ai ze ‐ S oy be an R ot at io n S ite 1 U S A 41 .1 8 −9 6. 44 36 1 C R O H ot su m m er co nt in en ta l U .S .‐ P F a W I ‐ P ar k F al ls /W L E F 1 U S A 45 .9 5 −9 0. 27 48 5 M F W ar m su m m er co nt in en ta l U .S .‐ S O 2 C A ‐ S ky O ak s ‐ O ld S ta nd 1 U S A 33 .3 7 −1 16 .6 2 13 92 C S H D ry ‐s um m er su bt ro pi ca l U .S .‐ S hd O K ‐ S hi dl er ‐ O kl ah om a 1 U S A 36 .9 3 −9 6. 68 35 0 G R A H um id su bt ro pi ca l U .S .‐ S yv M I ‐ S yl va ni a W ild er ne ss A re a 1 U S A 46 .2 4 −8 9. 35 54 0 M F W ar m su m m er co nt in en ta l U .S .‐ T on C A ‐ T on zi R an ch 1 U S A 38 .4 3 −1 20 .9 7 17 7 W S A D ry ‐s um m er su bt ro pi ca l U .S .‐ U M B M I ‐ U ni ve rs ity of M ic hi ga n B io lo gi ca l S ta tio n 1 U S A 45 .5 6 −8 4. 71 23 4 D B F W ar m su m m er co nt in en ta l U .S .‐ V ar C A ‐ V ai ra R an ch ‐ Io ne 1 U S A 38 .4 1 −1 20 .9 5 12 9 G R A D ry ‐s um m er su bt ro pi ca l U .S .‐ W C r W I ‐ W ill ow C re ek 1 U S A 45 .8 1 −9 0. 08 52 0 D B F W ar m su m m er co nt in en ta l S ite ID A nn ua l N E E (g C m 2 −) A nn ua l N E E E rr or (g C m 2 −) D ay tim e D at a C ov er ag e (% ) N ig ht tim e D at a C ov er ag e (% ) L A I A nn ua l P re ci pi ta tio n (m m ) M ea n A nn ua l A ir T em pe ra tu re (° C ) M ea su re m en t P er io d B io m e S ou rc e C A ‐C a1 −2 44 .3 61 .1 99 26 6. 1 12 56 8. 7 19 98 –2 00 6 E N F T Sc hw al m et al . [2 00 7] C A ‐C a2 57 1. 7 31 .5 96 23 4. 4 12 22 8. 8 20 01 –2 00 6 E N F T Sc hw al m et al . [2 00 7] C A ‐C a3 91 .2 37 .9 91 27 2. 2 15 54 9. 5 20 02 –2 00 6 E N F T Sc hw al m et al . [2 00 7] C A ‐G ro −3 6. 5 33 .5 93 34 4. 1 42 7 3. 3 20 04 –2 00 6 M F M cC au gh ey et al . [2 00 6] C A ‐L et −1 32 .9 14 .3 96 46 0. 7 33 5 6. 5 19 97 –2 00 6 G R A F la na ga n et al . [2 00 2] C A ‐M er −6 8. 5 21 .6 79 56 1. 3 93 5 6. 2 19 99 –2 00 6 W E T L af le ur et al . [2 00 3] C A ‐O as −1 58 .0 28 .5 94 56 3. 8 46 0 2. 3 19 97 –2 00 6 D B F B ar r et al . [2 00 4] C A ‐O bs −5 6. 3 16 .1 89 45 5. 6 47 0 1. 6 20 00 –2 00 6 E N F B G ri ffi s et al . [2 00 3] C A ‐O jp −2 9. 9 16 .6 91 50 3. 4 46 1 1. 5 20 00 –2 00 6 E N F B G ri ffi s et al . [2 00 3] C A ‐Q fo −1 3. 7 21 .0 93 40 4 81 9 2. 7 20 04 –2 00 6 E N F B B er ge ro n et al . [2 00 7] C A ‐S J1 28 .0 15 .3 87 31 0. 8 34 4 0. 6 20 02 –2 00 5 E N F B Z ha et al . [2 00 9] C A ‐S J2 11 7. 0 6. 1 89 47 1. 3 53 7 0. 1 20 03 –2 00 6 E N F B Z ha et al . [2 00 9] C A ‐S J3 −8 2. 0 17 .7 92 34 4. 3 69 4 0. 8 20 04 –2 00 5 E N F B Z ha et al . [2 00 9] C A ‐T P 4 −1 33 .2 29 .5 95 43 3. 5 95 9 8. 6 20 02 –2 00 7 E N F T P ei ch l an d A ra in [2 00 7] C A ‐W P 1 −1 95 .8 16 .4 96 50 2. 7 48 1 1. 7 20 03 –2 00 7 W E T Sy ed et al . [2 00 6] SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 6 of 22 T ab le 2. (c on tin ue d) S ite ID A nn ua l N E E (g C m 2 −) A nn ua l N E E E rr or (g C m 2 −) D ay tim e D at a C ov er ag e (% ) N ig ht tim e D at a C ov er ag e (% ) L A I A nn ua l P re ci pi ta tio n (m m ) M ea n A nn ua l A ir T em pe ra tu re (° C ) M ea su re m en t P er io d B io m e S ou rc e U .S .‐ A R M −1 28 .4 74 .4 89 36 3. 1 62 9 15 .6 20 00 –2 00 6 C R O F is ch er et al . [2 00 7] C A ‐T P 4 −1 33 .2 29 .5 95 43 3. 5 95 9 8. 6 20 02 –2 00 7 E N F T P ei ch l an d A ra in [2 00 7] U .S .‐ A tq −1 2. 8 ‐ 50 22 1. 1 11 8 −1 0. 6 19 99 –2 00 6 T U N O be rb au er et al . [2 00 7] U .S .‐ B rw −7 2. 0 ‐ 49 29 1. 5 10 8 −1 0. 9 19 99 –2 00 2 T U N H ar az on o et al . [2 00 3] U .S .‐ D k2 −7 18 .1 ‐ 48 1 7 10 91 15 .1 20 03 –2 00 5 D B F Si qu ei ra et al . [2 00 6] U .S .‐ D k3 −3 50 .0 13 9. 0 75 37 5. 6 11 26 14 .7 19 98 –2 00 5 E N F T Si qu ei ra et al . [2 00 6] U .S .‐ H a1 −2 17 .4 65 .9 78 34 3. 38 11 22 7. 9 19 91 –2 00 6 D B F U rb an sk i et al . [2 00 7] U .S .‐ H o1 −2 23 .0 33 .4 70 47 5. 2 81 8 6. 6 19 96 –2 00 4 E N F T R ic ha rd so n et al . [2 00 9] U .S .‐ IB 1 −2 69 .0 31 .3 92 46 1. 29 71 8 10 .1 20 05 –2 00 7 C R O P os t et al . [2 00 4] U .S .‐ IB 2 −8 6. 0 42 .0 80 49 5. 38 81 8 10 .4 20 04 –2 00 7 G R A P os t et al . [2 00 4] U .S .‐ L os −7 8. 0 19 .2 82 54 4. 24 66 6 3. 8 20 00 –2 00 6 W E T Su lm an et al . [2 00 9] U .S .‐ M M S −3 46 .1 66 .3 97 46 4. 9 11 09 12 .4 19 99 –2 00 6 D B F Sc hm id et al . [2 00 0] U .S .‐ M O z −3 05 .7 48 .9 94 33 3. 91 73 0 13 .3 20 04 –2 00 7 D B F G u et al . [2 00 6] U .S .‐ M e2 −5 36 .0 65 .8 63 46 3. 62 43 4 7. 6 20 02 –2 00 7 E N F T T ho m as et al . [2 00 9] U .S .‐ M e3 −1 98 .0 32 .7 83 28 0. 52 42 3 8. 5 20 04 –2 00 5 E N F T V ic ke rs et al . [2 00 9] U .S .‐ M e4 −6 12 .3 ‐ 55 41 2. 1 64 1 8. 3 19 96 –2 00 0 E N F T Ir vi ne et al . [2 00 4] U .S .‐ M e5 −2 06 .0 10 .6 97 48 1. 1 35 0 7. 6 19 99 –2 00 2 E N F T Ir vi ne et al . [2 00 4] U .S .‐ N R 1 −3 7. 2 27 .0 89 44 4. 2 66 3 2. 5 19 98 –2 00 7 E N F T B ra df or d et al . [2 00 8] U .S .‐ N e1 −4 24 .0 41 .8 93 42 6. 5 83 2 11 .1 20 01 –2 00 6 C R O V er m a et al . [2 00 5] U .S .‐ N e2 −3 82 .0 41 .8 96 51 6. 5 82 3 10 .8 20 01 –2 00 6 C R O V er m a et al . [2 00 5] U .S .‐ N e3 −2 58 .0 43 .3 94 55 6. 2 62 7 10 .9 20 01 –2 00 6 C R O V er m a et al . [2 00 5] U .S .‐ P F a 45 .0 41 .1 85 30 4. 05 73 6 5. 1 19 97 –2 00 5 M F D av is et al . [2 00 3] U .S .‐ S O 2 22 .4 25 .6 87 30 3 69 5 13 .8 19 98 –2 00 6 S H R L uo et al . [2 00 7] U .S .‐ S hd −7 5. 5 22 .0 96 49 5. 9 11 79 14 .8 19 97 –2 00 1 G R A Su yk er et al . [2 00 3] U .S .‐ S yv 48 .5 34 .7 53 51 4. 1 70 0 4. 4 20 01 –2 00 6 M F D es ai et al . [2 00 5] U .S .‐ T on −6 7. 8 52 .0 77 25 0. 6 54 9 16 .4 20 01 –2 00 7 W S A M a et al . [2 00 7] U .S .‐ U M B −1 32 .0 42 .4 86 39 4. 23 62 9 7. 4 19 98 –2 00 6 D B F Sc hm id et al . [2 00 3] U .S .‐ V ar 7. 3 11 0. 6 80 22 2. 4 56 3 15 .9 20 01 –2 00 7 G R A M a et al . [2 00 7] U .S .‐ W C r −2 22 .6 54 .1 48 55 5. 36 71 2 5. 3 19 98 –2 00 6 D B F C oo k et al . [2 00 4a ] a S ou rc es :I G B P cl as si fi ca tio n, L ov el an d et al .[ 20 01 ]; K öp pe n‐ G ei ge r, P ee le ta l. [2 00 7] ;L A I fo r U S A si te s, ht tp :// pu bl ic .o rn l.g ov /a m er if lu x/ ;L A I fo r C an ad ia n si te s, C he n et al .[ 20 06 ] an d Sc hw al m et al .[ 20 06 ]. A nn ua lp re ci pi ta tio n an d m ea n an nu al ai r te m pe ra tu re ar e m ea su re m en tp er io d av er ag es of m et eo ro lo gi ca li np ut s us ed to dr iv e m od el si m ul at io ns .N E E va lu es sh ow ye ar ly in te gr al s an d as so ci at ed er ro r: on e st an da rd de vi at io n ba se d on un ce rt ai nt y du e to ra nd om no is e an d th e fr ic tio n ve lo ci ty th re sh ol d ag gr eg at ed to ye ar ly va lu es an d su m m ed in qu ad ra tu re [B ar r et al ., 20 09 ]. D at a co ve ra ge s ar e pe rc en ta ge s of ha lf ‐h ou rl y N E E m ea su re m en ts th at sa tis fi ed qu al ity co nt ro ls ta nd ar ds (f ri ct io n ve lo ci ty th re sh ol d) fo r da y‐ an d ni gh tti m e se pa ra te ly .P ri or ity :( 1) P ri m ar y si te s w ith co m pl et e (i nc lu de s an ci lla ry an d bi ol og ic al da ta te m pl at es ) re co rd s; (2 ) S ec on da ry ch ro no se qu en ce si te s. S ta nd ar di ze d P re ci pi ta tio n In de x av ai la bl e on ly fo r P ri or ity 1 si te s ex cl ud in g U S ‐A tq , U S ‐B rw , U S ‐D k2 , U S ‐I B 1, an d U S ‐S hd . C A ‐T P 3, U S ‐A tq , U S ‐B rw , U S ‐D k2 , an d U S ‐M e4 si te s us ed po st pr oc es si ng pr ot oc ol fr om th e L a T hu ile an d A si lo m ar F L U X N E T S yn th es is da ta se t( ht tp :// w w w .f lu xd at a. or g/ ) [M of fa te ta l., 20 07 ;P ap al e et al ., 20 06 ] an d la ck N E E un ce rt ai nt ie s. B io m e is co m bi na tio n of IG B P cl as s an d K öp pe n‐ G ei ge r cl im at e. U S ‐A tq an d U S ‐B rw ‐A rc tic w et la nd s cl as si fi ed as tu nd ra bi om e C A ‐W P 1, tr ee d fe n (I G B P m ix ed fo re st ) gr ou pe d w ith w et la nd s bi om e U S ‐L os , sh ru b w et la nd si te (I G B P cl os ed sh ru bl an ds ) gr ou pe d w ith w et la nd s bi om e U S ‐S O 2, cl os ed sh ru bl an ds gr ou pe d w ith sh ru bl an ds (o pe n or cl os ed ) bi om e. IG B P cl as s an d bi om e co de s: C R O , cr op la nd s; C S H , cl os ed sh ru bl an ds ; G R A , gr as sl an ds , E N F , ev er gr ee n ne ed le le af fo re st ; E N F B , ev er gr ee n ne ed le le af fo re st ‐b or ea l zo ne ; E N F T , ev er gr ee n ne ed le le af fo re st ‐t em pe ra te zo ne ; D B F , de ci du ou s br oa dl ea f fo re st ; M F , m ix ed (d ec id uo us /e ve rg re en ) fo re st ; W S A , w oo dy sa va nn a; S H R , sh ru bl an ds ; T U N , tu nd ra ; W E T , w et la nd s. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 7 of 22 runs using gap‐filled observed weather at each site and locally observed values of soil texture according to a stan- dard protocol [Ricciuto et al., 2009] (http://www.nacarbon. org/nacp/), including a target NEE of zero integrated over the last 5 years of the simulation period. In addition, a mean model ensemble (hereafter: MEAN) was also analyzed. MEAN was calculated as the mean monthly value across all simulations. Furthermore, in contrast to other models, the parameter values used in the model LoTEC were optimized using data assimilation [Ricciuto et al., 2008]. LoTEC simulations were however retained when calculating MEAN as their effect on model skill was negligible due to the rel- atively small number of site‐months simulated. [6] Gaps in the meteorological data record occurred at EC sites due to data quality control or instrument failure. Missing values of air temperature, humidity, shortwave radiation, and precipitation data, i.e., key model inputs, were filled using DAYMET [Thornton et al., 1997] before 2003 or the nearest available climate station in the National Climatic Data Center’s Global Surface Summary of the Day (GSOD) database. Daily GSOD and DAYMET data were temporally downscaled to hourly or half‐hourly values using the phasing from observed mean diurnal cycles calculated from a 15 day moving window. The phasing used a sine wave assuming peak values at 1500 local standard time (LST) and lowest values at 0300 LST. In the absence of station data a 10 day running mean diurnal cycle was used [Ricciuto et al., 2009] (http://nacp.ornl.gov/docs/Site_Synthesis_Protocol_v7. pdf). [7] EC data were produced by AmeriFlux and Fluxnet‐ Canada investigators and processed as a synthesis product of the North American Carbon Program (NACP) Site Level Interim Synthesis (http://www.nacarbon.org/nacp/). The observed NEE were corrected for storage, despiked (i.e., outlying values removed), filtered to remove conditions of low turbulence (friction velocity filtered), and gap‐filled to create a continuous time series [Barr et al., 2004]. The time series included estimates of random uncertainty and uncer- tainty due to friction velocity filtering [Barr et al., 2004, 2009]. In this analysis, NEE was aggregated to monthly values using only non‐gap‐filled data, i.e., observed values deemed spurious and subsequently infilled were not con- sidered. Coincident modeled NEE values were similarly excluded. This removed the influence of gap‐filling algo- rithms in the comparison of observed and modeled NEE. [8] Drought level was quantified using the 3 month Standard Precipitation Index (SPI) [McKee et al., 1993]. Monthly SPI values were taken from the U.S. Drought Monitor (http://drought.unl.edu/DM/) whereby each tower was matched to nearby meteorological station(s) indicative of local drought conditions given proximity, topography, and human impact. This study used three drought levels: dry required SPI < −0.8, wet corresponded to SPI > +0.8, other- wise normal conditions existed. Climatic season was defined by four seasons of 3 months each with winter given by December, January, and February. 2.2. Model Skill [9] Model‐data mismatch was evaluated using normalized mean absolute error (NMAE) [Medlyn et al., 2005], the reduced c2 statistic (c2) [Taylor, 1996] as well as Taylor diagrams and skill (S) [Taylor, 2001]. The first metric quantifies bias, the “average distance” between observations and simulations in units of observed mean NEE: NMAE ¼ X ijkl NEEobs  NEEsim nNEEobs ; ð1Þ where the overbar indicates averaging across all values, n is sample size, the subscript obs is for observations and sim is for modeled estimates. The summation is for any arbitrary data group (denoted by subscripts on the summation oper- ator only) where subscript i is for site, j is for model, k is for climatic season, l is for drought level. [10] The second metric used to evaluate model perfor- mance was the reduced c2 statistic. This is the squared difference between paired model and data points over observational error normalized by degrees of freedom: 2 ¼ 1 n X ijkl NEEobs  NEEsim 2NEE  2 ; ð2Þ where dNEE is uncertainty of monthly NEE (see section 2.3), 2 normalizes the uncertainty in observed NEE to correspond to a 95% confidence interval, the summation is across any arbitrary data group (denoted by subscripts on the summa- tion operator). c2 values are linked to model‐data mismatch where a value of unity indicates that model and data are in agreement relative to data uncertainty. [11] A final characterization of model performance used Taylor diagrams [Taylor, 2001]; visual displays based on pattern matching, i.e., the degree to which simulations matched the temporal evolution of monthly NEE. Taylor plots are polar coordinate displays of the linear correlation coefficient (r), centered root mean squared error (RMSE; pattern error without considering bias), and the standard deviation of NEE (s). Taylor diagrams were constructed for the mean model ensemble (MEAN) and across‐site mean model performance using the full data record for each com- bination of site and model (ranging from 7 to 178 months). More generally, each polar coordinate point for any arbitrary data group can be scored: S ¼ 2 1þ ð Þ norm þ 1=normð Þ2 ; ð3Þ where S is the model skill metric bound by zero and unity where unity indicates perfect agreement, and snorm is the ratio of simulated to observed standard deviation [Taylor, 2001]. [12] To scale model skill metrics across gradients of site, biome, model, seasonality, and dryness level we aggregated across data groups weighting each by sample size. For example, c2 for model I, denoted by subscript j = I, is given by 2j¼I ¼ X ikl nikl2ikl nj¼I ð4Þ where the summation is over all sites, seasons, and levels of dryness where model I was used as denoted by subscripts i, k, and l, respectively; nj=I is the total site‐months simulated SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 8 of 22 with model I; and c2j=I is aggregated c 2 for model I. We did not evaluate model performance for any data group with n < 3. In sum, Taylor displays and skill examined models’ ability to mimic the monthly trajectory of observed NEE, the calculation of NMAE quantified bias in units of mean observed NEE, and c2 values quantified how well model‐ data mismatch scales with flux uncertainty. 2.3. Observational Flux Uncertainty [13] We calculated the standard error of monthly NEE (dNEE) [Barr et al., 2009] by combining random uncertainty and uncertainty associated with the friction velocity threshold (u* Th), a value used to identify and reject spurious nighttime NEE measurements. Random uncertainty was estimated following Richardson and Hollinger [2007]: (1) generate synthetic NEE data using the gap‐filling model [Barr et al., 2004, 2009] for a given site‐year, (2) introduce gaps as in the observed data with u* Th filtering, (3) add noise, (4) infill gaps using the gap‐filling model, and (5) repeat the process 1000 times for each site. The random uncertainty component of dNEE was then the standard deviation across all 1000 realizations aggregated to months. [14] The u* Th uncertainty component of dNEE was also estimated using Monte Carlo methods. Here 1000 realiza- tions of NEE were generated using 1000 draws from a distribution of u* Th. This distribution was based on binning the raw flux data with respect to climatic season, temperature, and site‐year and estimating u* Th in each bin [Papale et al., 2006]. The standard deviation across all realizations gave the u* Th uncertainty component of dNEE. Both components were combined in quadrature to one standard error of monthly NEE (= dNEE) [Barr et al., 2009]. 2.4. Relating Model Skill to Model Structure and Site History [15] The models evaluated here range widely in their emphasis and structure (Table 1). Some focus on biophy- sical calculations (SiB3, BEPS), where others emphasize biogeochemistry (DLEM), or ecosystem dynamics (ED2). However, as terrestrial biosphere models simulate carbon cycling with hydrological variables, most models contain both biophysics and biogeochemistry. This motivated characterizing model structure with definite attributes, e.g., prognostic versus prescribed canopy phenology, number of soil pools, and type of NEE algorithm (Table 3). To resolve how such characteristics and site history impacted model skill we calculated S for all observed combinations of site, model, seasonality, and drought level and cross‐referenced Table 3. Model Structural and Site History Predictors Used to Classify Taylor Skill With Regression Tree Analysisa Predictor Value Model temporal resolution Daily, half‐hourly or less, hourly, monthly Canopy Prognostic, semiprognostic, prescribed. Prescribed canopy from remote sensing, semiprognostic has some prescribed input into canopy leaf biomass but calculates phenology with other prognostic variables. Number of vegetation pools Number of pools, both dynamic and static Number of soil pools Number of pools, both dynamic and static Number of soil layers Number of layers Nitrogen True if the model has a nitrogen cycle; otherwise false. Steady state True if the simulated long‐term NEE integral approaches zero; otherwise false. Autotrophic respiration (AR) Fraction of annual GPP, fraction of instantaneous GPP, explicitly calculated, nil, proportional to growth Ecosystem respiration (R) AR + HR, explicitly calculated, forced annual balance Gross primary productivity (GPP) Enzyme kinetic model, light use efficiency model, nil, stomatal conductance model Heterotrophic respiration (HR) Explicitly calculated, first or greater order model, zero‐order model Net ecosystem exchange (NEE) Explicitly calculated, GPP ‐ R, NPP ‐ HR Net primary productivity (NPP) Explicitly calculated, fraction of instantaneous GPP, GPP ‐ AR, light use efficiency model Overall model complexity Low, average, high Values correspond to terciles of the total amount of first‐order functional arguments for the following model‐generated variables/outputs: AR, canopy leaf biomass, R, evapotranspiration, GPP, HR, NEE, NPP, soil moisture. Site history True if the below listed management activity or disturbance or event occurred on site; otherwise false. Grazed, fertilized, fire, harvest, herbicide, insects and pathogens, irrigation, natural regeneration, pesticide, planted, residue management, thinning Stand age class Young, intermediate, nil, mature, multicohort. Values based on stand age in forested sites; stands without a clear dominant stratum are treated as multicohort; nonforest types have nil. aTaylor skill (S; equation (3)) was divided into three classes using terciles. Model structural predictants are from the Metadata for Forward (Ecosystem) Model Intercomparison survey collated by the NACP Site Synthesis (http://daac.ornl.gov/SURVEY8/ survey_results.shtml). Site history data are from http://public.ornl.gov/ameriflux/, http://www.fluxnet.org, and Schwalm et al. [2006]. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 9 of 22 these with 13 site history variables and 14 model attributes (Table 3). Only 20 models were available for this exercise; MEAN and the optimized LoTEC were excluded. We used S as it is bound by zero (no agreement) and unity (perfect agreement) in contrast to NMAE and c2 which are unbound. The Taylor skill metric (S) was first discretized into three classes based on terciles. These classes, representing three tiers of model‐data agreement, were then related to biome, climatic season, drought level, site history, and model structure using regression tree analysis (RTA) as a super- vised classification algorithm. RTA is a form of binary recursive partitioning [Breiman et al., 1984] that succes- sively splits the data (Taylor skill classes as the response; all other attributes as predictors) into subsets (nodes) by mini- mizing within‐subset variation. The result is a pruned tree‐ like topology whereby predicted values (Taylor skill metric class) are derived by a top‐to‐bottom traversal following the rules (branches) that govern subset membership until a predicted value is reached (terminal node). The splitting rules at each node as well as its position allow for a cal- culation of relative variable importance [Breiman et al., 1984] with the most important variable given a score of 100. Variables of high importance were further analyzed using conditional means, i.e., comparing mean values for each predictor value, with statistical differences determined using Bonferroni corrections for multiple comparisons [Hochberg and Tamhane, 1987]. 3. Results 3.1. Model‐Data Agreement Relative to Climatic Season, Dryness, and Biome [16] Overall agreement across n = 31025 months was better in forested than nonforested biomes; both NMAE (Table 4) and c2 values (Table 5) were closer to zero and unity, respectively. At the biome level, model skill was loosely ranked in five tiers: evergreen needleleaf forests in the temperate zone, mixed forests > deciduous broadleaf forests, evergreen needleleaf forests in the boreal zone > grasslands, woody savannahs > croplands, shrublands, wetlands > tundra. These rankings were robust across models used in the majority of biomes, although some divergence was apparent for croplands and shrublands (Figure 1). Relative to seasonality and drought level models were most consistent with observations during periods of peak biological activity (climatic summer) and under dry conditions (Figure 2). However, across the three levels of dryness, changes in model‐data agreement were negligible Table 4. Normalized Mean Absolute Error by Climatic Season, Drought Level, and Biomea Biomeb Climatic Season Drought Level OverallWinter Spring Summer Fall Dry Normal Wet CRO 1.90 4.64 −0.79 12.73 −1.43 −1.54 −1.59 −1.55 DBF 0.81 93.7 −0.52 −2.14 −1.01 −1.00 −0.95 −1.00 ENFB 1.52 −1.12 −0.69 −1.92 −0.87 −1.15 −3.43 −1.12 ENFT −6.34 −0.66 −0.50 −0.76 −0.63 −0.72 −0.63 −0.68 GRA −25.46 −0.84 −1.11 5.19 −1.52 −1.32 −3.07 −1.51 MF 1.10 −7.48 −0.47 57.70 −1.42 −1.04 −1.15 −1.12 SHR −87.37 −1.37 −3.03 −140.17 −1.82 −2.18 −41.13 −2.88 TUN −1.43 −11.07 −20.63 6.38 19.22 −24.06 −1.81 −20.15 WET 1.80 −5.07 −0.59 −4.72 −1.21 −1.20 −2.38 −1.27 WSA −2.73 −0.75 −1.47 10.56 −1.39 −1.32 −1.51 −1.37 Overall 2.42 −1.35 −0.61 −1.94 −0.97 −1.01 −1.00 −1.00 aDrought level was based on monthly values of 3 month Standard Precipitation Index (SPI): dry value were < −0.8; wet > +0.8. Otherwise normal conditions existed. bBiome codes: CRO, cropland; GRA, grassland; ENFB, evergreen needleleaf forest‐boreal zone; ENFT, evergreen needleleaf forest‐temperate zone; DBF, deciduous broadleaf forest; MF, mixed (deciduous/evergreen) forest; WSA, woody savanna; SHR, shrubland; TUN, tundra; WET, wetland. Table 5. Reduced c2 Statistic by Climatic Season, Drought Level, and Biomea Biomeb Climatic Season Drought Level OverallWinter Spring Summer Fall Dry Normal Wet CRO 3.22 10.66 39.75 49.71 14.43 23.54 32.75 25.8 DBF 5.29 10.74 8.77 4.55 5.58 7.86 8.67 7.34 ENFB 21.25 17.75 4.98 6.61 11.64 12.02 18.51 12.61 ENFT 4.39 7.90 3.27 2.26 4.71 4.29 4.60 4.45 GRA 10.89 11.38 25.01 17.22 13.97 10.99 26.01 16.07 MF 3.74 4.67 2.05 2.02 2.92 3.24 2.98 3.08 SHR 13.34 27.98 12.52 11.2 9.26 21.31 10.31 16.26 WET 23.65 27.27 11.74 7.54 21.51 17.36 12.91 17.47 WSA 0.61 5.81 11.88 3.39 6.73 4.64 6.35 5.37 Overall 8.18 11.95 11.27 9.45 8.10 9.98 12.72 10.26 aDrought level was based on monthly values of 3 month Standard Precipitation Index (SPI): dry value were < −0.8; wet > +0.8. Otherwise normal conditions existed. bBiome codes: CRO, cropland; GRA, grassland; ENFB, evergreen needleleaf forest‐boreal zone; ENFT, evergreen needleleaf forest‐temperate zone; DBF, deciduous broadleaf forest; MF, mixed (deciduous/evergreen) forest; WSA, woody savanna; SHR, shrubland; WET, wetland. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 10 of 22 for NMAE (∼4% change, Table 4) but more pronounced for c2 (from 8.10 to 12.72, Table 5). Averaged over just the warm season (excluding climatic winter) dry conditions were coincident with worse model‐data agreement, e.g., NMAE was −0.99, −0.91, and −0.84 for dry, normal, and wet, respectively. In biomes with a clear seasonal cycle in leaf area index (LAI) a loss of model skill occurred during climatic spring and fall (Tables 4 and 5), especially for NMAE. 3.2. Skill Metrics by Model [17] Regardless of metric, model skill was highly variable. Of the three model skill metrics, NMAE was related to both Taylor skill and c2 (r = −0.65; p < 0.0001). Jointly, high Taylor skill co‐occurred with NMAE and c2 values closer to zero and unity, respectively (Figure 3). Across models NMAE ranged from −0.42 of the overall mean observed flux to −2.18 for LoTEC and DNDC, respectively. Values of c2 varied from 2.17 to 29.87 for LoTEC and CN‐CLASS, respectively. Alternatively, the degree of model‐data mis- match (the distance between observations and simulations) was at least 2.17 times the observational flux uncertainty. Similarly, Taylor skill showed a high degree of scatter (Figure 4), although two crop only models (SiBcrop and AgroIBIS), LoTEC, and ISOLSM were more conservative and showed a general high degree of consistency with observations. [18] Among crop models, SiBCrop and AgroIBIS per- formed well, especially in climatic spring and during wet conditions. In contrast, the crop only DNDCmodel exhibited poor model‐data agreement with c2 > 15 in climatic spring and summer as well as across all drought levels. Although four crop only simulators were analyzed, the best agreement in croplands (NMAE and c2 closer to zero and unity, respectively) was achieved by SiB3 and Ecosys, models used in multiple biomes. Based on all three skill metrics the LoTEC model (NMAE = −0.42, c2 = 2.17, S = 0.95) was most consistent with observations across all sites, dryness levels, and climatic seasons. This platform was optimized using a data assimilation technique, unique among model runs evaluated here, and was applied at 10 sites. In addi- tion, the mean model ensemble (MEAN) performed well (NMAE = −0.74, c2 = 3.35, S = 0.80). For individual models (n = 12) used at a wider range of sites (at least 24 sites), model consistency with observations was highest for Ecosys (NMAE = −0.69, c2 = 7.71, S = 0.94) and lowest for CN‐CLASS (NMAE = −1.50, c2 = 29.87, S = 0.48). [19] Site‐level model‐data agreement also showed a high degree of variability (Figure 4). At three croplands sites (US‐Ne1, US‐Ne2, and US‐Ne3) Taylor skill ranged from zero to unity. Both NMAE and c2 exhibited similar Figure 1. Normalized mean absolute error (NMAE) by biome for each model. Biomes in ascending order based on model‐specific NMAE; biomes on the left show better average agreement with observa- tions. NMAE is normalized by mean observed flux. Across all sites, seasons, and drought levels within a given biome this value is negative (NEE < 0), indicating a sink. NMAE values closer to zero coincide with a higher degree of model‐data agreement. Woody savannahs and shrublands not shown: only one site each. Tundra (n = 2 sites) has NMAE < −10 for all models. CN‐CLASS croplands value is off‐scale (= −8.98). Black cross, no observations; white circle, undersampled (n < 100 months). SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 11 of 22 scatter by site (not shown). Even for the best predicted site (US‐Syv), S ranged from 0.19 to 0.95. Only two forested sites (CA‐Qfo and CA‐TP4) were predicted well (S > 0.5) by all models; whereas only one tundra site (US‐Atq) was consistently poorly predicted (S < 0.5). Despite the wide range in model performance, model skill (NMAE, c2, and S) was not correlated with the number of sites (p > 0.5) or biomes (p > 0.3) simulated, i.e., using a more general rather than a specialized model did not result in a loss in model performance. Also, model‐data agreement was not better at sites with longer data records (p > 0.1). [20] The steady state protocol had negligible effect on model skill. Long‐term simulated NEE by site and model varied from −2904 to 2227 g C m−2 yr−1 with 90% of all values between −600 and 100 g C m−2 yr−1. The extreme values were primarily croplands simulated outside of crop only models. Overall, only 5 models achieved steady state (simulated NEE→ 0) over the full simulation: Biome‐BGC, LPJ, SiBCASA, SiB3, and TECO. Similar to simulated values, observed annual integrals at the 44 sites examined did not show steady state (Table 1) and varied from −718 to 571 g C m−2 yr−1. Nonetheless, model skill was not related to how close model spinup and initial conditions approxi- mated steady state or how close a given site was to an observed NEE of zero. All three skill metrics were uncor- related with long‐term observed or simulated average annual NEE (p > 0.05). However, two models did show significant relationships: For Ecosys, c2 increased (decrease in model skill) and S decreased as observed or simulated NEE approached zero; a system closer to steady state was coincident with less model‐data agreement. BEPS was similar, showing lower S and more negative NMAE (decrease in model skill) for sites closer to steady state. 3.3. Model and Site‐Specific Consistency With Observations Using Taylor Diagrams [21] Average model performance (both across‐site and across‐model) was evaluated using Taylor diagrams based on all simulated and observed monthly NEE data. Better model performance was indicated by proximity to the benchmark, representing the observed state. The benchmark was normalized by observed standard deviation such that the distance of s and RMSE from the benchmark was in observed s units. Similar to model skill metrics, forested sites were better predicted than nonforested ones. The MEAN model showed r ≥ 0.2, apart from CA‐SJ2 and US‐Atq, but generally (33 of 44 sites) underpredicted the variability associated with monthly NEE at forested (Figure 5) and nonforested (Figure 6) sites. Similarly, 40 of 44 sites were predicted with RMSE < s. Also 8 (6 forested and two croplands sites: CA‐Obs, CA‐Qfo, CA‐TP4, US‐Ho1, US‐IB1, US‐MMS, US‐Ne3, US‐UMB) of the Figure 2. Normalized mean absolute error (NMAE) by climatic season and drought level. NMAE is nor- malized by mean observed flux such that most values are negative (NEE < 0), indicating a sink. Positive values indicate a source (NEE > 0). These occur in winter for all models as well as spring and fall for all crop only models: AgroIBIS, DNDC, EPIC, SiBcrop. Such values are displayed on the same color bar but with opposite sign. Off‐scale values: AgroIBIS and SiBcrop in fall are −7.1 and −11.1, respectively. DNDC in fall and spring is −11.4 and −8.7, respectively. Black cross, no observations; white circle, undersampled (n < 100 months). SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 12 of 22 44 sites were predicted with r ≥ 0.95 and RMSE < 1. The worst predicted site was CA‐SJ2 with r = −0.67, s = 4.3, and RMSE = 5.1. [22] Overall model performance, aggregated across sites, was similar (Figure 7). Most models underpredicted vari- ability and showed RMSE < s. Of all 22 models only DNDC exhibited r < 0.2. Based on proximity to the benchmark, i.e., a high S value (Figure 3), the best models were: EPIC (crop only model used on one site), ISOLSM (used on 9 sites), LoTEC (data assimilation model), SiBcrop and AgroIBIS (crop only models), EDCM (used on 10 sites), Ecosys and SiBCASA (models used on most sites, 39 and 35, respectively), and MEAN (mean model ensemble for all 44 sites). All of these “best” models had r > 0.75, RMSE < 0.75 and slightly underpredicted variability; except the crop only models and Ecosys where variability was overpredicted. Models whose average behavior was furthest away from the benchmark were DNDC followed by BEPS. 3.4. Links Between Model Skill, Model Structure, and Site History [23] Biome classification was the most important factor in the distribution of model skill (Figure 8) sampled across all combinations of site, model, climatic season, and drought (n = 3132 groups). Climatic season and stand age, the highest scored site‐specific attribute, followed biome as lead determinants of model skill. Of the 12 evaluated site dis- turbances (Table 3) only grazing, which occurred on crop- lands, grasslands, and woody savannahs, achieved an importance score of at least 25. Apart from drought and grazing activity, the remaining determinants were model‐ specific: the number of soil layers, vegetation pools, canopy phenology, and soil pools. Two carbon flux calculations also had a variable score > 25, with NEE being the highest. [24] Comparing mean S for these relatively important model attributes (Figure 9) revealed three instances where model structure showed a statistically significant relation- ship with model skill: prescribed canopy phenology, a daily time step, and calculating NEE as the difference between GPP and ecosystem respiration. Models using canopy characteristics and phenology prescribed from remotely sensed products achieved higher skill (S = 0.54) than either prognostic or semiprognostic models (S = 0.43; p < 0.05). Using a daily time step showed lower model skill (S = 0.40) relative to nondaily time steps (S = 0.50; p < 0.05). Finally, calculating NEE as the difference between GPP and total ecosystem respiration showed greater skill (S = 0.50) than other calculation methods (S = 0.42; p < 0.05). None of the other model attributes we studied showed statistically sig- nificant relationships between model structure and skill. [25] While not statistically significant, both vegetation pools and soil layers exhibited a weak pattern whereby the simplest and most complex models showed higher skill than models of intermediate complexity (Figure 9). Models with no soil model (zero soil layers) or no vegetation pools showed greater skill than models with the simplest soil model or smallest number of vegetation pools. As the number of soil layers or pools increased, so did model skill, indicating that a more comprehensive treatment of biologi- cal and physical processes can improve model skill. For vegetation pools, there was a limit where increased com- Figure 3. Model skill metrics for all 22 models. Skill metrics are Taylor skill (S; equation (3)), normalized mean absolute error (NMAE), and reduced c2 statistic (c2). Better model‐data agreement corresponds to the upper left corner. Benchmark represents perfect model‐data agreement: S = 1,NMAE = 0, and c2 = 1. Gray interpolated surface added and model names jittered to improve readability. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 13 of 22 plexity beyond eight pools did not improve model‐data agreement. [26] Despite these effects, model attributes were of sec- ondary importance. The change in S relative to biome varied from 0.28 to 0.55; a much larger range than seen for model attributes. Similarly, the high variable importance scores for biome and climatic season, as well as the lower score for drought level, corroborated the relationships between these factors and model skill as seen with NMAE and c2. While the regression tree algorithm achieved an accuracy of 68.5% for predicting Taylor skill class, the site history and model characteristics considered here did not explain the underly- ing cause of biome and seasonal differences in model skill. 4. Discussion 4.1. Effect of Parameter Sets on Model Performance [27] Model parameter sets are a large source of variability in terms of model performance [Jung et al., 2007b]. They influence output and accuracy [Grant et al., 2005] and are more important for accurately simulating CO2 exchange than capturing effects of interannual climatic variability [Amthor et al., 2001]. For at least some of the models studied here this can be related to the use of biome‐specific parameters relative to within‐biome variability [Purves and Pacala, 2008]. A corollary occurs in the context of EC observations as tower footprints can exhibit heterogeneity, particularly in soils, that is not reproduced in site‐specific parameters [Amthor et al., 2001]. [28] The importance of model parameter sets was visible in this intercomparison in two ways. First, biome had the highest variable importance score. Insomuch as models rely on biome‐specific parameter values, this finding indicates that model parameter sets are a key factor in the distribution of model skill. This extends to plant functional types due to the high degree of overlap between both. Furthermore, the variability (Figure 4) in model skill across parameter sets, i.e., across models, underscores that biomes may be too het- erogeneous in time [Stoy et al., 2005, 2009] and space to be well‐represented by constant parameters relative to, e.g., within‐biome climate variability [Hargrove et al., 2003]. Second, the general high degree of site‐specific variation in model skill (Figure 4) suggested that model parameter sets may need to be refined to capture local, site‐specific realities. 4.2. Effect of Model Structure on Model Performance [29] In general, models with the highest model‐data agreement all used prescribed canopy phenology, calculated NEE as the difference between GPP and ecosystem respira- tion, and did not use a daily time step. Models that exhibited all of these structural characteristics (SiBCASA, SiB3, and ISOLSM) showed high degrees of model‐data agreement across all three skill metrics. Similarly, Ecosys, which used a prognostic canopy but otherwise had similar structural characteristics as SiBCASA, also performed well. Relative to model complexity, consistency with observations was highest in those models with either the simplest structure (e.g., one soil carbon pool in ISOLSM) or the most complex (e.g., SiBCASA with 13 carbon pools). Models with a prognostic canopy seem to perform better with more carbon pools and soil layers (e.g., Ecosys). No model with a prognostic canopy and a low number of carbon pools and soil layers placed in the top tercile of model skill for any skill metric, except SiBcrop and AgroIBIS for Taylor skill in croplands. Using multimodel ensembles (MEAN) or data assimilation to optimize model parameter sets (LoTEC) can compensate for differences in model structure to improve model skill. [30] The relationships between model structure and model skill were consistent across all biomes. As a whole, the models performed better at forested sites than nonforested sites, but the same models showed the highest consistency Figure 4. Boxplots of Taylor skill by model and site. Taylor skill (S; equation (3)) is a single value summary of a Taylor diagram where unity indicates perfect agreement with observations. Panels show interquartile range (blue box), median (solid red line), range (whiskers), and outliers (red cross; values more than 1.5 × interquartile range from the median). (top) Only models (n = 21) used on at least two sites shown. (bottom) Only sites (n = 32) simulated with at least 10 unique models, excluding the mean model ensemble (MEAN) and the assimilated LoTEC, shown. Models and sites sorted by median Taylor skill. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 14 of 22 with observations in each biome (Ecosys and SiB3). This is true even for agriculture sites, where Ecosys and SiB3 scored as high as crop only models. This suggests that any model with requisite structural attributes can successfully simulate carbon flux in all types of ecosystems. 4.3. Links Between Model Performance and Environmental Factors [31] Model skill was only weakly linked to drought, showing high variability across dryness level by biome and model. Only during the warm season (all climatic seasons excluding winter) did aggregate model skill decline under drought conditions. While this points to process uncertainty [Sitch et al., 2008], ecosystem response to longer‐term drought can exhibit lags and positive feedbacks [Arnone et al., 2008; Granier et al., 2007; Thomas et al., 2009; Williams et al., 2009] that were not explicitly included in the drought metric used here but did influence simulation behavior through model structure, e.g., soil moisture model and soil resolution. [32] In spring and fall, especially for biomes with a sig- nificant deciduous component, models showed a decline in model skill (Table 4) relative to periods of peak biological activity (climatic summer) [see also Morales et al., 2005]. While this was more pronounced for NMAE (Table 4) than c2 (Table 5), phenological cues are known to influence the annual carbon balance at multiple scales [Barr et al., 2007; Delpierre et al., 2009; Keeling et al., 1996]. The loss of model skill seen in this study during spring and fall was likely linked to poor treatment of leaf initiation and senes- cence as well as season‐specific effects of soil moisture and soil temperature on canopy photosynthesis [Hanson et al., 2004]. In this study seasonality was second only to biome in driving model skill (Figure 8). This and the lack of link between model skill and site history strongly implicate phenology as a needed refinement of terrestrial biosphere simulators. [33] The evergreen needleleaf forest biome diverged in performance based on whether the sites were located in the temperate or boreal zones. A similar divergence was reported using Biome‐BGC, LPJ, and ORCHIDEE to simulate gross CO2 uptake across a temperature gradient in Europe [Jung et al., 2007a]; average relative RMSE was higher for evergreen needleleaf forests in the boreal zone. This was linked to an overestimation of LAI at the boreal sites and relationships between resource availability and leaf area [Friedlingstein et al., 2006; Jung et al., 2007a; Sitch et al., 2008]. Additionally, recent observations in the circumboreal region, where all boreal evergreen needleleaf forested sites are located, suggest that transient effects of climate change, e.g., increased severity and intensity of natural disturbances (fire, pest outbreaks) and divergence from climate normals in temperature, have already occurred [Soja et al., 2007] and influence resource availability. We speculate the loss of Figure 5. Taylor diagram of normalized mean model performance for forested sites. Each circle (n = 26 sites) is the site‐specific mean model ensemble (MEAN). Benchmark (red square) corresponds to observed normalized monthly NEE; units of s and RMSE are multiples of observed s. Color coding of site letter and circles indicates biome: evergreen needleleaf forest‐ temperate zone (red), deciduous broadleaf forest (brown), mixed (deciduous/evergreen) forest (blue), evergreen needleleaf forest‐boreal zone (black). Outlying sites (evergreen needleleaf forest‐boreal zone) not shown: CA‐SJ1 (r = 0.81, s = 3.9, RMSE = 3.1) and CA‐SJ2 (r = −0.67, s = 4.3, RMSE = 5.1). SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 15 of 22 model skill in boreal relative to temperate evergreen needleleaf forests was linked to insufficient characterization of cold temperature sensitivity of metabolic processes and water flow in plants as well as freeze‐thaw dynamics [Schaefer et al., 2007, 2009] and that this was exacerbated by the effects of transient climate change. 4.4. Effects of Site History and Protocol on Model Evaluation [34] Disturbance regime and how a model treats distur- bance are known to impact model performance [Ito, 2008]. In this study, stand age impacted model skill whereas site history was of marginal importance (Figure 8). However, CA‐SJ2, the worst predicted site (Figure 5), was harvested in 2000 and scarified in 2002, and US‐SO2, a second poorly predicted shrubland site (Figure 6), suffered cata- strophic wildfire during the analysis period. The poor model performance for recently disturbed sites followed from assumed steady state as used in some simulations and the absence of modeling logic to accommodate disturbance. However, the distribution of site history metrics was skewed; only few sites were burned, harvested, or in the early stages of recovery from disturbance when NEE is more nonlinear relative to established stands. Furthermore, age class was biased toward older stands; of the 17 forested sites only one was classified as a young stand. Other site characteristics were also unbalanced; all nonforested biomes occurred on five or less sites; with only one site each for shrublands and woody savannahs. While regression trees are inherently robust, additional observed and simulated fluxes in rapidly growing young forested stands, recently burned or harvested sites, and undersampled biomes are desirable to better characterize model performance. [35] Aspects of the NACP site synthesis protocol and analysis framework also influenced the interpretation of our results. First, this analysis focused solely on non‐gap‐filled data to allow the model‐data intercomparison to inform model development. However, the low turbulence (friction velocity) filtering removed more data at night than during the day. Average data coverage across all sites was 82% for daytime and 39% at night, respectively (Table 2), so our analysis is skewed toward daytime conditions. Second, each model that used remotely sensed inputs (such as LAI) repeated an average seasonal cycle calculated from site‐ specific time series based on all pixels within 1 km of the tower site. This likely deflated relevant variable importance scores (Figure 8) and precluded a full comparison of pre- scribed versus prognostic LAI. While only few models used such inputs (Table 1), removing the inherent bias of an invariant seasonal cycle over multiple years may improve model performance. Incorporating disturbance information to recreate historical land use and disturbance, especially for recent site entries, could also improve model performance. Last, despite the model simulation protocol’s emphasis on steady state, this condition was not achieved for most sites (Table 2), even when discounting observational uncertainty, Figure 6. Taylor diagram of normalized mean model performance for nonforested sites. Each circle (n = 16 sites) is the site‐specific mean model ensemble (MEAN). Benchmark (red square) corresponds to observed normalized monthly NEE; units of s and RMSE are multiples of observed s. Color coding of site letter and circles indicates biome: croplands (red), grasslands (brown), wetlands (blue), all other biomes (black). SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 16 of 22 Figure 8. Variable importance scores for model‐specific (blue) and site‐specific (green) predictors. Scores were generated from a regression tree with the Taylor skill classes based on terciles (n = 3132) as the response. Only the 12 of 28 predictants with score > 25 shown; see Table 3 for complete listing of evaluated model structural and site attributes. Figure 7. Taylor diagram of normalized across‐site average model performance. Model s and RMSE were normalized by observed s. Each circle (n = 22 models) corresponds to the mean across all sites. Benchmark (red square) corresponds to observed normalized monthly NEE; units of s and RMSE are multiples of observed s. Color coding of model letter and circles indicates generality of model perfor- mance: specialist models used only in croplands (n ≤ 5 sites; black), generalist models used across a range of biomes and sites (n ≥ 30 sites, blue), all other models (red). The correlation for DNDC (r = −0.13) is displayed as zero for readability. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 17 of 22 or most models. None of the four crop only models achieved steady state. This followed from site history of croplands in general where active management precluded any system steady state, e.g., DNDC allowed for prescribed initial soil carbon pools. For those models (5 of the 21 evaluated) that achieved steady state in initialization this resulted in an inherent bias between simulated and observed NEE for all sites regardless of site history. However, as biome and seasonality largely governed the distribution of model skill, this bias was too small to manifest itself in this study. Relaxing the steady state assumption [Carvalhais et al., 2008] or initializing using observed wood biomass and the quasi‐steady state assumption [Schaefer et al., 2008] could improve these models’ performance. 5. Conclusion [36] We used observed CO2 exchange from 44 eddy covariance towers in North America with simulations from 21 terrestrial biosphere models and a mean model ensemble to examine model skill across gradients in dryness, sea- sonality, biome, site history, and model structure. Models’ ability to match observed monthly net ecosystem exchange was generally poor; the mean squared distance between observations and simulations was ∼10 times observational error. Overall, forested sites were better predicted than nonforested sites. Weaknesses in model performance concerned model parameter sets and phenology, especially for biomes with a clear seasonal cycle in leaf area index. Drought was weakly linked to model skill with abnormally dry conditions during the growing season showing mar- ginally worse model‐data agreement compared to nondry conditions. Sites with disturbances during the analysis period and undersampled biomes (grasslands, shrublands, wetlands, woody savannah, and tundra) also showed a large divergence between observations and simulations. The highest degree of model‐data agreement occurred in tem- perate evergreen forests in all climatic seasons and during summer across all biomes. Overall skill was higher for models that estimated net ecosystem exchange as the differ- ence between gross primary productivity and ecosystem respiration, used prescribed canopy phenology, and did not use a daily time step. The model ensemble (mean simulated value across all models) and an optimized model (para- meters tuned using data assimilation) also performed well. Models with preferred structural attributes included gener- alist models (models used at multiple sites and biomes, e.g., SiB3, Ecosys) that exhibited high degrees of model‐data agreement across all biomes, indicating that a single model can successfully simulate carbon flux in all types of eco- systems. That is, different model architectures were not needed for different types of ecosystems and model choice is recast as a function of ease of parameterization and initialization. [37] Acknowledgments. C.R.S., C.A.W., and K.S. were supported by the U.S. National Science Foundation grant ATM‐0910766. We would like to thank the North American Carbon Program Site‐Level Interim Synthesis team, the Modeling and Synthesis Thematic Data Center, and the Oak Ridge National Laboratory Distributed Active Archive Center for collecting, organizing, and distributing the model output and flux obser- vations required for this analysis. This study was in part supported by the U.S. National Aeronautics and Space Administration (NASA) grant NNX06AE65G, the U.S. National Oceanic and Atmospheric Administra- tion (NOAA) grant NA07OAR4310115, and the U.S. National Science Foundation (NSF) grant OPP‐0352957 to the University of Colorado at Boulder. Figure 9. Bar graphs of mean Taylor skill by model attribute. Whiskers represent one standard error of the mean. Only model‐specific attributes with variable important scores >25 shown. Note y axis on right panels starts at 0.4. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 18 of 22 References Amthor, J. S., et al. (2001), Boreal forest CO2 exchange and evapotranspira- tion predicted by nine ecosystem process models: Intermodel comparisons and relationships to field measurements, J. Geophys. Res., 106(D24), 33,623–33,648, doi:10.1029/2000JD900850. Arain, M. A., F. Yaun, and T. A. Black (2006), Soil‐plant nitrogen cycling modulated carbon exchanges in a western temperate conifer forest in Canada, Agric. For. Meteorol., 140, 171–192, doi:10.1016/j.agrformet. 2006.03.021. Arnone, J. A., et al. (2008), Prolonged suppression of ecosystem carbon dioxide uptake after an anomalously warm year, Nature, 455, 383– 386, doi:10.1038/nature07296. Baker, I. T., L. Prihodko, A. S. Denning, M. Goulden, S. Miller, and H. R. da Rocha (2008), Seasonal drought stress in the Amazon: Reconciling models and observations, J. Geophys. Res., 113, G00B01, doi:10.1029/ 2007JG000644. Baldocchi, D. (2008), Breathing of the terrestrial biosphere: Lessons learned from a global network of carbon dioxide flux measurement sys- tems, Aust. J. Bot., 56, 1–26, doi:10.1071/BT07151. Baldocchi, D., et al. (2001), FLUXNET: A new tool to study the temporal and spatial variability of ecosystem‐scale carbon dioxide, water vapor, and energy flux densities, Bull. Am. Meteorol. Soc., 82, 2415–2434, doi:10.1175/1520-0477(2001)082<2415:FANTTS>2.3.CO;2. Barr, A. G., et al. (2004), Inter‐annual variability in the leaf area index of a boreal aspen‐hazelnut forest in relation to net ecosystem production, Agric. For. Meteorol., 126, 237–255, doi:10.1016/j.agrformet.2004. 06.011. Barr, A. G., et al. (2007), Climatic controls on the carbon and water balances of a boreal aspen forest, 1994–2003, Global Change Biol., 13, 561–576, doi:10.1111/j.1365-2486.2006.01220.x. Barr, A., D. Hollinger, and A. D. Richardson (2009), CO2 flux measure- ment uncertainty estimates for NACP, Eos Trans. AGU, 90(52), Fall Meet. Suppl., Abstract B54A‐04. Bergeron, O., H. A. Margolis, T. A. Black, C. Coursolle, A. L. Dunn, A. G. Barr, and S. C. Wofsy (2007), Comparison of CO2 fluxes over three boreal black spruce forests in Canada, Global Change Biol., 13, 89–107, doi:10.1111/j.1365-2486.2006.01281.x. Bradford, J. B., R. A. Birdsey, L. A. Joyce, and M. G. Ryan (2008), Tree age, disturbance history, and carbon stocks and fluxes in subalpine Rocky Mountain forests, Global Change Biol., 14, 2882–2897, doi:10.1111/j.1365-2486.2008.01686.x. Breiman, L., J. H. Friedman, R. A. Olshen, and C. J. Stone (1984), Classi- fication and Regression Trees, 358 pp., Wadsworth, Belmont, Calif. Carvalhais, N., et al. (2008), Implications of the carbon cycle steady state assumption for biogeochemical modeling performance and inverse parameter retrieval, Global Biogeochem. Cycles, 22, GB2007, doi:10.1029/2007GB003033. Causarano, H. J., J. N. Shaw, A. J. Franzluebbers, D. W. Reeves, R. L. Raper, K. S. Balkcom, M. L. Norfleet, and R. C. Izaurralde (2007), Simulating field‐scale soil organic carbon dynamics using EPIC, Soil Sci. Soc. Am. J., 71, 1174–1185, doi:10.2136/sssaj2006.0356. Chen, J. M., A. Govind, O. Sonnentag, Z. Zhang, A. Barr, and B. Amiro (2006), Leaf area index measurements at Fluxnet‐Canada forest sites, Agric. For. Meteorol., 140, 257–268, doi:10.1016/j.agrformet.2006. 08.005. Ciais, P., et al. (2005), Europe‐wide reduction in primary productivity caused by the heat and drought in 2003, Nature, 437, 529–533, doi:10.1038/nature03972. Cook, B. D., et al. (2004a), Carbon exchange and venting anomalies in an upland deciduous forest in northern Wisconsin, USA, Agric. For. Meteorol., 126, 271–295, doi:10.1016/j.agrformet.2004.06.008. Cook, E. R., C. A. Woodhouse, C. M. Eakin, D. M. Meko, and D. W. Stahle (2004b), Long‐term aridity changes in the western United States, Science, 306, 1015–1018, doi:10.1126/science.1102586. Cook, E. R., R. Seager, M. A. Cane, and D. W. Stahle (2007), North American droughts: Reconstructions, causes and consequences, Earth Sci. Rev., 81, 93–134, doi:10.1016/j.earscirev.2006.12.002. Dai, A., K. E. Trenberth, and T. Qian (2004), A global data set of Palmer Drought Severity Index for 1870–2002: Relationship with soil moisture and effects of surface warming, J. Hydrometeorol., 5, 1117–1130, doi:10.1175/JHM-386.1. Davis, K. J., P. S. Bakwin, C. X. Yi, N.W. Berger, C. L. Zhao, R.M. Teclaw, and J. G. Isebrands (2003), The annual cycles of CO2 and H2O exchange over a northern mixed forest as observed from a very tall tower, Global Change Biol., 9, 1278–1293, doi:10.1046/j.1365-2486.2003.00672.x. Delpierre, N., et al. (2009), Exceptional carbon uptake in European forests during the warm spring of 2007: A data‐model analysis, Global Change Biol., 15, 1455–1474, doi:10.1111/j.1365-2486.2008.01835.x. Desai, A. R., P. V. Bolstad, B. D. Cook, K. J. Davis, and E. V. Carey (2005), Comparing net ecosystem exchange of carbon dioxide between an old‐growth and mature forest in the upper midwest, USA, Agric. For. Meteorol., 128, 33–55, doi:10.1016/j.agrformet.2004.09.005. Fischer, M. L., D. P. Billesbach, W. J. Riley, J. A. Berry, and M. S. Torn (2007), Spatiotemporal variations in growing season exchanges of CO2, H2O, and sensible heat in agricultural fields of the southern Great Plains, Earth Interact., 11, 1–21. Flanagan, L. B., L. A. Wever, and P. J. Carlson (2002), Seasonal and inter- annual variation in carbon dioxide exchange and carbon balance in a northern temperate grassland, Global Change Biol., 8, 599–615, doi:10.1046/j.1365-2486.2002.00491.x. Friedlingstein, P., et al. (2006), Climate‐carbon cycle feedback analysis, results from the C4MIP model intercomparison, J. Clim., 19, 3337– 3353, doi:10.1175/JCLI3800.1. Granier, A., et al. (2007), Evidence for soil water control on carbon and water dynamics in European forests during the extremely dry year: 2003, Agric. For. Meteorol., 143, 123–145, doi:10.1016/j.agrformet. 2006.12.004. Grant, R. F., et al. (2005), Intercomparison of techniques to model high temperature effects on CO2 and energy exchange in temperate and boreal coniferous forests, Ecol. Modell., 188, 217–252, doi:10.1016/j.ecolmodel. 2005.01.060. Griffis, T. J., T. A. Black, K. Morgenstern, A. G. Barr, Z. Nesic, G. B. Drewitt, D. Gaumont‐Guay, and J. H. McCaughey (2003), Ecophysio- logical controls on the carbon balances of three southern boreal forests, Agric. For. Meteorol., 117, 53–71, doi:10.1016/S0168-1923(03)00023-6. Gu, L., T. Meyers, S. G. Pallardy, P. J. Hanson, B. Yang, M. Heuer, K. P. Hosman, J. S. Riggs, D. Sluss, and S. D. Wullschleger (2006), Direct and indirect effects of atmospheric conditions and soil moisture on surface energy partitioning revealed by a prolonged drought at a temperate forest site, J. Geophys. Res., 111, D16102, doi:10.1029/2006JD007161. Hanson, P. J., et al. (2004), Oak forest carbon and water simulations: Model intercomparisons and evaluations against independent data, Ecol. Monogr., 74, 443–489, doi:10.1890/03-4049. Harazono, Y., M. Mano, A. Miyata, R. C. Zulueta, and W. C. Oechel (2003), Inter‐annual carbon dioxide uptake of a wet sedge tundra ecosys- tem in the Arctic, Tellus, 55B(2), 215–231. Hargrove, W. W., F. M. Hoffman, and B. E. Law (2003), New analysis reveals representativeness of the AmeriFlux network, EOS Trans. AGU, 84(48), 529. Hochberg, Y., andA. C. Tamhane (1987),Multiple Comparison Procedures, 480 pp., Wiley, New York. Houghton, J. T., Y. Ding, D. J. Griggs, M. Noguer, P. J. van der Linden, D. Xia, K. Maskell, and C. A. Johnson (Eds.) (2001), Climate Change 2001: The Scientific Basis: Contributions of Working Group I to the Third Assessment Report of the Intergovernmental Panel on Climate Change, 881 pp., Cambridge Univ. Press, New York. Huntington, T. G. (2006), Evidence for intensification of the global water cycle: Review and synthesis, J. Hydrol., 319, 83–95, doi:10.1016/j.jhydrol. 2005.07.003. Ichii, K., et al. (2009), Multi‐model analysis of terrestrial carbon cycles in Japan: Reducing uncertainties in model outputs among different terres- trial biosphere models using flux observations, Biogeosci. Discuss., 6, 8455–8502, doi:10.5194/bgd-6-8455-2009. Irvine, J., B. E. Law, M. Kurpius, P. M. Anthoni, D. Moore, and P. Schwarz (2004), Age related changes in ecosystem structure and function and the effects on carbon and water exchange in ponderosa pine, Tree Physiol., 24, 753–763. Ito, A. (2008), The regional carbon budget of East Asia simulated with a terrestrial ecosystem model and validated using AsiaFlux data, Agric. For. Meteorol., 148, 738–747, doi:10.1016/j.agrformet.2007.12.007. Jung, M., et al. (2007a), Assessing the ability of three land ecosystem models to simulate gross carbon uptake of forests from boreal to Medi- terranean climate in Europe, Biogeosciences, 4, 647–656, doi:10.5194/ bg-4-647-2007. Jung, M., et al. (2007b), Uncertainties of modeling gross primary produc- tivity over Europe: A systematic study on the effects of using different drivers and terrestrial biosphere models, Global Biogeochem. Cycles, 21, GB4021, doi:10.1029/2006GB002915. Keeling, C. D., J. F. Chin, and T. P. Whorf (1996), Increased activity of northern vegetation inferred from atmospheric CO2 measurements, Nature, 382, 146–149, doi:10.1038/382146a0. Krinner, G., N. Viovy, N. de Noblet‐Ducoudré, J. Ogée, J. Polcher, P. Friedlingstein, P. Ciais, S. Sitch, and I. C. Prentice (2005), A dynamic global vegetation model for studies of the coupled atmosphere‐biosphere system, Global Biogeochem. Cycles, 19, GB1015, doi:10.1029/ 2003GB002199. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 19 of 22 Kucharik, C. J., and T. E. Twine (2007), Residue, respiration, and residuals: Evaluation of a dynamic agroecosystem model using eddy flux measure- ments and biometric data, Agric. For. Meteorol., 146, 134–158, doi:10.1016/j.agrformet.2007.05.011. Lafleur, P. M., N. T. Roulet, J. L. Bubier, T. R. Moore, and S. Frolking (2003), Interannual variability in the peatland‐atmosphere carbon dioxide exchange at an ombrotrophic bog, Global Biogeochem. Cycles, 17(2), 1036, doi:10.1029/2002GB001983. Larcher, W. (1995), Physiological Plant Ecology, 506 pp., Springer, Berlin. Li, H., J. Qiu, L. Wang, H. Tang, C. Li, and E. Van Ranst (2010), Modeling impacts of alternative farming management practices on greenhouse gas emissions from a winter wheat‐maize rotation system in China, Agric. Ecosyst. Environ., 135, 24–33, doi:10.1016/j.agee.2009.08.003. Liu, J., J. M. Chen, J. Cihlar, and W. Chen (1999), Net primary productivity distribution in the BOREAS region from a process model using satellite and surface data, J. Geophys. Res., 104, 27,735–27,754, doi:10.1029/ 1999JD900768. Liu, S., N. Bliss, E. Sundquist, and T. G. Huntington (2003), Modeling car- bon dynamics in vegetation and soil under the impact of soil erosion and deposition, Global Biogeochem. Cycles, 17(2), 1074, doi:10.1029/ 2002GB002010. Lokupitiya, E., S. Denning, K. Paustian, I. Baker, K. Schaefer, S. Verma, T. Meyers, C. J. Bernacchi, A. Suyker, and M. Fischer (2009), Incorpo- ration of crop phenology in Simple Biosphere Model (SiBcrop) to improve land‐atmosphere carbon exchanges from croplands, Biogeos- ciences, 6, 969–986, doi:10.5194/bg-6-969-2009. Loveland, T. R., B. C. Reed, J. F. Brown, D. O. Ohlen, J. Zhu, L. Yang, and J. W. Merchant (2001), Development of a global land cover characteristics database and IGBP DISCover from 1‐km AVHRR data, Int. J. Remote Sens., 21, 1303–1330. Luo, H., W. C. Oechel, S. J. Hastings, R. Zulueta, Y. Qian, and H. Kwon (2007), Mature semiarid chaparral ecosystems can be a significant sink for atmospheric carbon dioxide, Global Change Biol., 13, 386–396, doi:10.1111/j.1365-2486.2006.01299.x. Ma, S., D. D. Baldocchi, L. Xu, and T. Hehn (2007), Inter‐annual variability in carbon dioxide exchange of an oak/grass savanna and open grassland in California,Agric. For. Meteorol., 147, 157–171, doi:10.1016/j.agrformet. 2007.07.008. McCaughey, J. H., M. R. Pejam, M. A. Arain, and D. A. Cameron (2006), Carbon dioxide and energy fluxes from a boreal mixedwood forest eco- system in Ontario, Canada, Agric. For. Meteorol., 140, 79–96, doi:10.1016/j.agrformet.2006.08.010. McKee, T. B., N. J. Doeskin, and J. Kleist (1993), The relationship of drought frequency and duration to time scales, in Proceedings of the 8th Conference on Applied Climatology, January 17–22, 1993, pp. 179– 184, Am. Meteorol. Soc., Boston. Medlyn, B. E., A. P. Robinson, R. Clement, and R. E. McMurtrie (2005), On the validation of models of forest CO2 exchange using eddy covari- ance data: Some perils and pitfalls, Tree Physiol., 25, 839–857. Medvigy, D., S. C. Wofsy, J. W. Munger, D. Y. Hollinger, and P. R. Moorcroft (2009), Mechanistic scaling of ecosystem function and dynamics in space and time: Ecosystem demography model version 2, J. Geophys. Res., 114, G01002, doi:10.1029/2008JG000812. Meehl, G. A., and C. Tebaldi (2004), More intense, more frequent, and longer lasting heat waves in the 21st Century, Science, 305, 994–997, doi:10.1126/science.1098704. Moffat, A., et al. (2007), Comprehensive comparison of gap‐filling techni- ques for eddy covariance net carbon fluxes, Agric. For. Meteorol., 147, 209–232, doi:10.1016/j.agrformet.2007.08.011. Morales, P., et al. (2005), Comparing and evaluating process‐based eco- system model predictions of carbon and water fluxes in major European forest biomes, Global Change Biol., 11, 2211–2233, doi:10.1111/ j.1365-2486.2005.01036.x. Oberbauer, S. F., et al. (2007), Tundra CO2 fluxes in response to experi- mental warming across latitudinal and moisture gradients, Ecol. Monogr., 77, 221–238, doi:10.1890/06-0649. Papale, D., et al. (2006), Towards a standardized processing of net ecosys- tem exchange measured with eddy covariance technique: Algorithms and uncertainty estimation, Biogeosciences, 3, 571–583, doi:10.5194/bg-3- 571-2006. Peel, M. C., B. L. Finlayson, and T. A. McMahon (2007), Updated world map of the Köppen‐Geiger climate classification, Hydrol. Earth Syst. Sci., 11, 1633–1644, doi:10.5194/hess-11-1633-2007. Peichl, M., and M. A. Arain (2007), Allometry and partitioning of above‐ and below‐ground tree biomass in an age‐sequence of white pine forests, For. Ecol. Manage., 253, 68–80, doi:10.1016/j.foreco.2007.07.003. Post, W. M., et al. (2004), Enhancement of carbon sequestration in U. S. soils, BioScience, 54(10), 895–908, doi:10.1641/0006-3568(2004)054 [0895:EOCSIU]2.0.CO;2. Purves, D. W., and S. Pacala (2008), Predictive models of forest dynamics, Science, 320, 1452–1453, doi:10.1126/science.1155359. Ricciuto, D. M., A. W. King, L. Gu, and W. M. Post (2008), Estimates of terrestrial carbon cycle model parameters by assimilation of FLUXNET data: Do parameter variations cause bias in regional flux estimates?, Eos Trans. AGU, 89(53), Fall Meet. Suppl., Abstract B54A‐03. Ricciuto, D. M., P. E. Thornton, K. Schaefer, R. B. Cook, and K. J. Davis (2009), How uncertainty in gap‐filled meteorological input forcing at eddy covariance sites impacts modeled carbon and energy flux, Eos Trans. AGU, 90(52), Fall Meet. Suppl., Abstract B54A‐03. Richardson, A. D., and D. Y. Hollinger (2007), A method to estimate the additional uncertainty in gap‐filled NEE resulting from long gaps in the CO2 flux record, Agric. For. Meteorol., 147, 199–208, doi:10.1016/j. agrformet.2007.06.004. Richardson, A. D., D. Y. Hollinger, D. B. Dail, J. T. Lee, W. Munger, and J. O’Keefe (2009), Influence of spring phenology on seasonal and annual carbon balance in two contrasting New England forests, Tree Physiol., 29, 321–331, doi:10.1093/treephys/tpn040. Riley, W. J., C. J. Still, M. S. Torn, and J. A. Berry (2002), A mechanistic model of H2 18O and C18OO fluxes between ecosystems and the atmo- sphere: Model description and sensitivity analyses, Global Biogeochem. Cycles, 16(4), 1095, doi:10.1029/2002GB001878. Schaefer, K., T. Zhang, P. Tans, and R. Stöckli (2007), Temperature anom- aly reemergence in seasonally frozen soils, J. Geophys. Res., 112, D20102, doi:10.1029/2007JD008630. Schaefer, K., et al. (2008), Combined Simple Biosphere/Carnegie‐Ames‐ Stanford approach terrestrial carbon cycle model, J. Geophys. Res., 113, G03034, doi:10.1029/2007JG000603. Schaefer, K., T. Zhang, A. G. Slater, L. Lu, A. Etringer, and I. Baker (2009), Improving simulated soil temperatures and soil freeze/thaw at high‐latitude regions in the Simple Biosphere/Carnegie‐Ames‐Stanford Approach model, J. Geophys. Res., 114, F02021, doi:10.1029/ 2008JF001125. Schmid, H. P., C. S. B. Grimmond, F. Cropley, B. Offerle, and H. B. Su (2000), Measurements of CO2 and energy fluxes over a mixed hardwood forest in the mid‐western United States, Agric. For. Meteorol., 103, 357– 374, doi:10.1016/S0168-1923(00)00140-4. Schmid, H. P., H. B. Su, C. S. Vogel, and P. S. Curtis (2003), Ecosystem‐ atmosphere exchange of carbon dioxide over a mixed hardwood forest in northern lower Michigan, J. Geophys. Res. , 108(D14), 4417, doi:10.1029/2002JD003011. Schwalm, C. R., et al. (2006), Photosynthetic light use efficiency of three biomes across an east‐west continental‐scale transect in Canada, Agric. For. Meteorol., 140, 269–286, doi:10.1016/j.agrformet.2006.06.010. Schwalm, C. R., T. A. Black, K. Morgenstern, and E. R. Humphreys (2007), A method for deriving net primary productivity and component respiratory fluxes from tower‐based eddy covariance data: A case study using a 17‐year data record from a Douglas‐fir chronosequence, Global Change Biol., 13, 370–385, doi:10.1111/j.1365-2486.2006.01298.x. Schwalm, C. R., et al. (2010), Assimilation exceeds respiration sensitivity to drought: A FLUXNET synthesis, Global Change Biol., 16, 657–670, doi:10.1111/j.1365-2486.2009.01991.x. Seager, R. (2007), The turn of the century drought across North America: Global context, dynamics and past analogues, J. Clim., 20, 5527–5552, doi:10.1175/2007JCLI1529.1. Seager, R., M. F. Ting, and I. M. Held (2007), Model projections of an imminent transition to a more arid climate in southwestern North America, Science, 316, 1181–1184, doi:10.1126/science.1139601. Seager, R., A. Tzanova, and J. Nakamura (2009), Drought in the southeastern United States: Causes, variability over the last millennium and the poten- tial for future hydroclimate change, J. Clim., 22, 5021–5045, doi:10.1175/ 2009JCLI2683.1. Sheffield, J., and E. F. Wood (2008), Projected changes in drought occur- rence under future global warming from multi‐model, multi‐scenario, IPCC AR4 simulations, Clim. Dyn., 13, 79–105, doi:10.1007/s00382- 007-0340-z. Siqueira, M. B., G. G. Katul, D. A. Sampson, P. C. Stoy, J.‐Y. Juang, H. R. McCarthy, and R. Oren (2006), Multiscale model intercomparisons of CO2 and H2O exchange rates in a maturing southeastern US pine forest, Global Change Biol., 12, 1189–1207, doi:10.1111/j.1365-2486.2006. 01158.x. Sitch, S., et al. (2003), Evaluation of ecosystem dynamics, plant geography and terrestrial carbon cycling in the LPJ dynamic global vegetation model, Global Change Biol., 9, 161–185, doi:10.1046/j.1365-2486.2003. 00569.x. Sitch, S., et al. (2008), Evaluation of the terrestrial carbon cycle future plant geography and climate‐carbon cycle feedbacks using five Dynamic Global Vegetation Models (DGVMs), Global Change Biol., 14, 2015– 2039, doi:10.1111/j.1365-2486.2008.01626.x. SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 20 of 22 Soja, A. J., N. M. Tchebakova, N. H. F. French, M. D. Flannigan, H. H. Shugart, B. J. Stocks, A. I. Sukhinin, E. I. Varfenova, F. S. Chapin, and P. W. Stackhouse (2007), Climate induced boreal forest change: Pre- dictions versus current observations, Global Planet. Change, 56, 274– 296, doi:10.1016/j.gloplacha.2006.07.028. Stoy, P., G. Katul, M. Siqueira, J. Juang, H. McCarthy, H. Kim, A. Oishi, and R. Oren (2005), Variability in net ecosystem exchange from hourly to inter‐annual time scales at adjacent pine and hardwood forests: A wavelet analysis, Tree Physiol., 25, 887–902. Stoy, P., et al. (2009), Biosphere‐atmosphere exchange of CO2 in relation to climate: A cross‐biome analysis across multiple time scales, Biogeos- ciences, 6, 2297–2312, doi:10.5194/bg-6-2297-2009. Sulman, B. N., A. R. Desai, B. D. Cook, N. Saliendra, and D. S. Mackay (2009), Contrasting carbon dioxide fluxes between a drying shrub wet- land in northern Wisconsin, USA, and nearby forests, Biogesociences, 6, 1115–1126, doi:10.5194/bg-6-1115-2009. Suyker, A. E., S. B. Verma, and G. G. Burba (2003), Interannual variability in net CO2 exchange of a native tallgrass prairie, Global Change Biol., 9, 255–265, doi:10.1046/j.1365-2486.2003.00567.x. Syed, H. K., L. B. Flanagan, P. J. Carlson, A. J. Glenn, and K. E. Van Gaalen (2006), Environmental control of net ecosystem CO2 exchange in a treed, moderately rich fen in northern Alberta, Agric. For. Meteorol., 140, 97– 114, doi:10.1016/j.agrformet.2006.03.022. Taylor, J. R. (1996), An Introduction to Error Analysis, 327 pp., Univ. Sci. Books, Mill Valley, Calif. Taylor, K. E. (2001), Summarizing multiple aspects of model performance in a single diagram, J. Geophys. Res., 106, 7183–7192, doi:10.1029/ 2000JD900719. Thomas, C. K., B. E. Law, J. Irvine, J. G. Martin, J. C. Pettijohn, and K. J. Davis (2009), Seasonal hydrology explains interannual and seasonal var- iation in carbon and water exchange in a semi‐arid mature ponderosa pine forest in central Oregon, J. Geophys. Res., 114, G04006, doi:10.1029/2009JG001010. Thornton, P. E., S. W. Running, and M. A. White (1997), Generating sur- faces of daily meteorological variables over large regions of complex terrain, J. Hydrol., 190(3–4), 214–251, doi:10.1016/S0022-1694(96) 03128-9. Thornton, P. E., S. W. Running, and E. R. Hunt (2005), Biome‐BGC: Ter- restrial Ecosystem Process Model, Version 4.1.1. model product, Oak Ridge Natl. Lab. Distrib. Act. Arch. Cent., Oak Ridge, Tenn., doi:10.3334/ORNLDAAC/805. (Available at http://daac.ornl.gov) Tian, H. Q., G. Chen, M. Liu, C. Zhang, G. Sun, C. Lu, X. Xu, W. Ren, P. Pan, and A. Chappelka (2010), Model estimates of ecosystem net pri- mary productivity, evapotranspiration, and water use efficiency in the southern United States during 1895–2007, For. Ecol. Manage., 259, 1311–1327, doi:10.1016/j.foreco.2009.10.009. Trenberth, K. E., et al. (2007), Observations: Surface and atmospheric cli- mate change, in Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, edited by S. D. Solomon et al., pp. XX–XX, Cambridge Univ. Press, Cambridge, U. K. Urbanski, S., et al. (2007), Factors controlling CO2 exchange on timescales from hourly to decadal at Harvard Forest, J. Geophys. Res., 112, G02020, doi:10.1029/2006JG000293. Verma, S. B., et al. (2005), Annual carbon dioxide exchange in irrigated and rainfed maize‐based agroecosystems, Agric. For. Meteorol., 131, 77–96, doi:10.1016/j.agrformet.2005.05.003. Vickers, D., C. Thomas, and B. E. Law (2009), Random and systematic CO2 flux sampling errors for tower measurements over forests in the con- vective boundary layer, Agric. For. Meteorol., 149, 73–83, doi:10.1016/j. agrformet.2008.07.005. Weng, E., and Y. Luo (2008), Soil hydrological properties regulate grassland ecosystem responses to multifactor global change: A modeling analysis, J. Geophys. Res., 113, G03003, doi:10.1029/2007JG000539. Williams, C. A., N. P. Hanan, R. J. Scholes, and W. L. Kutsch (2009), Complexity in water and carbon dioxide fluxes following rain pulses in an African savanna, Oecologia, 161, 469–480, doi:10.1007/s00442- 009-1405-y. Williamson, T. B., D. T. Price, J. L. Beverly, P. M. Bothwell, B. Frenkel, J. Park, and M. N. Patriquin (2008), Assessing potential biophysical and socioeconomic impacts of climate change on forest‐based communities: A methodological case study, Nat. Resour. Can., Can. For. Serv., North. For. Cent., Edmonton, AB. Inf. Rep. NOR‐X‐415E. Zha, T., et al. (2009), Carbon sequestration in boreal jack pine stands following harvesting, Global Change Biol . , 15 , 1475–1487, doi:10.1111/j.1365-2486.2008.01817.x. Zhan, X. W., Y. K. Xue, and G. J. Collatz (2003), An analytical approach for estimating CO2 and heat fluxes over the Amazonian region, Ecol. Modell., 162, 97–117, doi:10.1016/S0304-3800(02)00405-2. Zhou, X. L., C. H. Peng, Q. L. Dang, J. F. Sun, H. B. Wu, and D. Hua (2008), Simulating carbon exchange in Canadian Boreal forests I: Model structure, validation, and sensitivity analysis, Ecol. Modell., 219, 287– 299, doi:10.1016/j.ecolmodel.2008.07.011. K. Schaefer, National Snow and Ice Data Center, University of Colorado at Boulder, Boulder, CO 80309, USA. (kevin.schaefer@nsidc.org) C. R. Schwalm and C. A. Williams, Graduate School of Geography, Clark University, Worcester, MA 01610, USA. (cschwalm@clarku.edu; cwilliams@clarku.edu) R. Anderson, Numerical Terradynamic Simulation Group, University of Montana, Missoula, MT 59812, USA. (ryan.anderson@ntsg.umt.edu) M. A. Arain, School of Geography and Earth Sciences, McMaster University, Hamilton, ON L8S 4K1, Canada. (arainm@mcmaster.ca) I. Baker and E. Lokupitiya, Atmospheric Science Department, Colorado State University, Fort Collins, CO 80523, USA. (baker@atmos.colostate. edu; erandi@atmos.colostate.edu) A. Barr, Climate Research Division, Atmospheric Science and Technology Directorate, Saskatoon, SK S7N 3H5, Canada. (alan.barr@ ec.gc.ca) T. A. Black, Faculty of Land and Food Systems, University of British Columbia, Vancouver, BC V6T 1Z4, Canada. (andrew.black@ubc.ca) G. Chen and H. Tian, School of Forestry and Wildlife Sciences, Auburn University, Auburn, AL 36849, USA. (chengu1@auburn.edu; tianhan@ auburn.edu) J. M. Chen and M. Sprintsin, Department of Geography and Program in Planning, University of Toronto, Toronto, ON M5S 3G3, Canada. (chenj@ geog.utoronto.ca; misprin@gmail.com) P. Ciais and L. Li, Laboratoire des Sciences du Climat et de l’Environnement, CE Orme des Merisiers, Gif sur Yvette, 91191 France. (philippe.ciais@cea.fr; longhui.li@lsce.ipsl.fr) K. J. Davis, Department of Meteorology, Pennsylvania State University, University Park, PA 16802, USA. (davis@meteo.psu.edu) A. Desai, Center for Climatic Research, University of Wisconsin‐ Madison, Madison, WI 53706, USA. (desai@aos.wisc.edu) M. Dietze, Department of Plant Biology, University of Illinois‐Urbana Champaign, Urbana, IL 61801, USA. (mdietze@life.uiuc.edu) D. Dragoni , Department of Geography, Indiana Universi ty, Bloomington, IN 47405, USA. (ddragoni@indiana.edu) M. L. Fischer, Atmospheric Science Department, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA. (mlfischer@lbl.gov) L. B. Flanagan, Department of Biological Sciences, University of Lethbridge, Lethbridge, AB T1K 3M4, Canada. (larry.flanagan@uleth.ca) R. Grant, Department of Renewable Resources, University of Alberta, Edmonton, AB T6G 2E3, Canada. (robert.grant@afhe.ualberta.ca) L. Gu and D. M. Riciutto, Environmental Sciences Division, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA. (lianhong‐gu@ornl. gov; ricciutodm@ornl.gov) D. Hollinger, Northern Research Station, USDA Forest Service, Durham, NH 03824, USA. (davidh@hypatia.unh.edu) R. C. Izaurralde, Joint Global Change Research Institute, Pacific Northwest National Laboratory and University of Maryland, College Park, MD 20740, USA. (cesar.izaurralde@pnl.gov) C. Kucharik, Department of Agronomy and Nelson Institute Center for Sustainability and the Global Environment, University of Wisconsin‐ Madison, Madison, WI 53706, USA. (kucharik@wisc.edu) P. Lafleur, Department of Geography, Trent University, Peterborough, ON K9J 7B8, Canada. (plafleur@trentu.ca) B. E. Law, College of Forestry, Oregon State University, Corvallis, OR 97331, USA. (bev.law@oregonstate.edu) Z. Li, ASRC Research and Technology Solutions, Sioux Falls, SD 57198, USA. (zli@usgs.gov) S. Liu, Earth Resources Observation and Science, Sioux Falls, SD 57198, USA. (sliu@usgs.gov) Y. Luo, Department of Botany and Microbiology, University of Oklahoma, Norman, OK 73019, USA. (yluo@ou.edu) S. Ma, Department of Environmental Science, Policy and Management and Berkeley Atmospheric Science Center, University of California, Berkeley, Berkeley, CA 94720, USA. (sma@berkeley.edu) H. Margolis, Centre d’études de la forêt, Faculté de foresterie, de géographie et de géomatique, Université Laval, Québec, QC G1V 0A6, Canada. (hank.margolis@sbf.ulaval.ca) R. Matamala, Argonne National Laboratory, Biosciences Division, Argonne, IL 60439, USA. (matamala@anl.gov) H. McCaughey, Department of Geography, Queen’s University, Kingston, ON K7L 3N6, Canada. (mccaughe@post.queensu.ca) R. K. Monson, Department of Ecology and Evolutionary Biology, University of Colorado at Boulder, Boulder, CO 80309, USA. (russell. monson@colorado.edu) SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 21 of 22 W. C. Oechel, Department of Biology, San Diego State University, San Diego, CA 92182, USA. (oechel@sunstroke.sdsu.edu) C. Peng and J. Sun, Department of Biology Sciences, University of Quebec at Montreal, Montreal, QC H3C 3P8, Canada. (peng.changhui@ uqam.ca; jianfeng_sun@yahoo.ca) B. Poulter, Swiss Federal Research Institute WSL, Birmensdorf, CH‐8903, Switzerland. (benjamin.poulter@wsl.ch) D. T. Price, Northern Forestry Centre, Canadian Forest Service, Edmonton, AB T6H 3S5, Canada. (dprice@nrcan.gc.ca) W. Riley, Climate and Carbon Sciences, Earth Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA. (wjriley@lbl.gov) A. K. Sahoo, Department of Civil and Environmental Engineering, Princeton University, Princeton, NJ 08544, USA. (sahoo@princeton.edu) C. Tonitto, Department of Ecology and Evolutionary Biology, Cornell University, Ithaca, NY 14853, USA. (ctonitto@cornell.edu) H. Verbeeck, Laboratory of Plant Ecology, Ghent University, 9000 Ghent, Belgium. (hans.verbeeck@ugent.be) S. B. Verma, School of Natural Resources, University of Nebraska‐ Lincoln, Lincoln, NE 68583, USA. (sverma1@unl.edu) SCHWALM ET AL.: MODELED VERSUS OBSERVED CO2 EXCHANGE G00H05G00H05 22 of 22