Decision-Support Experiments and Evaluations Using Seasonal to Interannual Forecasts and Observational Data

Comments reveived from
14 December 2005 through 30 January 2006

REVIEWERS:

Name:  Lenny Bernstein
Organization: L.S. Bernstein & Associates, L.L.C.
Mailing Address: 488 Kimberly Avenue, Asheville, NC 28804
Phone: (828) 250-0245
Fax:   None
E-mail:  
Area of Expertise: Climate change policy

Name:  Michael MacCracken
Organization: Climate Institute
Mailing Address: 1785 Massachusetts Avenue, NW, Washington DC 20036
Phone: 202-547-0104
Fax: 202-547-0111
E-mail(s):
Area of Expertise: Climate Change

GENERAL COMMENTS

Michael MacCracken, Climate Institute

First General Comment:

While the title is quite broad, the actual scope of this product is much narrower, focusing only on the water cycle. The title of the prospectus needs to be changed to conform to its scope in order not to be misleading to the public, Congress, or the scientific community.

Second General Comment:

I would suggest that this Prospectus, as written, is not consistent with the requirements and process of the Federal Advisory Committee Act (FACA). In particular, the description included in Section 5 about the drafting process and the description in Section 6 on the review process seems seriously at variance with what is required under FACA, under which this synthesis and assessment is supposed to take place; in fact, it reads much more as if this Prospectus was prepared under the original CCSP guidelines, which I understand the legal counsel offices of the agencies have said did not conform with FACA. In revising this draft Prospectus, it really needs to be made clearer that FACA has a number of requirements for the meetings of the advisory committee being public, the documents (including drafts) being available on a continuing basis through agencies at their Freedom of Information libraries, etc. I do not believe that FACA requires an external review process, though I think this is a good idea and the agency group could include certain types of provisions regarding the final review process, but I do not think all of the ones indicated here. In revising the text included in the Prospectus, I would urge a careful recheck of the text of FACA on the requirements and, as text is written, imagine imposing the requirements on other major advisory committees that have in the past been led by prominent citizens, former government officials, etc. This report is an advisory committee report and thus must, in its final form, be considered by all members of the advisory committee (e.g., through a final vote or formal sign-off process). While it is acceptable for agencies to offer comments for consideration during the review process and for the agencies to not accept the report until after the advisory committee has prepared a report that considers the scope called for in the report and after a suitable review has been conducted, in the end, this is a report of this committee, and the agencies are free to take it or leave it (recall the settlement of the first lawsuit against the US National Assessment, which made clear that advisory committee reports are not federal policy). The CCSP (in particular NOAA) is creating a federal advisory committee, seeking their input. Agencies, together or separately, do not get to dictate what the report says—if the these are to be agency reports (that is, the agencies have the last word on what they say), then they must be written by government employees without calling together an author team including external members. For the credibility of the report, and of the participating scientists, the FACA rules must be rigorously applied.

SPECIFIC COMMENTS

Lenny Bernstein, L.S. Bernstein & Associates, L.L.C.

Pg. 1, lines 14-15: Calling the demonstration of the prediction of ENSO a success story is unjustified.  The final sentence of this paragraph (lines 23-24) is far more accurate in saying: “The forecasts have demonstrated modest skill for strong ENSO events…”  What we have learned is that ENSO is a far more difficult phenomenon to predict than was once believed.  A little more modesty in this prospectus, to match the modest skill of prediction, would be appropriate.

Michael MacCracken, Climate Institute

Page 1, Line 15: “Earth” should be consistent capitalized throughout—it deserves this level of respect. On line 40 of page 1, the same phrase has “Earth” capitalized.

Page 3, Line 15: The phrase “relative skill and certainty” needs to be changed to something like “relative skill and level of confidence of the results.” It is meaningless to talk about “relative level of certainty” — something is either certain or it is not. One might talk about the degree of uncertainty (and everything in science will always be at least a bit uncertain, but if this is to be done a well-defined metric needs to be developed and defined. I would instead suggest, however, that the two-dimensional metric and lexicon being used by the IPCC be considered, giving both the relative likelihood of an outcome and a measure of the level of confidence in it.

Page 4, Line 17: Based on the text, it would seem that these additional authors do not become members of the Advisory Committee. It is fine for an advisory committee to invite additional authors to contribute text. Just note that FACA requires that materials provided to the committee need to become part of the record and be available as a record of the committee’s meetings and actions.  And, at the conclusion, it is the members of the advisory committee itself who need to be responsible for the report.

Page 4, Line 27 to Page 5, Line 8: It is somewhat surprising to me that the listings did not include a scientist or stakeholder representing either the Illinois State Water Survey (the preeminent state hydrologic service on agricultural and mid-continent issues) nor of the State of California Department of Water Resources (the preeminent state department in supplying water for irrigation and the public); these two cases would seem to be leading examples of potential case studies.

Page 6, Lines 15-36: The Prospectus here seems overly prescriptive given that a federal advisory committee is being appointed to offer their advice rather than simply be writers of what the agencies want written. I would suggest changing the use of the word “will” to something indicating that these are suggestions or they types of information wanted.

Page 6, Line 25: By FACA rules, all documents considered by the advisory committee need to be available for public inspection throughout the process, whether they are drafts or not.

Page 6, Line 34 to Page 8, Line 2: It is not at all likely that this process is consistent with FACA. The advisory committee needs to prepare its report, solicit review comments (and they can come from the agencies), and then prepare its final report, which it submits to the lead agency. NOAA and the agencies can then choose to endorse it or take exception—it is an advisory report. In the end, the full advisory committee, not just its leaders, need to indicate its concurrence or to make any final revisions, include appropriate dissenting views, etc.—but this needs to be the advisory committee’s report (and the report gains its credibility from the credibility of the overall process and the credibility of each author—and so each member of the committee is entitled to a final review of the report). REMEMBER, this report is NOT the agencies report—it is an advisory committee report that agencies can choose on their own to adopt in full or not on their own. Insisting that all agencies formally agree to it seems inappropriate to me—each agency is likely to want to incorporate the advisory committee’s findings into their considerations in their own specific manner. It would be fine to encourage all agencies to read the report and to indicate how they are incorporating its findings, but insisting that all agencies formally agree on everything seems a formula for delay and inefficiency.