Program Reviews

Peer Review

Since 1990, a group of conservation experts has been meeting ad hoc to discuss Gap Analysis, solve problems by brainstorming, and provide intellectual direction and leadership for its development. The group became known as the GAP working group. In the summer of 1993, the working group recommended that GAP undergo a peer review process by an independent panel. A panel was formed of 11 leading experts in the fields of land use planning, biogeography, plant and animal community ecology, remote sensing, geographic information systems, social science, and spatial statistics. The panel met twice in the second half of 1993 and produced a report in January of 1994 (Zube 1994).

The panel was charged "to evaluate the design of the program's methodology in light of its goals, and to characterize the potential limits of the program to address key questions and concerns." The following specific goals were identified for the panel:

• assess the biological rationale on which the Gap Analysis methodology is based;
• evaluate the adequacy of the methodology used to conduct Gap Analysis, including quality control and quality assurance procedures employed by the program;
• evaluate the utility of the program for policy-related and regional-to-site-level conservation, planning, and management;
• based on the above assessments, make recommendations grounded in science for any necessary or desirable improvements in the program.

In summary, the panel found that "GAP is truly a visionary program that brings modern computer-based technology to bear on the question of biodiversity and related changes in land management. In so doing, it has anticipated and broken ground for the National Biological Survey."

Strengths of the program identified by the panel included: its conceptual focus on the question of how to determine where conservation gaps occur; innovations in remote sensing and geographic data analysis; integration of state-level activities; flexibility in building partnerships; development of user-friendly products; interdisciplinary and interagency team-building; a bottom-up decentralized approach; and dissemination of findings in professional and peer-reviewed literature.

Challenges and unrealized opportunities that the panel identified were: the program is under-funded by a factor of four to five; communication among different GAP researchers needs to be improved; maintaining coordination among participating organizations without rigid control will remain a challenge; relationships with other federal programs (i.e., EMAP, C-CAP, NWI, NAWQA) will become more important as the NBS develops; the risk of misuse and misunderstanding of GAP information is likely to increase as GAP becomes increasingly popular.

The panel recommended:

• Adding professional staff having technical management and advisory expertise to work at the national level with full-time responsibilities in: statistics and quantitative validation; remote sensing; modeling of relations among habitat types, vegetation, and fauna; data management and dissemination; and long-term program planning and development.
• Producing an explicit statement of GAP goals and products as part of a long-range plan. The plan should be initiated prior to hiring additional staff.
• Producing a handbook for Gap Analysis.
• Producing a user's guide for new GAP participants.
• Developing and promoting an electronic bulletin board.
• Involving other federal agencies in the GAP process.
• Establishing an advisory panel.
• Continuing to provide regular, formal training programs and workshops for cooperators.
• Continuing dissemination of GAP findings through the professional and peer review literature.

The peer review process and its outcome were of great value to the program and its progress over the 1994 and 1995 fiscal years, providing direction and a focus on specific program issues. Since then, GAP has sought to meet each recommendation. For example, a handbook containing a statement of goals, standards, a user's guide, and products is now available. An electronic bulletin board is in place on the GAP World Wide Web Home Page, workshops on accuracy assessment have been held, at least 31 peer-reviewed publications have been produced, and 40 manuscripts are in press since these recommendations were made. Although it has not been possible to create scientific staff positions under the national program in the five areas recommended, GAP is striving to meet the panels recommendation for technical expertise through participating organizations and contracts with specialists. Efforts to meet the other recommendations can be found throughout the following sections.

Review by The National Council of the Paper Industry for Air and Stream Improvement, Inc. (NCASI)

In January of 1993, NCASI issued a request for proposals to review Gap Analysis. The topics to be reviewed included: (a) the reliability of databases used in Gap Analysis, (b) consequences of assumptions, (c) methods for quantifying and reporting uncertainties, (d) uses for Gap Analysis, (e) opportunities for improving the process, and (f) future research needs. The work was awarded to Curtis H. Flather of the USDA Forest Service and colleagues at Colorado State University and Oregon State University.

The review (Flather et al. 1994) provided three central findings and three recommendations. In the first finding, the authors state that literature supporting the concept of basing conservation goals on an overall spatial index of species richness is ambiguous. They suggest that a "set-coverage" approach (e.g., using subsets of taxa with which to plan for conservation) might be more robust. Second, that the impact of error on delineating species-rich areas needs to be examined further. Third, that pattern, as well as ecological process, should be considered simultaneously in geographically explicit conservation recommendations. In addition, the report notes that one of the strengths of GAP is its flexibility, in that any attribute that can be mapped (e.g., vegetation communities, species distributions, human population growth patterns, road densities, etc.) can be incorporated into Gap Analysis (Flather et al. 1994). This allows the opportunity to directly build and evaluate multiple scenarios for GAP-based conservation strategies.

Flather et al. (1994) then recommended that, based on their review of the literature, the assumption of using species richness subsets (such as all vertebrate species) as an indicator for biodiversity as a general rule should be abandoned, noting, however, that there were specific cases where the assumption held. The authors call for additional research to determine if the assumption is tenable and to evaluate the tradeoffs of using different diversity criteria in developing a conservation strategy. The second recommendation was to add the capability for assessing potential errors and error propagation in GAP data through sensitivity analysis. The third recommendation was that ecological processes be incorporated into the GAP data, though the authors recognize that doing so beyond current scientific capability would require substantial new research and development.

In closing, the authors state:

"An undeniable benefit of GAP is the motivation for states to update or develop vegetation and species distribution databases. This basic ecological information, once compiled, places state Gap Analysis projects in a unique position to describe biodiversity patterns, to propose and test hypotheses regarding the process behind the pattern, and to delineate those regions where such research may be the most fruitful. Such compilation and synthesis has the potential to be extremely useful in a wide range of natural resources and conservation management issues."

The Gap Analysis Program found this review constructive and helpful in scoping research needs. The issues of biodiversity indicators and error detection are part of ongoing research by GAP investigators and among partners, for example, the Biodiversity Research Consortium (BRC). Some of the work by the BRC and others shows that, in the strict sense, overall species richness may not efficiently capture and indicate broad-based biodiversity (e.g., Prendergast et al. 1993, Camm et al. in press, Kiester et al. in press). Rather, identifying areas based on complementarity using major taxonomic groups, such as birds or mammals, may be a better approach. The "sweep analysis" developed by Kiester et al. represents significant progress in this area. The issue of incorporating ecological process into the characterization of ecological pattern is important, however, the field is so lacking that it requires substantial basic research and development. While GAP researchers will attempt to make gains on this issue as they proceed with their state projects, and the GAP map may provide a useful sampling universe for the study of ecological processes, new funding for a project dedicated to solving this issue is required.