Volume No. 11, 2002
In reading O’Connor’s thought-provoking essay, we found ourselves having two very different reactions. The first is that we agree with virtually all of his points about habitat modeling. He wrote an equally thought-provoking chapter for the book “Predicting Species Occurrences: Issues of Accuracy and Scale” (O’Connor 2002) that raised many of the same criticisms about the field of habitat modeling as a whole, and the arguments were just as compelling there as they are here. However, our second response stems from this point: if the issues he raises are systematic problems with the science of habitat modeling, what should the response of a product-oriented, applied project such as the Gap Analysis Program (GAP) be? We will argue in our comment that the answer to this question is twofold: GAP should continue to use the best available data and standard practices in meeting its primary mission but should simultaneously encourage research into the sort of basic issues that O’Connor raises in his essay. Our response reflects O’Connor’s success in his stated purpose of stimulating discussion, because we have strayed substantially from a point-by-point response and offer additional thoughts of our own.
To begin, we feel it is necessary to review the original goals and objectives of GAP. Gap analysis is a coarse-filter approach to biodiversity protection (Scott et al. 1993). The process assumes that distributions of land cover and vertebrates correspond to overall biodiversity, and habitat can be used as an indirect indicator of animal species distribution. Scott et al.’s underlying belief was that we could prevent species from becoming endangered by maintaining biodiversity in natural landscapes. In other words, the mission of GAP was, and is, to provide the information necessary to keep common species common.
We recognize that publicly available data sets will be used in a variety of ways beyond their original intent, and we feel it is worthwhile to reiterate some of the limitations of gap analysis data originally identified by Scott et al. (1993). First, maps of vegetation only show habitat patches larger than the minimum mapping unit, and species responding to habitat at finer scales will likely be misrepresented. Second, GAP vegetation maps only portray the distribution of dominant, overstory cover types. Third, map boundaries are sharper than ecological boundaries. Fourth, because predicted vertebrate distribution maps do not reflect habitat quality, population density, or within-habitat distributions, they are best treated as hypotheses to be tested, refined, and retested. GAP models represent a simple (and perhaps often too simple) conceptual relationship between animals and their habitats, but one that can be applied broadly to a great many species, including ones about which little is known; information on problems of population viability, differences between source and sink habitats, interrelationships between species, and disturbance regimes must come from other sources. Therefore, GAP represents only a first step in comprehensive land conservation planning for any region.
The GAP approach is a GIS model overlay. Rather than representing statistical associations between animals and habitats, GAP models are mapped depictions of the description of habitat and known range extent for the species of interest. Therefore, appropriate uses of GAP models vary with development methods, scales, species knowledge base, and research objectives (see http://www.gap.uidaho.edu/Projects/Use.htm). With the historical development of GAP in mind, we need to address perceived deficiencies in these methods if the program is to continue being a useful part of conservation. In that spirit, we move on to current issues.
We second the point O’Connor calls the "organochlorines test”: habitat models do indeed only model habitat. A habitat model will not predict changes in distribution or abundance independent of the habitat. We also generally agree with his assertion that any habitat model, including GAP models, cannot be the sole basis for population monitoring. We may make an exception when the primary threat to a species is known to be habitat loss or alteration, in which case a habitat model may be a good predictor of population change (and may even be better than the typical, noisy population time series data that are used for population monitoring). Habitat change would need to be tracked on a reasonable time scale for this to be effective, but it is possible. GAP data could also be used to monitor changes in spatial configuration of habitat, which for some species could be a good predictor of population change. However, a great deal of information about the species is needed before this criterion can be met. For most vertebrates, we lack the kind of basic ecological information that was common in 19th century biology needed to meet this assumption (Laymon and Barrett 1986, Karl et al. 1999, Heglund 2002).
We also point out that the counterpoint to the "organochlorines test" is that absence of a species that has been extirpated from its habitat (due to poisoning or other reasons) does not indicate that the habitat is poor, nor will the species recover in the wild without adequate habitat. In short, GAP may be able to contribute the habitat information needed for comprehensive conservation planning, but the other needed information will have to come from other sources.
We generally concur with O’Connor about the need to consider omission and commission error separately in habitat models. The list of possible sources of prediction errors is long, and growing longer. We are currently working on what we believe to be a formerly unappreciated source of error that comes from rounding the predictions from continuous functions to whole numbers (i.e., rounding probabilities to 0 for absence and 1 for presence). This seemingly benign, completely standard practice can under some circumstances result in large numbers of prediction errors, because it lumps together model predictions that are essentially the toss of a coin with those that are near certainty. Although O’Connor’s point about omission and commission is well taken, we suggest that a comprehensive approach to error assessment is not yet available.
This point is important, because we do not yet know what constitutes a reasonable expectation of prediction accuracy. The disconnect between the strikingly obvious habitat associations we see everywhere we look, and the poor predictive accuracy of models based on these associations never ceases to amaze us. The difference between the questions, "Given that we are looking at an Acorn Woodpecker, what habitat are we standing in?" and "Given that we're in oak woodland, what are the chances that we will see an acorn woodpecker?" can be much greater than we would expect. When the uncertainty in the answer to the second question is great, we begin to lose confidence in our answer to the first. However, despite their rhetorical similarity, the two questions are actually very different. Consider the following example, adapted from an introductory mathematical statistics textbook (Bain and Engelhardt 1992). Consider a model of the woodpecker habitat association whose confusion matrix (i.e., the relationship between the model predicted outcome and the observed outcome) appears in Table 1. This is a good habitat model by anyone’s standards, with only 10 errors out of 200 predictions for the data that were used to build the model. Given that the model explains the occurrence of the woodpecker so well, should it not also predict well? In other words, shouldn’t the probability of seeing a woodpecker when we are standing in oak woodland be fairly close to 95%? To answer that question, we need Bayes' Theorem, which states:
The probabilities of observing the animal (Bj), or not observing the animal (Bi, or 1-Bj), at a random location in the landscape, without reference to the habitat, are called the “simple” or “prior” probabilities. For example, if the animal occurs in one of 50 survey points, Bj is 1/50, and Bi is 49/50. The conditional probabilities represent the information provided by the habitat model, which come from Table 1. For example, the probability that the model predicted a presence given an observed presence (or P(A | Bj)) is found by dividing the correctly predicted presences by the total predicted presences, which is 95/100. Similarly, the probability of a model-predicted presence given that the species was absent (or P(A | Bi )) is 5/100. Plugging in the values from Table 1, we get:
This means that, even in light of the enviably good habitat model shown in Table 1, we would only have a 27% chance of observing an Acorn Woodpecker, given that we were standing in the habitat where the species was predicted to occur. Thus, the predictive accuracy of the model depends both on how good the model is and on the amount of suitable habitat present in the landscape. If woodpeckers occur in 5/50 instead of 1/50 locations, the probability rises to 0.68―better, but far short of the 95% we would hope for. At a frequency of occurrence of 1/2, there is no difference between Table 1 and the Bayes' Theorem calculation, and as the frequency of occurrence increases above 0.5, the Bayes' Theorem calculation is even better than anticipated from Table 1 (for example, if woodpeckers are present at 80% of sampled locations, the probability of seeing one given that you are in oak woodland rises to 0.99).
|
Predicted |
|||
|
Present |
Absent |
||
|
Observed |
Present |
95 |
5 |
|
Absent |
5 |
95 |
|
We present this example because it points out that explanation and prediction are fundamentally different, and that under some circumstances even a model that accurately represents a strong animal-habitat relationship can predict spectacularly poorly. We share O'Connor's discomfort with some of the more conspicuous prediction failures in habitat ecology, but we also assert that we should not be too quick to abandon approaches until we understand the reasons for their failures (and anyone who is surprised by our example should acknowledge that we don't yet fully understand the reasons for our habitat model failures). We may find that habitat models will never predict some species well, but we want to be able to reach this conclusion with confidence and not wonder whether we actually didn’t really know the species-habitat relationship after all.
We would like nothing better than for the 10% “rule” to die a rapid and merciful death. We also agree with O’Connor that the spatial arrangement of habitat is not considered in GAP models, and that greater attention to the lessons of biogeography and landscape ecology could benefit GAP (more on this point in “GAP Research and Development” below). We add that we are extremely dubious that any fixed percentage of protected habitat will be adequate for all species (including 100% for species whose habitat is already mostly lost), and even if a conservative rule-of-thumb is to be chosen, 10% has no particular claim to being the right number; others have suggested levels of 12%, 20% and 50% (Odum and Odum 1972, Specht et al. 1974, Ride 1975, Miller 1984, Noss and Cooperrider 1994). Given the points raised in the Acorn Woodpecker example and the spatial (in)accuracies of vertebrate models, what would a fixed percentage mean for the conservation status of the species? Models such as GAP models that are based on current literature and expert opinion (Hepinstall et al. 2002) tend to overestimate habitat. However, returning to our Acorn Woodpecker example, if we protect 20% of predicted Acorn Woodpecker habitat, but we only have a 27% chance that the species will be present in any given sample, then those protected areas would really only be capturing approximately 5% of theoretically occupied habitat under the model. However convenient, fixed percentages do not represent these intricacies.
So, how much is enough? Although GAP “seeks to identify habitat types and species not adequately represented in the current network of biodiversity management areas” (GAP Handbook, Preface, Version 1, pg. I), it is unrealistic to create a standard definition of “adequate representation” for either land cover types or individual species (Noss et al. 1995). It is not known how much area is needed to protect biodiversity over the long term (Scott et al. 1987). The 10% threshold often reported in GAP reports is arbitrary and, while protecting 10% of a cover type may be a heroic accomplishment (Soulé and Sanjayan 1998), it lacks biological relevance and needs to be tested against the biological criteria of representation, redundancy, and resiliency (Shaffer and Stein 2000). A practical solution suggested by Scott et al. (2002a) is to report both percentages and absolute area in biodiversity management areas and allow the user to determine which vegetation types or vertebrate species are adequately represented, based on additional detailed studies of the ecology and population viability of the species as well as spatial and temporal dimensions of ecological processes. We suggest that probability of occurrence should also be reported.
Now that we have largely agreed with O’Connor’s points and added some concerns of our own, we would like to address how we believe GAP should respond to them. We see two very different kinds of research and development needs for GAP. The first kind of project is meant to refine existing methodology. Studies that fall into this category include improved remote sensing and classification techniques, studies of the best spatial resolution for models of particular species, accuracy assessment techniques, and other improvements in the methods used to conduct a traditional gap analysis. This sort of work may appear to be tinkering with details rather than addressing fundamental problems, but we expect incremental―but ultimately substantial―improvements in gap analysis to result from this kind of work. Most current GAP research falls in part or in total into this category, and it is necessary work. The second kind of research can be viewed as "futures" research, the kind of research and development that is intended to address fundamental problems with current methods and ultimately to allow us to move beyond them. Advances in the underlying scientific fields of population biology, landscape ecology, and biogeography should be investigated, interpreted, and brought to bear on conservation problems as soon as possible. We, as a field, may be chagrined that “developments” such as the Fretwell-Lucas dynamics mentioned by O’Connor that have been known for 30 years are not yet easily applied to habitat modeling, but it is so. We see O'Connor's suggested changes to GAP as one of each of these basic kinds of research; Schaefer's method for accuracy assessment is a proposed refinement of existing techniques, and Dufrêne-Legendre's index of habitat selection is a possible new direction based on a putative measure of habitat quality.
We feel this distinction is important, because GAP should respond differently to the findings of these different kinds of research projects. Having adopted a conceptual approach, we see no reason to delay implementing improvements in its application. Therefore, improvements in existing methods should be tested and deployed as soon as possible, so that the full potential of the method can be realized. In contrast, the findings from futures projects require a greater degree of evaluation and careful consideration. O'Connor's example of the Dufrêne-Legendre index, for example, represents a particular measure of only one of the many ways that patterns of distribution and abundance can overlay habitat quality. We have known for years that density can be a misleading indicator of habitat quality (Van Horne 1983), and the recent explosion of interest in ecological traps (e.g., Schlaepfer et al. 2002, Delibes et al. 2001, Donovan and Thompson 2001, Kristan in press) has shown that even an animal’s habitat preference can be misleading. Animals may be strongly attracted to others of their species, either because conspecifics are used as indicators of habitat quality, or because being close together increases mating opportunities independent of the habitat (Muller et al. 1997). Metapopulation dynamics can cause patches of habitat that are clustered together to be more consistently occupied than identical patches of habitat that are far apart; it stands to reason that in some cases lower-quality patches of habitat that are close together may contain animals more frequently than higher-quality patches that are far apart. Each of these different insights from basic ecological research suggests distinct, and sometimes contradictory, implications for habitat modeling. In other words, unless we are able to tell which of the long list of possible problems are actually occurring for a particular species, the list does not provide any information that we can use to improve our models. So, what should we do?
We suggest that these points do not mean that all of habitat-based conservation is valueless, but rather that until these issues are thoroughly understood, attempting to incorporate them into applied projects runs the risk of chasing the latest fads and infatuations. We do not suggest that GAP should stand still and wait for others to bring these advances to us, but rather we suggest that a substantial fraction of GAP-funded research and development should be designed to bring advances in population biology, landscape ecology, and biogeography to the program.
We agree whole-heartedly with O'Connor's assertion that habitat modeling needs to move forward, and we will close our comment with some suggestions for future directions (including research and development needs) we think will allow GAP to make continued progress.
1. Dynamic models - Information on the distributions and habitat associations of vertebrate species is probably the most incomplete of any of the information used in GAP projects. This informational deficiency, coupled with dynamic landscapes, requires an adaptive management approach. As has been stated since the beginning, models of predicted habitat are testable hypotheses and need to be treated as such. This means field verification at appropriate spatial and temporal scales, refinement of the models, and reevaluation. Examples of this approach are rare in the literature. Such an approach will allow us to incorporate changes in landscape characteristics, address regional differences in habitat associations, and create models that are effective as management tools.
2. Hierarchical approaches - As data at a variety of spatial resolutions become available, GAP should move beyond emphasizing a single spatial and temporal scale in species models. Plants and animals are exposed to multiple scales simultaneously, and the interacting effects of coarse- and fine-grained habitat features need to be better understood.
Similarly, hierarchical approaches to conservation planning are necessary (e.g., The Nature Conservancy). The original concept of gap analysis was proposed as one specific component of an integrated conservation program aimed at addressing the problem of declining biodiversity (Scott et al. 1987, Scott et al. 1988, Scott et al. 1993). Today’s products from GAP provide the context to proactively identify and manage species before they are threatened or endangered, but fine-scale assessments are still needed.
3. Move beyond presence/absence - The primary advantages of presence/absence predictions are that they are less likely to fail than are predictions of abundance, they are easy to explain to the lay-public, and they are in the same units as the observations, so that errors are easy to count. In truth, though, the presence or absence of a species at a point in space and time is a chance event and would more accurately be represented by a probability. Currently, we risk confusing a model that predicts badly (that is, fails to predict the presence or absence of species even when the probabilities of occurrence are close to 0 or 1) with a good model applied to a landscape full of marginal habitat (and in which probabilities are thus close to 0.5, where predictive accuracy is expected to be poor). Promising work is being done in this area (see Scott et al. 2002b).
4. Use of models as planning tools - GAP models are famous (and to some, infamous) for being "coarse-filter" models and numerous example applications exist (see Crist and Maxwell 2000). While coarse-filter approaches will not be sufficiently precise for some applications, we feel they are underutilized as a guide for sampling frameworks. One of us (LS) has successfully used models (both GAP and finer-scale versions) to guide survey efforts for the pygmy rabbit, a lesser-known species recently proposed for listing under the Endangered Species Act.
One of the original goals of GAP was, and still is, to facilitate conservation planning. While much has been done in this arena, much is left to do. Coarse-filter models may be sufficiently accurate to be used for "What If" analyses of different changes in land use (e.g., Matthews et al. 2002) in the way that population viability analysis is used to predict changes in viability under different management alternatives. Changes in species distributions due to climate change are occurring (Root et al. 2003), and GAP models may provide input into predicting these changes. Further assessments of the vulnerability or risk to biodiversity from human impacts such as roads and urban expansion are needed.
5. Education and awareness - If any of the information gathered, produced, and reported by GAP is going to make a difference in the overall conservation of biodiversity, the chasm between researchers and data users (land managers, nonprofits, etc.) must be crossed. As Wiens (2002) states, the variability, complexity, and contingencies that fascinate ecologists are not so appealing to managers, who seek simple and timely solutions. Not only is it important that we continue to educate ourselves to new ideas in the fields of conservation biology, biogeography, and statistics, but also to test, refine, and apply these ideas. As we stated earlier, publicly available data sets will be used in a variety of ways beyond their original intent, and those data will only be as valuable as the skill of the user. It is our responsibility as the GAP community to develop and implement methods to facilitate learning.
6. Facilitation- With thirty years since Landsat satellites were launched, and with more than two dozen space-borne sensors dedicated to recording land cover, it is time to take a hard look at how successful we have been in mapping the world’s land cover. Practitioners from around the world undoubtedly have much to share. Emerging research suggests that using data from more than one sensor can appreciably improve accuracy of our maps. GAP, as a leading practitioner of the art of land cover mapping, can facilitate retrospective analyses of common practices by hosting an international symposium where researchers from around the world are invited to present and discuss results of past research efforts and future research directions. This approach proved fruitful at the Snowbird, Utah, symposium on predicting species occurrences, which was born in part out of questions raised by GAP investigators (Scott et al. 2002b). It is important that all points of view are represented, and attendees should be asked to document what has and has not worked, explore emerging methods, identify the really tough issues, and critically address their assumptions.
We wish to reiterate that we agree with virtually all of O’Connor’s points. However, we chose to focus our comments on what the GAP program as a whole should do in light of these criticisms, because GAP is more than habitat modeling. As pointed out repeatedly in Scott et al. (2002b), there are no “silver bullet” methods of predicting species occurrences. Improved model performance will come with increased understanding of species ecology. Though our enthusiasm for the new and improved should push us to look for better solutions, we must also be cautious about trading old problems for new ones.
Bain, L.J., and M. Engelhardt. 1992. Introduction to probability and mathematical statistics. Duxbury Press, Belmont, California. 644 pp.
Crist, P., and J. Maxwell. 2000. Reporting the results of Gap Analysis. Version 2.1.0. A handbook for conducting Gap Analysis. Internet WWW page, at URL http://www.gap.uidaho.edu/handbook/FinalReportTemplate/default.htm.
Delibes, M., P. Gaona, and P. Ferreras. 2001. Effects of an attractive sink leading into maladaptive habitat selection. American Naturalist 158:277-285.
Donovan, T.M., and F.R. Thompson, III. 2001. Modeling the ecological trap hypothesis: A habitat and demographic analysis for migrant songbirds. Ecological Applications 11:871-882.
Heglund, P.J. 2002. Foundations of species-environment relations. Pages 35-41 in J.M. Scott, P J. Heglund, M.L. Morrison, J.B. Haufler, M.G. Raphael, W.A. Wall, and F.B. Samson, editors. Predicting species occurrences: Issues of accuracy and scale. Island Press, Washington, DC.
Hepinstall, J.A, W.B. Krohn, and S.A. Sader. 2002. Effects of niche width on the performance and agreement of avian habitat models. Pages 593-606 in J.M. Scott, P J. Heglund, M.L. Morrison, J.B. Haufler, M.G. Raphael, W.A. Wall, and F.B. Samson, editors. Predicting species occurrences: Issues of accuracy and scale. Island Press, Washington, DC.
Karl, J.W., N.M. Wright, P.J. Heglund, and J.M. Scott. 1999. Obtaining environmental measures to facilitate vertebrate habitat modeling. Wildlife Society Bulletin 27:357-365.
Kristan, W.B. III. In press. The role of habitat selection behavior in population dynamics: Source-sink systems and ecological traps. Oikos.
Laymon, S.A., and R.H. Barrett. 1986. Developing and testing habitat-capability models: Pitfalls and recommendations. Pages 87-92 in J. Verner, M.L. Morrison, and C.J. Ralph, editors. Wildlife 2000: Modeling habitat relationships of terrestrial vertebrates. University of Wisconsin Press, Madison, Wisconsin.
Matthews, S., R.J. O'Connor, and A.J. Plantinga. 2002. Quantifying the impacts on biodiversity of policies for carbon sequestration in forests. Ecological Economics 40:71-87.
Miller, K.R. 1984. The natural protected areas of the world. Pages 20-23 in J.A. McNeely and K.R. Miller, editors. National parks, conservation and development: The role of protected areas in sustaining society. Smithsonian Institution, Washington, DC.
Muller, K.L., J.A. Stamps, V.V. Krishnan, and N.H. Willits. 1997. The effects of conspecific attraction and habitat quality on habitat selection in territorial birds (Troglodytes aedon). The American Naturalist 150:650-661.
Noss, R.F., and A.Y. Cooperrider. 1994. Saving nature's legacy: Protecting and restoring biodiversity. Island Press, Washington, DC.
Noss, R.F., E.T. LaRoe III, and J.M. Scott. 1995. Endangered ecosystems of the United States: A preliminary assessment of loss and degradation. Biological Report 28, National Biological Service, Washington, DC.
O’Connor, R.J. 2002. The conceptual basis of species distribution modeling: Time for a paradigm shift? Pages 25-33 in J.M. Scott, P J. Heglund, M.L. Morrison, J.B. Haufler, M.G. Raphael, W.A. Wall, and F.B. Samson, editors. Predicting species occurrences: Issues of accuracy and scale. Island Press, Washington, DC.
Odum, E.D., and H.T. Odum. 1972. Natural areas as necessary components of man’s total environment. Transactions of the North American Wildlife and Natural Resources Conference 39:178-189.
Ride, W.L.D. 1975. Towards an integrated system: A study of selection and acquisition of national parks and nature reserves in Western Australia. In F. Fenner, editor. A national system of ecological reserves in Australia. Australian Academy of Science, Canberra.
Root, T.L., J.T. Price, K.R. Hall, S.H. Schneider, C. Rosenzweig, and J.A. Pounds. 2003. Fingerprints of global warming on wild animals and plants. Nature 421:57-60.
Schlaepfer, M.A., M.C. Runge, and P.W. Sherman. 2002. Ecological and evolutionary traps. TREE 17:474-480.
Scott, J.M., B. Csuti, J.D. Jacobi, and J.E. Estes. 1987. Species richness: A geographic approach to protecting future biological diversity. BioScience 37:782-788.
Scott, J.M., B. Csuti, K. Smith, J.E. Estes, and S. Caicco. 1988. Beyond endangered species: An integrated conservation strategy for the preservation of biological diversity. Endangered Species Update 5:43-48.
Scott, J.M., F. Davis, B. Csuti, R. Noss, B. Butterfield, C. Groves, H. Anderson, S. Caicco, F. D’Erchia, T.C. Edwards, Jr., J. Ulliman, and R.G. Wright. 1993. Gap analysis: A geographic approach to protection of biological diversity. Wildlife Monographs 123: 1-41.
Scott, J.M., C.R. Peterson, J.W. Karl, E. Strand, L.K. Svancara, and N.M. Wright. 2002a. A Gap Analysis of Idaho: Final Report. Idaho Cooperative Fish and Wildlife Research Unit, Moscow, Idaho.
Scott, J. M., P. J. Heglund, M. L. Morrison, J. B. Haufler, M. G. Raphael, W. A. Wall, and F. B. Samson. 2002b. Predicting species occurrences: Issues of accuracy and scale. Island Press, Washington, DC.
Shaffer, M.L., and B.A. Stein. 2000. Safeguarding our precious heritage. Pages 301-321 in B.A. Stein, L.S. Kutner, and J.S. Adams, editors. Precious heritage: The status of biodiversity in the United States. The Nature Conservancy, Oxford University Press, New York.
Soulé, M.E., and M.A. Sanjayan. 1998. Conservation targets: Do they help? Science 279:2060-2061.
Specht, R.L., E.M. Roe, and V.H. Boughlon. 1974. Conservation of major plant communities in Australia and Papua New Guinea. Australian Journal of Botany Supplement No. 7.
Van Horne, B. 1983. Density as a misleading indicator of habitat quality. Journal of Wildlife Management 47:893-901.
Wiens, J.A. 2002. Predicting species occurrences: Progress, problems, and prospects. Pages 739-750 in J.M. Scott, P J. Heglund, M.L. Morrison, J.B. Haufler, M.G. Raphael, W.A. Wall, and F.B. Samson, editors. Predicting species occurrences: Issues of accuracy and scale. Island Press, Washington, DC.
Return to Table of Contents