home

<- Previous | First | Next ->

Expert Review for Land Cover: Integrating Information from Specific Comments and Evaluating the Results

Geoffrey M. Henebry¹, James W. Merchant¹, Justin W. Fischer¹, and Douglas Garrison²

¹Center for Advanced Land Management Information Technologies, University of Nebraska, Lincoln

²USDA Natural Resources Conservation Service, Lincoln, Nebraska

Introduction

The Landsat TM data used to generate the base land cover map for Nebraska were acquired nearly a decade ago in the early 1990s. This situation poses a challenge for evaluation of the accuracy of the land cover product. Land cover changes subsequent to image acquisition result in temporal decorrelation that is interpreted as classification error in a formal accuracy assessment, if the ground data for the accuracy assessment are not collected concurrently with image acquisition. As we did not have concurrent ground data, we sought a procedure to not only evaluate the accuracy of our draft land cover map but also to increase its relevance to a potential user community.

To solicit expert assessment of the draft land cover map, the Nebraska Gap Analysis Project and the Nebraska State Office of the Natural Resources Conservation Service (NRCS) sent out relevant county-level maps to district conservationists at NRCS offices during late 1999 and early 2000. The district conservationists coordinated review of the hard-copy maps, utilizing staff from 81 NRCS offices statewide. Local experts reviewed the draft maps and identified misclassifications by annotating the hard-copy map with a series of general and specific comments. As of December 15, 2000, 75 of 93 county maps had been returned, a response rate of over 80%.

Methods

Of the 75 maps returned, 65 were annotated with specific comments and 10 indicated no change. General and specific comments were recorded from each map. Specific comments, defined as comments noting misclassification of particular groups of pixels, were then tabulated into a special confusion matrix reporting only misclassification errors; thus, all elements of the matrix were located off the principal diagonal (Table 1).

Table 1: Special confusion matrix based on original draft land cover map. Matrix elements corresponds to total numbers of comments made by reviewers. No elements occur on the diagonal.

Misclassifications identified on the draft land cover map were then compared against a subsequent version of the map that incorporated additional sources of information, viz., a thematically simplified subset of NWI polygons, fallow agricultural lands generated by an independent project, and urban land cover classes pulled from the USGS National Land Cover Data.  A second special confusion matrix was generated to determine whether misclassifications had been corrected by incorporation of multiple data sources (Table 2).  Remaining misclassifications deemed significant were manually recoded.  The decision to recode pixels into the “Agricultural Fields” class was made on a case-by-case basis.   Adjustments were made by comparing the latest draft with the National Land Cover Data product, relevant DOQQs, and a map of Nebraska’s native vegetation.  A third special confusion matrix was then generated (Table 3).

Table 2: Special confusion matrix based on subsequent draft land cover map incorporating additional data sources. Diagonal elements correspond to corrected comments. Off-diagonal elements correspond to comments that were not corrected with additional data.

Results

The inclusion of additional data sources took care of 302 (31%) of the specific comments.  Manual editing of the significant misclassifications took care of 241 (35%) remaining comments.  The two-stage revision eliminated 543 (55%) of the specific comments made by the expert reviewers.  Of the remaining 446 misclassifications, 372 (83%) were identified by the reviewers as “Agricultural Fields”.  The classes contributing to most of this remaining error were “Barren/Sand/Outcrop” (144 or 39%) and “Lowland Tallgrass Prairie” (124 or 33%).  The second-most confused class was “Little Bluestem-Grama Mixedgrass Prairie” at 53 (12%) remaining comments.  Two woodland classes contributed to most of the error remaining after revisions: “Deciduous Forest/Woodland” (26 or 49%) and “Evergreen Forest/Woodland” (21 or 40%).

Table 3. Special confusion matrix following manual recoding of significant misclassifications. Diagonal elements correspond to corrected comments. Off-diagonal elements correspond to comments that were not corrected with additional data.

Discussion

Inclusion of additional data significantly improved the land cover map. Further revision by manual recoding yielded a reduction of misclassification by 55% from the original draft map. Most of the misclassifications identified by the field experts were agricultural fields. Increasing agricultural activity across Nebraska has resulted in new lands going into production. Most of the remaining misclassifications involved grassland-woodland confusion. It is important to keep in mind that the comments referred to specific groups of pixels that varied widely in size; thus conventional map error analyses and metrics were inappropriate. While the results of the expert review did not yield a formal pixel-centric accuracy assessment, it did yield a land cover map more in accordance with the understanding of local experts. This classification post-processing has better prepared us to conduct a conventional accuracy assessment and interpret the results in the light of land cover changes subsequent to image acquisition.