scholarly journals Thematic Accuracy Quality Control by Means of a Set of Multinomials

2019 ◽  
Vol 9 (20) ◽  
pp. 4240 ◽  
Author(s):  
Francisco J. Ariza-López ◽  
José Rodríguez-Avi ◽  
María V. Alba-Fernández ◽  
José L. García-Balboa
Keyword(s):  
Quality Control ◽  
Hypothesis Test ◽  
Accuracy Assessment ◽  
Confusion Matrix ◽  
Multinomial Distribution ◽  
Remotely Sensed Data ◽  
Error Matrix ◽  
Quality Specifications ◽  
Control Column ◽  
Thematic Accuracy

The error matrix has been adopted as both the “de facto” and the “de jure” standard way to report on the thematic accuracy assessment of any remotely sensed data product. This perspective assumes that the error matrix can be considered as a set of values following a unique multinomial distribution. However, the assumption of the underlying statistical model falls down when true reference data are available for quality control. To overcome this problem, a new method for thematic accuracy quality control is proposed, which uses a multinomial approach for each category and is called QCCS (quality control column set). The main advantage is that it allows us to state a set of quality specifications for each class and to test if they are fulfilled. These requirements can be related to the percentage of correctness in the classification for a particular class but also to the percentage of possible misclassifications or confusions between classes. In order to test whether such specifications are achieved or not, an exact multinomial test is proposed for each category. Furthermore, if a global hypothesis test is desired, the Bonferroni correction is proposed. All these new approaches allow a more flexible way of understanding and testing thematic accuracy quality control compared with the classical methods based on the confusion matrix. For a better understanding, a practical example of an application is included for classification with four categories.

Accuracy assessment and validation of remotely sensed and other spatial information

2001 ◽  
Vol 10 (4) ◽  
pp. 321 ◽  
Author(s):  
Russell G. Congalton
Keyword(s):  
Data Quality ◽  
Fire Management ◽  
Spatial Information ◽  
Accuracy Assessment ◽  
Remotely Sensed ◽  
Remotely Sensed Data ◽  
Error Matrix ◽  
Remotely Sensed Imagery ◽  
Ecological Principles ◽  
In Fire

This paper was presented at the conference ‘Integrating spatial technologies and ecological principles for a new age in fire management’, Boise, Idaho, USA, June 1999 Today, validation or accuracy assessment is an integral component of most mapping projects incorporating remotely sensed data. Other spatial information may not be so stringently evaluated, but at least requires meta-data that documents how the information was generated. This emphasis on data quality was not always the case. In the 1970s only a few brave scientists and researchers dared ask the question, ‘How good is this map derived from Landsat MSS imagery?’ In the 1980s, the use of the error matrix became a common tool for representing the accuracy of individual map categories. By the 1990s, most maps derived from remotely sensed imagery were required to meet some minimum accuracy standard. A similar progression can be outlined for other spatial information. However, this progression is about 5 years behind the validation of remotely sensed data. This paper presents a series of steps moving towards better assessment and validation of spatial information and asks the reader to evaluate where they are in this series and to move forward.

scholarly journals The Effect of Environmental Conditions on the Quality of UAS Orthophoto-Maps in the Coastal Environment

2021 ◽  
Vol 10 (1) ◽  
pp. 18
Author(s):  
Michaela Doukari ◽  
Stelios Katsanevakis ◽  
Nikolaos Soulakellis ◽  
Konstantinos Topouzelis
Keyword(s):  
Environmental Conditions ◽  
Accuracy Assessment ◽  
Marine Conservation ◽  
Unmanned Aerial Systems ◽  
Remotely Sensed Data ◽  
Rough Sea Surface ◽  
Thematic Accuracy ◽  
Aerial Systems ◽  
Remote Sensing Methods

Marine conservation and management require detailed and accurate habitat mapping, which is usually produced by collecting data using remote sensing methods. In recent years, unmanned aerial systems (UAS) are used for marine data acquisition, as they provide detailed and reliable information through very high-resolution orthophoto-maps. However, as for all remotely sensed data, it is important to study and understand the accuracy and reliability of the produced maps. In this study, the effect of different environmental conditions on the quality of UAS orthophoto-maps was examined through a positional and thematic accuracy assessment. Selected objects on the orthophoto-maps were also assessed as to their position, shape, and extent. The accuracy assessment results showed significant errors in the different maps and objects. The accuracy of the classified images varied between 2.1% and 27%. Seagrasses were under-classified, while the mixed substrate class was overclassified when environmental conditions were not optimal. The highest misclassifications were caused due to sunglint presence in combination with a rough sea-surface. A change detection workflow resulted in detecting misclassifications of up to 45%, on orthophoto-maps that had been generated under non-optimal environmental conditions. The results confirmed the importance of optimal conditions for the acquisition of reliable marine information using UAS.

The Research of Production Process Statistical Steady-State Based on the Hypothesis Test

2011 ◽  
Vol 314-316 ◽  
pp. 2433-2438
Author(s):  
Wei Zhi Wang
Keyword(s):  
Quality Control ◽  
Steady State ◽  
Production Process ◽  
Quantitative Method ◽  
Large Scale ◽  
Hypothesis Test ◽  
Influence Factors ◽  
Normal Operation ◽  
Scale Production ◽  
Large Scale Production

By only applying a after the event exam in the quality control of the batch production is not enough to meet the needs of modern large-scale production. To a certain extent, modern quality control is a dynamic process of the steady-state judge and adjustment. A simple and reliable steady-state judge rule and method is the premise to guarantee the normal operation. This paper provides a quantitative method to evaluate production process steady-state by analyzing influence factors based on mathematical statistics. The method is both suitable for simple production process and complex production process with sub-processes.

scholarly journals A New Approach to the Quality Control of Slope and Aspect Classes Derived from Digital Elevation Models

2021 ◽  
Vol 13 (11) ◽  
pp. 2069
Author(s):  
M. V. Alba-Fernández ◽  
F. J. Ariza-López ◽  
M. D. Jiménez-Gamero
Keyword(s):  
Quality Control ◽  
Hypothesis Test ◽  
Simulated Data ◽  
Slope Aspect ◽  
New Approach ◽  
Reference Set ◽  
Digital Elevation ◽  
Using Data ◽  
Elevation Model ◽  
Degree Of Similarity

The usefulness of the parameters (e.g., slope, aspect) derived from a Digital Elevation Model (DEM) is limited by its accuracy. In this paper, a thematic-like quality control (class-based) of aspect and slope classes is proposed. A product can be compared against a reference dataset, which provides the quality requirements to be achieved, by comparing the product proportions of each class with those of the reference set. If a distance between the product proportions and the reference proportions is smaller than a small enough positive tolerance, which is fixed by the user, it will be considered that the degree of similarity between the product and the reference set is acceptable, and hence that its quality meets the requirements. A formal statistical procedure, based on a hypothesis test, is developed and its performance is analyzed using simulated data. It uses the Hellinger distance between the proportions. The application to the slope and aspect is illustrated using data derived from a 2×2 m DEM (reference) and 5×5 m DEM in Allo (province of Navarra, Spain).

scholarly journals Consistency Analysis and Accuracy Assessment of Three Global 30-m Land-Cover Products over the European Union using the LUCAS Dataset

2020 ◽  
Vol 12 (21) ◽  
pp. 3479
Author(s):  
Yuan Gao ◽  
Liangyun Liu ◽  
Xiao Zhang ◽  
Xidong Chen ◽  
Jun Mi ◽  
...  
Keyword(s):  
European Union ◽  
Land Cover ◽  
Hydrological Cycle ◽  
Accuracy Assessment ◽  
Confusion Matrix ◽  
The European Union ◽  
Mountain Areas ◽  
Land Cover Types ◽  
Spatial Consistency ◽  
The Eu

Land-cover plays an important role in the Earth’s energy balance, the hydrological cycle, and the carbon cycle. Therefore, it is important to evaluate the current global land-cover (GLC) products and to understand the differences between these products so that they can be used effectively in different applications. In this study, three 30-m GLC products, namely GlobeLand30-2010, GLC_FCS30-2015, and FROM_GLC30-2015, were evaluated in terms of areal consistency and spatial consistency using the Land Use/Cover Area frame statistical Survey (LUCAS) reference dataset over the European Union (EU). Given the limitations of the traditional confusion matrix used in accuracy assessment, we adjusted the confusion matrices from sample counts by accounting for the class proportions of the map and reported the standard errors of the descriptive accuracy measures in the accuracy assessment. The results revealed the following. (1) The overall accuracy of the GlobeLand30-2010 product was the highest at 88.90 ± 0.68%; this was followed by GLC_FCS30-2015 (84.33 ± 0.80%) and FROM_GLC2015 (65.31 ± 1.0%). (2) The consistency between the GLC_FCS30-2015 and GlobeLand30-2010 is higher than the consistency between other products, with an area correlation coefficient of 0.930 and a proportion of consistent pixels of 52.41%, respectively. (3) Across the area of the EU, the dominant land-cover types such as forest and cropland are the most consistent across the three products, whereas the spatial consistency for bare land, grassland, shrubland, and wetland is relatively low. (4) The proportion of pixels for which the consistency is low accounts for less than 16.17% of pixels, whereas the proportion of pixels for which the consistency is high accounts for about 39.12%. The disagreement between these products primarily occurs in transitional zones with mixed land cover types or in mountain areas. Overall, the GlobeLand30 and GLC-FCS30 products were found to be the most consistent and to have good classification accuracy in the EU, with the disagreement between the three 30-m GLC products mainly occurring in heterogeneous regions.

scholarly journals Trends in Remote Sensing Accuracy Assessment Approaches in the Context of Natural Resources

2019 ◽  
Vol 11 (19) ◽  
pp. 2305 ◽  
Author(s):  
Lucia Morales-Barquero ◽  
Mitchell Lyons ◽  
Stuart Phinn ◽  
Chris Roelfsema
Keyword(s):  
Remote Sensing ◽  
Natural Resources ◽  
Sampling Design ◽  
Accuracy Assessment ◽  
Assessment Practices ◽  
Screening Process ◽  
Error Matrix ◽  
Remote Sensing Classification ◽  
Considerable Work ◽  
Land Cover Maps

The utility of land cover maps for natural resources management relies on knowing the uncertainty associated with each map. The continuous advances typical of remote sensing, including the increasing availability of higher spatial and temporal resolution satellite data and data analysis capabilities, have created both opportunities and challenges for improving the application of accuracy assessment. There are well established accuracy assessment methods, but their underlying assumptions have not changed much in the last couple decades. Consequently, revisiting how map error and accuracy have been performed and reported over the last two decades is timely, to highlight areas where there is scope for better utilization of emerging opportunities. We conducted a quantitative literature review on accuracy assessment practices for mapping via remote sensing classification methods, in both terrestrial and marine environments. We performed a structured search for land and benthic cover mapping, limiting our search to journals within the remote sensing field, and papers published between 1998–2017. After an initial screening process, we assembled a database of 282 papers, and extracted and standardized information on various components of their reported accuracy assessments. We discovered that only 56% of the papers explicitly included an error matrix, and a very limited number (14%) reported overall accuracy with confidence intervals. The use of kappa continues to be standard practice, being reported in 50.4% of the literature published on or after 2012. Reference datasets used for validation were collected using a probability sampling design in 54% of the papers. For approximately 11% of the studies, the sampling design used could not be determined. No association was found between classification complexity (i.e. number of classes) and measured accuracy, independent from the size of the study area. Overall, only 32% of papers included an accuracy assessment that could be considered reproducible; that is, they included a probability-based sampling scheme to collect the reference dataset, a complete error matrix, and provided sufficient characterization of the reference datasets and sampling unit. Our findings indicate that considerable work remains to identify and adopt more statistically rigorous accuracy assessment practices to achieve transparent and comparable land and benthic cover maps.

scholarly journals Distribution Statistics Preserving Post-Processing Method With Plot Level Uncertainty Analysis for Remotely Sensed Data-Based Forest Inventory Predictions

2018 ◽  
Vol 10 (11) ◽  
pp. 1677
Author(s):  
Virpi Junttila ◽  
Tuomo Kauranne
Keyword(s):  
Forest Inventory ◽  
Laser Scanning ◽  
Linear Models ◽  
Accuracy Assessment ◽  
Remotely Sensed ◽  
Processing Method ◽  
Post Processing ◽  
Remotely Sensed Data ◽  
Training Set ◽  
Study Sites

Remotely sensed data-based models used in operational forest inventory usually give precise and accurate predictions on average, but they often suffer from systematic under- or over-estimation of extreme attribute values resulting in too narrow or skewed attribute distributions. We use a post-processing method based on the statistics of a proper, representative training set to correct the predictions and their probability intervals, attaining corrected predictions that reproduce the statistics of the whole population. Performance of the method is validated with three forest attributes from seven study sites in Finland with training set sizes from 50 to over 400 field plots. The results are compared to those of the uncorrected predictions given by linear models using airborne laser scanning data. The post-processing method improves the accuracy assessment linear fit between the predictions and the reference set by 35.4–51.8% and the distribution fit by 44.5–95.0%. The prediction root mean square error declines on the average by 6.3%. The systematic under- and over-estimation are reduced consistently with all training set sizes. The level of uncertainty is maintained well as the probability intervals cover the real uncertainty while keeping the average probability interval width similar to the one in uncorrected predictions.

scholarly journals GHS-POP Accuracy Assessment: Poland and Portugal Case Study

2020 ◽  
Vol 12 (7) ◽  
pp. 1105
Author(s):  
Beata Calka ◽  
Elzbieta Bielecka
Keyword(s):  
Population Distribution ◽  
Accuracy Assessment ◽  
Local Level ◽  
National Level ◽  
Error Rates ◽  
Research Centre ◽  
Percentage Error ◽  
Remotely Sensed Data ◽  
Joint Research ◽  
Population Number

The Global Human Settlement Population Grid (GHS-POP) the latest released global gridded population dataset based on remotely sensed data and developed by the EU Joint Research Centre, depicts the distribution and density of the total population as the number of people per grid cell. This study aims to assess the GHS-POP data accuracy based on root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) and the correlation coefficient. The study was conducted for Poland and Portugal, countries characterized by different population distribution as well as two spatial resolutions of 250 m and 1 km on the GHS-POP. The main findings show that as the size of administrative zones decreases (from NUTS (Nomenclature of Territorial Units for Statistics) to LAU (local administrative unit)) and the size of the GHS-POP increases, the difference between the population counts reported by the European Statistical Office and estimated by the GHS-POP algorithm becomes larger. At the national level, MAPE ranges from 1.8% to 4.5% for the 250 m and 1 km resolutions of GHS-POP data in Portugal and 1.5% to 1.6%, respectively in Poland. At the local level, however, the error rates range from 4.5% to 5.8% in Poland, for 250 m and 1 km, and 5.7% to 11.6% in Portugal, respectively. Moreover, the results show that for densely populated regions the GHS-POP underestimates the population number, while for thinly populated regions it overestimates. The conclusions of this study are expected to serve as a quality reference for potential users and producers of population density datasets.

Sign in / Sign up

Export Citation Format

Share Document