Showing results for 
Search instead for 
Do you mean 

Imperfect Ground Reference – Binary Change Accuracy Assessment Report

by on ‎01-13-2017 06:15 AM - edited on ‎02-21-2020 06:56 AM by Community Manager (987 Views)

Download Model



This model was based on the theory within the paper entitled “Assessing the accuracy of land cover change with imperfect ground reference data” by Giles M. Foody (2010). The core concept is, without perfect ground reference, you can infer relative accuracy of binary-change detection results by relating statistics to a multi-date binary image as reference. This model will prompt input of two binary (thematic) images and output a CSV Accuracy Assessment file. The CSV will automatically be created in the same directory as the Binary Map Class Image with an appended name _errormatrix.csv


In order to run this model, the relevant binary thematic images are assumed already created with the classes 1 = no change and 2 = change. For demonstration purposes, small theoretical images are provided to demonstrate the statistics used.


Input Parameters:




Input ‘Binary Map Class Image’ Assumes thematic (binary) image of class 1 = no change and class 2 = change.

Input ‘Multi-date Binary Ground Reference’ Image Binary image created from multiple dates to be used as pseudo-ground reference for the binary change detection error matrix. Assumes thematic (binary) image of class 1 = no change and class 2 = change.


Model Output:

CSV Report written to the same directory as the Input Binary Map Class Image with the appended name _errormatrix.csv

The CSV Report has the following structure:



The first table is the total pixels and the appropriate values for change/no change. The second table is the percentage values for the stated statistic.


Spatial Model Screenshot:



Expression One; creates a 4 class image using the two binary images as input. The image is recoded to values 1 to 4 to represent the values A – D in the table referenced by Foody (2010), Figure One:



With the Ground Reference as Input 1 ($I1) and the Remote Sensing Input as Input 2 ($I2), the following expression was used: Conditional { ($I1 == 2 AND $I2 == 2) 1, ($I1 == 1 AND $I2 ==2) 2, ($I1 == 2 AND $I2 == 1) 3, ($I1 == 1 AND $I2 == 1) 4}

This expression recodes the two binary inputs so all four possible values are present 1-4 representing A-D in the table above. Next each value is pulled out separately ignoring all other values by recoding to 0 then ignoring these 0 values in the statistics operators.

A: IF $I1 == 1 THEN 1 ELSE 0

B: IF $I1 == 2 THEN 1 ELSE 0

C: IF $I1 == 3 THEN 1 ELSE 0

D: IF $I1 == 4 THEN 1 ELSE 0


The GIS Statistics operators within the model were used to ‘sum’ the number of pixels in each. These were renamed A to D respectively.


Next, the values E > H and N were calculated using the following arithmetic;

e = a + c

f = b + d

g = a + b

h = c + d

n = e + f


Once all values A > H and N were derived, the expressions for sensitivity, specificity, predicted positive and negative and prevalence was possible. Following on from the Binary Confusion Matrix, the following expressions were derived and used from the research article by Foody (2010).

Sensitivity          a / (a + c)

Specificity          d / (b + d)

Predicted_pos    a / (a + b)

Predicted_neg    d / (c + d)

Prevalence         (a + c) / (a + b + c + d)


The next step in the model was to collate all 14 calculated values into a Dynamic Matrix List. A matrix list was chosen as this can easily be converted to string format, split and indexed ready for inputs into the final report.