Hexagon Geospatial
MENU

Spatial Modeler Tutorials

Learn more about our products, find answers, get the latest updates, and connect with other Hexagon Geospatial product users, or get support from our professional service team.
Showing results for 
Search instead for 
Do you mean 

Landsat 8 Automated Cloud Mask

by Technical Evangelist ‎08-03-2017 07:27 AM - edited ‎08-03-2017 09:08 AM (7,227 Views)

Download model

 

Description:

 

Landsat 8 image on the left, results of running this model overlaid in the right view
L8_Cloud_Mask_REsults.PNG

 

 

Satellite imagery suffers from the influences of weather, especially the presence of clouds. The clouds not only obscure the features below them but also cast shadows that attenuate the light reflected back from the features falling in those  shadows. For many sensors, such as Landsat, a basic cloud mask is often provided with the data. But sometimes this can be inaccurate or lacking entirely. An automated way to map the locations of clouds and, optionally, areas of shadowing/shading, is frequently helpful in excluding these locations from further processing, or to identify the locations which need "back filling" using older data of the same location.

 

This article looks at one such research area. It is not offered as a finished, fool-proof approach, but as a topic for further investigation and fine tuning. Feedback is, as always, encouraged and welcomed.

 

 

Landsat8_Cloud_Mask_v16_1_4.gmdx
L8_Cloud_Mask.PNG

 

The basic premise is based on the paper "Generation of Cloud-free Imagery Using Landsat-8" by Byeonghee Kim, et al . The premise is that clouds generally reflect strongly in certain bands (such as the Cirrus and Coastal Blue bands) where as they are also cold and therefore have low returns in the thermal band(s). Consequently you should be able to threshold the bands to help identify cloud locations. But the traditional problem with this is that the threshold has to be manually chosen, or is fixed based on certain assumptions of pre-processing, making it sometimes difficult to apply scene to scene. For example, see the article entitled "Simple Cloud Mask for Landsat Imagery" for an example of fixed thresholds dependant on data that has been corrected to Surface Reflectance. Instead Kim et al recommend using a clustering technique to automatically identify the threshold points. In this model we have used ISODATA to perform the automated thresholding. 

 

The paper uses a similar assumption to attempt to identify shadows cast by the clouds. The authors suggested first identifying the cloud location and then searching (buffering) to a distance of 200 pixels (which has been provided as a user-definable variable in this model) and looking for a cluster of dark pixels in the lower SWIR (Band 6). Note: The paper actually states "band 6 (NIR)" which is contradictory. Observation of several test images seemed to indicate that Band 6 (the lower SWIR band) gave both a low return in shadows and a high return in other areas, and so was used. 

 

Based on these automated thresholds, 3 sets of criteria are evaluated and combined:

 

Type of Mask
Threshold Conditions
Thick Cloud Band 9 > threshold B9 (i.e. bright) & Band 10 < threshold B10 (i.e. cold/dark)
Thin Cloud

Band 1 > threshold B1 (i.e. bright) & Band 9 < threshold B9 (i.e. dark) & Band 10 < threshold B10 (i.e. cold/dark)

Shadow Band 6 < Lowest threshold B6 (i.e. dark)

 

Using these rules produced decent results. However analysis of several test images highlighted at least two major concerns which needed addressing:

 

  1. In images where snow or ice occurs, those pixels are also cold (low in Band 10) and highly reflective in the Cirrus band (Band 9). Snow and Ice pixels therefore fall the side of the thresholds which identifies then as Thick Cloud. Consequently a Snow/Ice Index (using Bands 6 and 3) was used to exclude easily identifiable Snow/Ice pixels from the cloud analysis.

  2. Water pixels are easily confused with the Shadow criteria. So a basic water index (using Bands 5 and 4) was applied to exclude easily identifiable Water pixels from the cloud analysis

This appeared to increase the accuracy of the Cloud and Shadow identification. However cloud locations were still generally under-classified (errors of omission). So an option to Dilate the cloud locations is provided and should generally be used.

 

Shadows also continued to be over classified (errors of commission). Consequently an option is provided to sieve (remove) shadows that fall below a user-provided size threshold.

 

Data Requirements

 

The model is designed for Landsat 8 data with both the Cirrus (Band 9) and Thermal 1 (Band 10) bands being available for processing. It also expects the Coastal Blue (Band 1) and bands 3, 4, 5, and 6 (Green, Red, NIR and SWIR1 respectively). The model could be modified to work on other sensors which provide those wavelengths.

 

Assumptions

 

  1. This model is built to expect Landsat 8 data which has been downloaded as individual TIFF files (one band per file) with a standard naming convention where the band number is encoded into the end of the root name. For example "LC80190362017118LGN00_B10.TIF" indicates the file containing Band 10 (via the *_B10.tif suffix)  and uses a common root for all other band files (e.g "LC80190362017118LGN00_B9.TIF" indicates band 9). The user only needs to pick one of the input files (Band 10)  and the model uses the pattern derived from this file's name to automatically identify the other bands it requires. If your data has had all bands pre-stacked into a single image file, or uses a different naming convention, you will need to modify the model accordingly.

  2. When running the model you may see the following error occur:

    "Error - Cannot create true color table because of no image statistics"

    This message can be OK'd and ignored. It refers to the attempt to set colors on the temporary ISODATA results which can't be done if there are no statistics associated with the TIFF images. Since these are temporary files anyway, the colors of the classes do not matter. 

  3. This approach does assume that clouds are present and do not cover the entire scene. It is not going to produce good results in a cloud free scene or one entirely covered by cloud. I didn't have the opportunity to test to see if there was a threshold, but I'd estimate that there should be at least 10% and less than 90% cloud cover in the scene to achieve good results.

    This limitation stems from the approach for automatically deriving threshold points for the thermal and cirrus bands - the assumption is that these bands can be broken into cloud and not cloud. Obviously this assumption breaks down if there are no clouds, or all clouds. In this instance the algorithm is still going to split the bands via a derived threshold, but it will be based on some other land cover differences, rather than cloud / not cloud.

 

Results

 

Based on the sample images tested the model as it currently stands provided good, but by no means perfect, results. Further research is required to fine-tune the approach. Feedback from the community is encouraged.

 

However if the purpose is to identify pixels contaminated with clouds (and their effects, such as shadows) for the purpose of replacing those pixels with data from other sources (such as older, but cloud free imagery), the technique produces usable results. 

 

 

Input parameters:

 

Band 10 of Landsat 8: From a directory containing all (relevant) TIFF band files of a Landsat 8 scene select the file representing Band 10 (Thermal Infrared (TIRS) 1). Based on the naming pattern of this file, the other relevant band files will be automatically identified and used. For example: "LC80190362017118LGN00_B10.TIF"

Dilate Clouds?: The algorithm tends to under-detect the edges of clouds as they thin. Consequently a boolean (true/false) option is provided. If set to True (1), a filter is applied to the basic cloud locations to grow the initial detections a distance of 2 pixels.

Mask Shadows?: Optionally a second output class can be created for the shadows of the clouds. If the Shadow class is desired, set this option to True (1).

Maximum Shadow Offset Distance (in pixels): The distance away from the Cloud class to consider when looking for shadows, 200 pixels tends to be the maximum in most Landsat 8 scenes. If you can visually review the scene prior to running the model setting an appropriate, shorter distance will increase both the speed and accuracy of the Shadow class. 

Shadow Sieve Size (pixels): Over classification of shadows can be minimised by sieving out small detections (which therefore probably aren't associated with clouds). Enter the size (in pixels) below which clumps of detected shadow pixels will be discarded.

Output Mask Filename: The name of the output image file containing the Cloud class (DN 1) and the optional Shadow class (DN 2)

 

L8_Cloud_Mask_Run.PNG

   

 

 

 
Comments
by Technical Evangelist
on ‎08-03-2017 08:52 AM

This looks like an interesting option for Sentinel-2 data that could be implemented using Spatial Modeler. But it's another of those techniques which relies on hardwired thresholds, rather than scene-derived ones.

 

https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-2a/algorithm

 

by Technical Evangelist
on ‎09-06-2017 10:47 AM

I've never liked the fact that this technique looks for cloud shadows in a radial buffer zone all the way round the detected clouds. If you know the Sun's Azimuth at the time of image acquisition (which you should be able to find in the metadata flies which come with Landsat 8 data), then you should be able to constrain the search only in the direction opposite that angle and thereby stand a much better chance of correctly identifying cloud shadows.

 

Perhaps some enterprising Spatial Model builder could come up with an angle constrained buffer model?

 

Cheers

 

Ian

 

 

by Tharu
on ‎12-06-2017 01:09 AM

I downloaded landsat 8 Automated cloud mask model for landsat 8 .But it couldnt open from Erdas imagine software .Are there any other models for cloud mask using Erdas Imagine?

 

 

by Technical Evangelist
on ‎12-06-2017 05:16 AM

Hello Tharu,

 

Are you using ERDAS IMAGINE 2016 v16.1? The model uses Operators which were introduced in that version.

 

This technique should be possible in ERDAS IMAGINE 2015 v15.1 or higher:

 

http://community.hexagongeospatial.com/t5/Spatial-Recipes/Simple-Cloud-Mask-for-Landsat-Imagery/ta-p...

 

Cheers

 

Ian

by Tharu
on ‎12-06-2017 06:07 AM

Hello Ian,

 

I'm using Erdas imagine 2014 version.Are  there any other models for erdas imagine 2014  version?

Courses
Contributors