Hexagon Geospatial
MENU

Spatial Modeler Tutorials

Learn more about our products, find answers, get the latest updates, and connect with other Hexagon Geospatial product users, or get support from our professional service team.
Showing results for 
Search instead for 
Do you mean 

Using the Python Script operator to ingest sensor metadata (2016)

by Technical Evangelist ‎08-10-2016 09:29 AM - edited ‎08-15-2016 05:30 AM (1,893 Views)

Download models

 

Description:

 

Models frequently need parameters that can be found in an image's metadata file. This script shows an example of how to use a Python Script to parse a Landsat *mtl.txt file into a Dictionary so it can be used to extract values that could be used elsewhere in a Model.

 

Please refer to the earlier "Landsat 8 Top of Atmosphere Reflectance Conversion" article showing how to calculate the ToA reflectance values for Landsat 8 imagery. That article mentions one limitation of the approach described, which is that the correction values must either be assumed, or need to be manually read from the input image's metadata header file and entered into the model each time.

 

An alternative approach is to provide a means to parse the metadata file, extract the necessary parameters and automatically ingest those within the model. With the introduction of ERDAS IMAGINE 2016 we have a simple way of doing this by using the Python Script Operator. A Python script was written which is capable of parsing the metadata file and converting it into a Dictionary. This Dictionary can then be interrogated by the Dictionary Item operator to extract the necessary parameters and pass them into the remainder of the model. The screenshot below shows the basic operation of extracting information from a Landsat 8 *_mtl.txt file.

 

 ParseMetadata-v16-0-0.gmdx
 Parse_Metadata_Model.PNG

 

The section below shows the contents of the Python Script operator.

 

 Content of the GetLandsatMetadata Python Script operator
 


def GetLandsatMetadata(filename):
    s = ReadFile(filename)
    (items, junk) = Parse(s)
    return items

def ReadFile(filename):
    f = open(filename, "r", encoding="utf-8")
    s = f.read()
    f.close()
    return s

def ParseValue(str):
    try:
        try:
            return int(str)
        except ValueError:
            return float(str)
    except ValueError:
        if str[0] == str[-1] == '"':
            str = str[1:-1]
        return str

def GetLine(s, newline='\n'):
    next = s.find(newline, 0, None)
    line = s[0:next]
    s = s[next+1:]
    return (line, s)


def Parse(contents, newline='\n'):
    result = {}
    separator = ' = '
    while len(contents) > 0:
        (line, contents) = GetLine(contents)
        line = line.strip()
        if line == 'END':
            return (result, contents)
        if len(line) == 0:
            continue
        parts = line.split(separator)
        if len(parts) < 2:
            continue
        if parts[0] == 'GROUP':
            subgroup = parts[1]
            (result[subgroup], contents) = Parse(contents, newline)
        elif parts[0] == 'END_GROUP':
            return (result, contents)
        else:
            name = parts[0]
            value = line[len(name) + len(separator):].strip()
            value = ParseValue(value)
            result[name] = value
    return (result, contents)

 

The ParseMetadata model can then be combined with the old Landsat8 Top of Atmosphere Correction model to create a new model which automatically extracts the information necessary to correct the input imagery to Top of Atmosphere reflectance.

 

 L8_ToA_Reflectance_v16_0_1.gmdx
 L8_TOA_Reflectance_Python_2016.PNG

 

The operation of the model shown above to convert DN values to ToA Reflectance is the same as is described in the Landsat 8 Top of Atmosphere Reflectance Conversion article, so we'll not repeat it's full description here.

 

The main deviation from that older methodology is the inclusion of an option to rescale the Reflectance values from their normal floating point values to unsigned 16-bit integers (generally to conserve disk space). This is done by the industry-standard approach of multiplying the reflectance values by 10000 and truncating to the nearest integer. This is achieved using the If Else operator introduced with ERDAS IMAGINE 2016 - if the user opts for True (1) on the Scale to Integer prompt the data is pulled through the branch of the model which scales by 10,000 and outputs a 16-bit integer data set. If the user selects False (0) the data instead goes through a branch which leaves the reflectance values un-scaled and writes out a 64-bit Float data set instead.

 

Assumptions:

 

To execute this model you will need to have Python 3.4.x installed and configured for use with ERDAS IMAGINE. See here for instructions on how to install Python and also how to start using it within Spatial Modeler.

 

Input parameters:

 

L8 OLI MSI: Select an input image file containing the Multi-spectral (MSI) bands (1 to 7) from the Landsat 8 OLI sensor

Landsat 8 Metadata File: Select the *_MTL.txt file corresponding to the input image.

Scale to Integer: If set to true (1), the output ToA Reflectance data will be scaled up by a factor of 10,000 and truncated to a 16-bit integer to save disk space. Defaults to false (float output).

Reflectance Out: Name of the output image file containing the ToA Reflectance values.

 

 
 L8_TOA_Reflectance_Python_GUI_2016.PNG

   

Reference

 

http://landsat.usgs.gov/Landsat8_Using_Product.php

 

Courses
Contributors