Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure

A sample of methane plumes from point sources observed since 2019 by the U.S. Geostationary Operational Environmental Satellites (GOES) over North and South America.
Author

Siddharth Chaudhary

Published

November 15, 2024

Access this Notebook

You can launch this notebook in the US GHG Center JupyterHub by clicking the link below. If you are a new user, you should first sign up for the hub by filling out this request form and providing the required information.

Access the Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure notebook in the US GHG Center JupyterHub.

Table of Contents

Data Summary and Application

  • Spatial coverage: North and South America
  • Spatial resolution: 1 km x 1 km (nadir)
  • Temporal extent: April 7, 2019 - July 26, 2023
  • Temporal resolution: 5 minutes
  • Unit: Total column methane enhancement above background, moles of methane per square meter
  • Utility: Climate Research

For more, visit the Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure data overview page.

Approach

  1. Identify available dates and temporal frequency of observations for the given collection using the GHGC API /stac endpoint. The collection processed in this notebook is the gridded methane emissions data product.
  2. Pass the STAC item into the raster API /collections/{collection_id}/items/{item_id}/tilejson.jsonendpoint.
  3. Using folium.plugins.DualMap, we will visualize two tiles (side-by-side), allowing us to compare time points.

About the Data

Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure

A sample of methane plumes from point sources observed since 2019 by the U.S. Geostationary Operational Environmental Satellites (GOES) over North and South America.

For more information regarding this dataset, please visit the Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure data overview page.

Terminology

Navigating data via the GHGC API, you will encounter terminology that is different from browsing in a typical filesystem. We’ll define some terms here which are used throughout this notebook. - catalog: All datasets available at the /stac endpoint - collection: A specific dataset, e.g. Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure - item: One granule in the dataset, e.g. one file of GOES observations - asset: A variable available within the granule, e.g. CO₂ fluxes - STAC API: SpatioTemporal Asset Catalogs - Endpoint for fetching metadata about available datasets - Raster API: Endpoint for fetching data itself, for imagery and statistics

Install the Required Libraries

Required libraries are pre-installed on the GHG Center Hub. If you need to run this notebook elsewhere, please install them with this line in a code cell:

%pip install requests folium rasterstats pystac_client pandas matplotlib –quiet

# Import the following libraries
# For fetching from the Raster API
import requests
# For making maps
import folium
import folium.plugins
from folium import Map, TileLayer
# For talking to the STAC API
from pystac_client import Client
# For working with data
import pandas as pd
# For making time series
import matplotlib.pyplot as plt
# For formatting date/time data
import datetime
# Custom functions for working with GHGC data via the API
import ghgc_utils

Query the STAC API

STAC API Collection Names

Now, you must fetch the dataset from the STAC API by defining its associated STAC API collection ID as a variable. The collection ID, also known as the collection name, for the Geostationary Satellite Observations of Extreme and Transient Methane Emissions from Oil and Gas Infrastructure dataset is goes-ch4plume-v1.*

**You can find the collection name of any dataset on the GHGC data portal by navigating to the dataset landing page within the data catalog. The collection name is the last portion of the dataset landing page’s URL, and is also listed in the pop-up box after clicking “ACCESS DATA.”*

# Provide STAC and RASTER API endpoints
STAC_API_URL = "https://earth.gov/ghgcenter/api/stac"
RASTER_API_URL = "https://earth.gov/ghgcenter/api/raster"

# Please use the collection name similar to the one used in STAC collection.

# Name of the collection for gosat budget methane. 
collection_name = "goes-ch4plume-v1"
# Fetch the collection from the STAC API using the appropriate endpoint
# The 'pystac_client' library enables an HTTP request
catalog = Client.open(STAC_API_URL)
collection = catalog.get_collection(collection_name)

# Print the properties of the collection to the console
collection

Examining the contents of our collection under the temporal variable, we see that the data is available starting April 7, 2019 . By looking at the dashboard:time density, we observe that the data is available for multiple days.

# The search function lets you search for items within a specific date/time range
search = catalog.search(
    collections=collection_name,
    datetime=['2019-05-11T00:00:00Z','2019-05-13T00:00:00Z']
)
items = search.item_collection()
# Print how many items we found in our search
print(f"# items found: {len(items)}")
# items found: 52
for item in items:
    print(item.properties["datetime"])
2019-05-12T19:41:00+00:00
2019-05-12T19:36:00+00:00
2019-05-12T19:31:00+00:00
2019-05-12T19:26:00+00:00
2019-05-12T19:21:00+00:00
2019-05-12T19:16:00+00:00
2019-05-12T19:11:00+00:00
2019-05-12T19:06:00+00:00
2019-05-12T19:01:00+00:00
2019-05-12T18:56:00+00:00
2019-05-12T18:51:00+00:00
2019-05-12T18:46:00+00:00
2019-05-12T18:41:00+00:00
2019-05-12T18:36:00+00:00
2019-05-12T18:31:00+00:00
2019-05-12T18:26:00+00:00
2019-05-12T18:21:00+00:00
2019-05-12T18:16:00+00:00
2019-05-12T18:11:00+00:00
2019-05-12T18:06:00+00:00
2019-05-12T18:01:00+00:00
2019-05-12T17:56:00+00:00
2019-05-12T17:51:00+00:00
2019-05-12T17:46:00+00:00
2019-05-12T17:41:00+00:00
2019-05-12T17:36:00+00:00
2019-05-12T17:31:00+00:00
2019-05-12T17:26:00+00:00
2019-05-12T17:21:00+00:00
2019-05-12T17:16:00+00:00
2019-05-12T17:11:00+00:00
2019-05-12T17:06:00+00:00
2019-05-12T17:01:00+00:00
2019-05-12T16:56:00+00:00
2019-05-12T16:51:00+00:00
2019-05-12T16:46:00+00:00
2019-05-12T16:41:00+00:00
2019-05-11T16:11:00+00:00
2019-05-11T16:06:00+00:00
2019-05-11T16:01:00+00:00
2019-05-11T15:56:00+00:00
2019-05-11T15:51:00+00:00
2019-05-11T15:46:00+00:00
2019-05-11T15:41:00+00:00
2019-05-11T15:36:00+00:00
2019-05-11T15:31:00+00:00
2019-05-11T15:26:00+00:00
2019-05-11T15:21:00+00:00
2019-05-11T15:16:00+00:00
2019-05-11T15:11:00+00:00
2019-05-11T15:06:00+00:00
2019-05-11T15:01:00+00:00
# Examining the first item in the collection
items[0]
# Restructure our items into a dictionary where keys are the datetime items
# Then we can query more easily by date/time, e.g. "2020"
items_dict = {item.properties["datetime"]: item for item in items} 
# Before we go further, let's pick which asset to focus on for the remainder of the notebook.
# This dataset only has one variable, so our asset is:
asset_name = "rad"

Creating Maps Using Folium

You will now visualize the evolution of a methane plume across for three different time steps using folium.

Fetch Imagery Using Raster API

Here we get information from the Raster API which we will add to our map in the next section.

# Take a look at the datetime options we have to choose from based on our search
list(items_dict.keys())
['2019-05-12T19:41:00Z',
 '2019-05-12T19:36:00+00:00',
 '2019-05-12T19:31:00+00:00',
 '2019-05-12T19:26:00+00:00',
 '2019-05-12T19:21:00+00:00',
 '2019-05-12T19:16:00+00:00',
 '2019-05-12T19:11:00+00:00',
 '2019-05-12T19:06:00+00:00',
 '2019-05-12T19:01:00+00:00',
 '2019-05-12T18:56:00+00:00',
 '2019-05-12T18:51:00+00:00',
 '2019-05-12T18:46:00+00:00',
 '2019-05-12T18:41:00+00:00',
 '2019-05-12T18:36:00+00:00',
 '2019-05-12T18:31:00+00:00',
 '2019-05-12T18:26:00+00:00',
 '2019-05-12T18:21:00+00:00',
 '2019-05-12T18:16:00+00:00',
 '2019-05-12T18:11:00+00:00',
 '2019-05-12T18:06:00+00:00',
 '2019-05-12T18:01:00+00:00',
 '2019-05-12T17:56:00+00:00',
 '2019-05-12T17:51:00+00:00',
 '2019-05-12T17:46:00+00:00',
 '2019-05-12T17:41:00+00:00',
 '2019-05-12T17:36:00+00:00',
 '2019-05-12T17:31:00+00:00',
 '2019-05-12T17:26:00+00:00',
 '2019-05-12T17:21:00+00:00',
 '2019-05-12T17:16:00+00:00',
 '2019-05-12T17:11:00+00:00',
 '2019-05-12T17:06:00+00:00',
 '2019-05-12T17:01:00+00:00',
 '2019-05-12T16:56:00+00:00',
 '2019-05-12T16:51:00+00:00',
 '2019-05-12T16:46:00+00:00',
 '2019-05-12T16:41:00+00:00',
 '2019-05-11T16:11:00+00:00',
 '2019-05-11T16:06:00+00:00',
 '2019-05-11T16:01:00+00:00',
 '2019-05-11T15:56:00+00:00',
 '2019-05-11T15:51:00+00:00',
 '2019-05-11T15:46:00+00:00',
 '2019-05-11T15:41:00+00:00',
 '2019-05-11T15:36:00+00:00',
 '2019-05-11T15:31:00+00:00',
 '2019-05-11T15:26:00+00:00',
 '2019-05-11T15:21:00+00:00',
 '2019-05-11T15:16:00+00:00',
 '2019-05-11T15:11:00+00:00',
 '2019-05-11T15:06:00+00:00',
 '2019-05-11T15:01:00+00:00']
# Specify dates/times to visualize
# We'll select three observations, each 15 minutes apart
datetimes = ["2019-05-12T17:21:00+00:00","2019-05-12T17:36:00+00:00","2019-05-12T17:51:00+00:00"]
# Extract collection name and item ID for the first date
observation_date_1 = items_dict[datetimes[0]]
collection_id = observation_date_1.collection_id
item_id = observation_date_1.id
# Select relevant asset and get raster bands
object = observation_date_1.assets[asset_name]
raster_bands = object.extra_fields.get("raster:bands", [{}])
# Print the raster bands' information
raster_bands
[{'scale': 1.0,
  'nodata': 0.0,
  'offset': 0.0,
  'sampling': 'area',
  'data_type': 'float64',
  'histogram': {'max': 0.27649252674814284,
   'min': -0.10735534384080142,
   'count': 11,
   'buckets': [3, 4, 11, 19, 28, 19, 18, 6, 6, 4]},
  'statistics': {'mean': 0.08132970903405966,
   'stddev': 0.07654679032735916,
   'maximum': 0.27649252674814284,
   'minimum': -0.10735534384080142,
   'valid_percent': 51.75438596491229}}]
# Use min and max to define desired colorbar range
rescale_values = {
    "max": raster_bands[0]['statistics']['maximum'],
    "min": raster_bands[0]['statistics']['minimum'],
}

print(rescale_values)
{'max': 0.27649252674814284, 'min': -0.10735534384080142}

Now, you will pass the item id, collection name, asset name, and the rescale values to the Raster API endpoint, along with a colormap. This step is required for every date/time you want to visualize, and tells the Raster API which collection, item, and asset you want to view, specifying the colormap and colorbar ranges to use for visualization. The API returns a JSON with information about the requested image. Each image will be referred to as a tile.

# Choose a colormap for displaying the data
# Make sure capitilization follows Matplotlib standard practice
# For more information on Colormaps in Matplotlib, please visit https://matplotlib.org/stable/users/explain/colors/colormaps.html
color_map = "plasma" # please select the color ramp from matplotlib library.
items.keys()
# Make a GET request to retrieve information for your first date/time
observation_1_tile = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map}"
    f"&rescale={rescale_values['min']},{rescale_values['max']}", 
).json()
# Print the resulting JSON
observation_1_tile
{'tilejson': '2.2.0',
 'version': '1.0.0',
 'scheme': 'xyz',
 'tiles': ['https://earth.gov/ghgcenter/api/raster/collections/goes-ch4plume-v1/items/goes-ch4plume-v1-GOES-CH4_Mexico_Durango_BV2_BV2-5_2019-05-12T17:21:00Z/tiles/WebMercatorQuad/{z}/{x}/{y}@1x?assets=rad&color_formula=gamma+r+1.05&colormap_name=plasma&rescale=-0.10735534384080142%2C0.27649252674814284'],
 'minzoom': 0,
 'maxzoom': 24,
 'bounds': [-104.38926051070443,
  26.073879500563976,
  -104.11975109944395,
  26.24409597083376],
 'center': [-104.25450580507419, 26.15898773569887, 0]}
# Repeat the previous step, fetching the tiles for the second and third datetimes of interest
observation_date_2 = items_dict[datetimes[1]]
collection_id = observation_date_2.collection_id
item_id = observation_date_2.id
observation_2_tile = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map}"
    f"&rescale={rescale_values['min']},{rescale_values['max']}", 
).json()
observation_date_3 = items_dict[datetimes[2]]
collection_id = observation_date_3.collection_id
item_id = observation_date_3.id
observation_3_tile = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map}"
    f"&rescale={rescale_values['min']},{rescale_values['max']}", 
).json()

Generate Map

# Set initial zoom and center of map
# We'll use the "center" variable from our first methane plume tile to set the center of the map
# Note that we specify "tiles=None" because in the next step we're going to set a custom tile to serve as our underlying world map.
map_ = folium.Map(
    location=(observation_1_tile["center"][1], observation_1_tile["center"][0]), 
    zoom_start=9, 
    tiles=None
)

# Specify a custom imagery source for the underlying map
folium.TileLayer(
    tiles='https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}.png', 
    name='ESRI World Imagery', 
    attr='Tiles © Esri — Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP, and the GIS User Community',
    overlay='True').add_to(map_)

# Add place labels on top
folium.TileLayer(tiles='https://server.arcgisonline.com/arcgis/rest/services/Reference/World_Boundaries_and_Places/MapServer/tile/{z}/{y}/{x}.png',
                 name='ESRI World Boundaries and Places',
                 attr='Tiles © Esri — Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP, and the GIS User Community',
                 overlay='True').add_to(map_)

# Use the 'TileLayer' library to create the map layer for your first date/time tile
map_layer = TileLayer(
    tiles=observation_1_tile["tiles"][0], # Path to retrieve the tile
    name=f'{items_dict[datetimes[0]].assets[asset_name].title}, {items_dict[datetimes[0]].properties["datetime"]}', # Give this layer a title
    overlay='True', # The layer can be overlaid on the map
    attr="GHG", # Set the attribution
    opacity=1, # Adjust the transparency of the layer
)
# Add layer to the map
map_layer.add_to(map_)

# Repeat for second date/time tile
map_layer = TileLayer(
    tiles=observation_2_tile["tiles"][0], # Path to retrieve the tile
    name=f'{items_dict[datetimes[1]].assets[asset_name].title}, {items_dict[datetimes[1]].properties["datetime"]}', # Give this layer a title
    overlay='True', # The layer can be overlaid on the map
    attr="GHG", # Set the attribution
    opacity=1, # Adjust the transparency of the layer
)
map_layer.add_to(map_)

# And finally for the third date/time tile
map_layer = TileLayer(
    tiles=observation_3_tile["tiles"][0], # Path to retrieve the tile
    name=f'{items_dict[datetimes[2]].assets[asset_name].title}, {items_dict[datetimes[2]].properties["datetime"]}', # Give this layer a title
    overlay='True', # The layer can be overlaid on the map
    attr="GHG", # Set the attribution
    opacity=1, # Adjust the transparency of the layer
)
map_layer.add_to(map_)

# Add controls to switch elements of the map on/off
folium.LayerControl(collapsed=False, position='topright').add_to(map_)

# Add colorbar
# We can use 'generate_html_colorbar' from the 'ghgc_utils' module 
# to create an HTML colorbar representation.
legend_html = ghgc_utils.generate_html_colorbar(
                color_map,
                rescale_values,
                label='Methane Column Enhancement (mol/m2)'
    )

# Add colorbar to the map
map_.get_root().html.add_child(folium.Element(legend_html))


# Visualizing the map
map_
Make this Notebook Trusted to load map: File -> Trust Notebook

Click through the three methane layers to see the evolution of this plume over time!

# Set initial zoom and center of map for CH₄ Layer
# Centre of map [latitude,longitude]
map_ = folium.Map(location=(26.29, -104.53), zoom_start=10)

# April 2019
map_layer_2019 = TileLayer(
    tiles=april_2019_tile["tiles"][0],
    attr="GHG",
    opacity=0.7,
)
map_layer_2019.add_to(map_)
map_
Make this Notebook Trusted to load map: File -> Trust Notebook

Summary

In this notebook we have successfully completed the following steps for the STAC collection for the GOSAT-based Top-down Total and Natural Methane Emissions dataset.

  1. Install and import the necessary libraries
  2. Fetch the collection from STAC collections using the appropriate endpoints
  3. Count the number of existing granules within the collection
  4. Map the methane emission levels

If you have any questions regarding this user notebook, please contact us using the feedback form.

Back to top