GOSAT-based Top-down Total and Natural Methane Emissions

Total and wetland yearly methane emissions derived using the GEOS-Chem global chemistry transport model with inclusion of GOSAT data for 2010 to 2022 on a 4°(latitude) x 5°(longitude) grid
Author

Siddharth Chaudhary, Vishal Gaur

Published

November 15, 2024

Access this Notebook

You can launch this notebook in the US GHG Center JupyterHub by clicking the link below. If you are a new user, you should first sign up for the hub by filling out this request form and providing the required information.

Access the GOSAT-based Top-down Total and Natural Methane Emissions notebook in the US GHG Center JupyterHub.

Table of Contents

Data Summary and Application

  • Spatial coverage: Global
  • Spatial resolution: 4° x 5°
  • Temporal extent: 2010 - 2022
  • Temporal resolution: Annual
  • Unit: Teragrams of methane per yea
  • Utility: Climate Research

For more, visit the GOSAT-based Top-down Total and Natural Methane Emissions data overview page.

Approach

  1. Identify available dates and temporal frequency of observations for the given collection using the GHGC API /stac endpoint. The collection processed in this notebook is the gridded methane emissions data product.
  2. Pass the STAC item into the raster API /collections/{collection_id}/items/{item_id}/tilejson.jsonendpoint.
  3. Using folium.plugins.DualMap, we will visualize two tiles (side-by-side), allowing us to compare time points.
  4. After the visualization, we will perform zonal statistics for a given polygon.

About the Data

GOSAT-based Top-down Total and Natural Methane Emissions

The NASA Carbon Monitoring System Flux (CMS-Flux) team analyzed remote sensing observations from Japan’s Greenhouse gases Observing SATellite (GOSAT) to produce the global Committee on Earth Observation Satellites (CEOS) CH₄ Emissions data product. They used an analytic Bayesian inversion approach and the GEOS-Chem global chemistry transport model to quantify annual methane (CH₄) emissions and their uncertainties at a spatial resolution of 1° by 1° and then projected these to each country for 2019.

For more information regarding this dataset, please visit the GOSAT-based Top-down Total and Natural Methane Emissions data overview page.

Terminology

Navigating data via the GHGC API, you will encounter terminology that is different from browsing in a typical filesystem. We’ll define some terms here which are used throughout this notebook. - catalog: All datasets available at the /stac endpoint - collection: A specific dataset, e.g. GOSAT-based Top-down Total and Natural Methane Emissions - item: One granule in the dataset, e.g. one annual file of fluxes - asset: A variable available within the granule, e.g. anthropogenic methane emissions - STAC API: SpatioTemporal Asset Catalogs - Endpoint for fetching metadata about available datasets - Raster API: Endpoint for fetching data itself, for imagery and statistics

Install the Required Libraries

The libraries below allow better execution of a query in the GHG Center Spatio Temporal Asset Catalog (STAC) Application Programming Interface (API), where the granules for this collection are stored. You will learn the functionality of each library throughout the notebook.

Required libraries are pre-installed on the GHG Center Hub. If you need to run this notebook elsewhere, please install them with this line in a code cell:

%pip install requests folium rasterstats pystac_client pandas matplotlib –quiet

# Import the following libraries
# For fetching from the Raster API
import requests
# For making maps
import folium
import folium.plugins
from folium import Map, TileLayer
# For talking to the STAC API
from pystac_client import Client
# For working with data
import pandas as pd
# For making time series
import matplotlib.pyplot as plt
# For formatting date/time data
import datetime
# Custom functions for working with GHGC data via the API
import ghgc_utils

Query the STAC API

STAC API Collection Names

Now, you must fetch the dataset from the STAC API by defining its associated STAC API collection ID as a variable. The collection ID, also known as the collection name, for the GOSAT-based Top-down Total and Natural Methane Emissions dataset is gosat-based-ch4budget-yeargrid-v1*

**You can find the collection name of any dataset on the GHGC data portal by navigating to the dataset landing page within the data catalog. The collection name is the last portion of the dataset landing page’s URL, and is also listed in the pop-up box after clicking “ACCESS DATA.”*

# Provide STAC and RASTER API endpoints
STAC_API_URL = "https://earth.gov/ghgcenter/api/stac"
RASTER_API_URL = "https://earth.gov/ghgcenter/api/raster"

# Please use the collection name similar to the one used in STAC collection.

# Name of the collection for gosat budget methane. 
collection_name = "gosat-based-ch4budget-yeargrid-v1"
# Using PySTAC client
# Fetch the collection from the STAC API using the appropriate endpoint
# The 'pystac' library makes an HTTP request
catalog = Client.open(STAC_API_URL)
collection = catalog.get_collection(collection_name)

# Print the properties of the collection to the console
collection

Examining the contents of our collection under the temporal variable, we see that the data is available from January 2010 to December 2022. By looking at the dashboard:time density, we observe that the data is annual over that time period.

items = list(collection.get_items())  # Convert the iterator to a list
print(f"Found {len(items)} items")
Found 13 items
# Examining the first item in the collection
items[0]
# Restructure our items into a dictionary where keys are the datetime items
# Then we can then query more easily by date/time, e.g. "2020"
items_dict = {item.properties["datetime"][:4]: item for item in collection.get_items()}
# Before we go further, let's pick which asset to focus on for the remainder of the notebook.
# For this collection, we'll focus on the posterior total methane emissions, or:
asset_name = "post-total"

Create Maps Using Folium

You will now explore differences in methane emissions at two different date/times. You will visualize the outputs on a map using folium.

Fetch Imagery from Raster API

Here we get information from the Raster API which we will add to our map in the next section.

# Specify two dates/times to visualize
dates = ['2010','2020']
first_date = items_dict[dates[0]]
# Extract collection name and item ID
collection_id = first_date.collection_id
item_id = first_date.id
# Isolate chosen asset
object = first_date.assets[asset_name]
# Load raster bands to obtain statistics
raster_bands = object.extra_fields.get("raster:bands", [{}])
# Print raster bands
print(raster_bands)
[{'scale': 1.0, 'nodata': -9999.0, 'offset': 0.0, 'sampling': 'area', 'data_type': 'float64', 'histogram': {'max': 6.551439731574392, 'min': -1.0882688201549864, 'count': 11, 'buckets': [2, 2966, 230, 61, 31, 9, 6, 3, 2, 2]}, 'statistics': {'mean': 0.15064792279767236, 'stddev': 0.48261592630870437, 'maximum': 6.551439731574392, 'minimum': -1.0882688201549864, 'valid_percent': 100.0}}]
# 
rescale_values = {
    "max": raster_bands[0].get("histogram", {}).get("max"),
    #"min": raster_bands[0].get("histogram", {}).get("min"),
    "min": -1*raster_bands[0].get("histogram", {}).get("max")
}

print(rescale_values)
{'max': 6.551439731574392, 'min': -6.551439731574392}

Now, you will pass the item id, collection name, asset name, and the rescale values to the Raster API endpoint, along with a colormap. This step is done twice, one for each date/time you will visualize, and tells the Raster API which collection, item, and asset you want to view, specifying the colormap and colorbar ranges to use for visualization. The API returns a JSON with information about the requested image. Each image will be referred to as a tile.

# Choose a color map for displaying the data
# For more information on Colormaps in Matplotlib, please visit https://matplotlib.org/stable/users/explain/colors/colormaps.html
color_map = "Spectral_r" 
# Make a GET request to retrieve information for the first date
# Note that the colormap needs to be all lowercase for this to work properly.
tile_1 = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map.lower()}"
    f"&rescale=-0,3" #{rescale_values['min']},{rescale_values['max']}"
).json()

# Print the properties of the retrieved granule to the console
tile_1
{'tilejson': '2.2.0',
 'version': '1.0.0',
 'scheme': 'xyz',
 'tiles': ['https://earth.gov/ghgcenter/api/raster/collections/gosat-based-ch4budget-yeargrid-v1/items/gosat-based-ch4budget-yeargrid-v1-2010/tiles/WebMercatorQuad/{z}/{x}/{y}@1x?assets=post-total&color_formula=gamma+r+1.05&colormap_name=spectral_r&rescale=-0%2C3'],
 'minzoom': 0,
 'maxzoom': 24,
 'bounds': [-182.5, -90.9777777777778, 177.5, 90.97777777777777],
 'center': [-2.5, -1.4210854715202004e-14, 0]}
second_date = items_dict[dates[1]]
# Extract collection name and item ID
collection_id = second_date.collection_id
item_id = second_date.id
# Make a GET request to retrieve information for the second date
tile_2 = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map.lower()}"
    f"&rescale=-0,3" #{rescale_values['min']},{rescale_values['max']}"
).json()

# Print the properties of the retrieved granule to the console
tile_2
{'tilejson': '2.2.0',
 'version': '1.0.0',
 'scheme': 'xyz',
 'tiles': ['https://earth.gov/ghgcenter/api/raster/collections/gosat-based-ch4budget-yeargrid-v1/items/gosat-based-ch4budget-yeargrid-v1-2020/tiles/WebMercatorQuad/{z}/{x}/{y}@1x?assets=post-total&color_formula=gamma+r+1.05&colormap_name=spectral_r&rescale=-0%2C3'],
 'minzoom': 0,
 'maxzoom': 24,
 'bounds': [-182.5, -90.9777777777778, 177.5, 90.97777777777777],
 'center': [-2.5, -1.4210854715202004e-14, 0]}

Generate Map

# Set initial zoom and center of map
# Centre of map set via location=(latitude,longitude)
map_ = folium.plugins.DualMap(location=(25, -80), zoom_start=3)

# Generate map layer showing first date
map_layer_1 = TileLayer(
    tiles=tile_1["tiles"][0],
    attr="GHG",
    name=f"{dates[0]}",
    opacity=0.7,
    overlay=True,
)
# Add layer to the map
map_layer_1.add_to(map_.m1)

# Mapy layer showing second date
map_layer_2 = TileLayer(
    tiles=tile_2["tiles"][0],
    attr="GHG",
    name=f"{dates[1]}",
    opacity=0.7,
    overlay=True,
)
# Add layer to the map
map_layer_2.add_to(map_.m2)

# Add controls to toggle map elements on/off
folium.LayerControl(collapsed=False).add_to(map_)

# Add colorbar
# We'll use a function from the'ghgc_utils' module create an HTML colorbar representation.
legend_html = ghgc_utils.generate_html_colorbar(color_map,rescale_values,label='tC/km2/year')

# Add colorbar to the map
map_.get_root().html.add_child(folium.Element(legend_html))

map_
Make this Notebook Trusted to load map: File -> Trust Notebook

Calculating Zonal Statistics

To perform zonal statistics, first we need to create a polygon. In this use case we are creating a polygon in Texas (USA).

aoi_name = "Texas, USA"
# Texas, USA
aoi = {
    "type": "Feature",
    "properties": {},
    "geometry": {
        "coordinates": [
            [
                [-95, 29],
                [-95, 33],
                [-104, 33],
                [-104,29],
                [-95, 29]
            ]
        ],
        "type": "Polygon",
    },
}

Generate the statistics for the AOI using a function from the ghgc_utils module, which fetches the data and its statistics from the Raster API.

%%time

# Statistics will be returned as a Pandas DataFrame
df = ghgc_utils.generate_stats(items,aoi,url=RASTER_API_URL,asset=asset_name)
# Print the first five rows of our statistics DataFrame
df.head(5)
Generating stats...
Done!
CPU times: user 49 ms, sys: 0 ns, total: 49 ms
Wall time: 2.63 s
datetime min max mean count sum std median majority minority unique histogram valid_percent masked_pixels valid_pixels percentile_2 percentile_98 date
0 2022-01-01T00:00:00+00:00 0.45840875428297767069 2.56343412333124387459 1.35011469853848287137 1.97999989986419677734 2.67322696791137648376 0.56693396422232944509 1.08787203828849254883 0.45840875428297767069 0.45840875428297767069 6.00000000000000000000 [[2, 0, 1, 0, 0, 2, 0, 0, 0, 1], [0.4584087542... 100.00000000000000000000 0.00000000000000000000 6.00000000000000000000 0.45840875428297767069 2.56343412333124387459 2022-01-01 00:00:00+00:00
1 2021-01-01T00:00:00+00:00 0.41740630436583853191 2.97732164356297479557 1.40986536395883654826 1.97999989986419677734 2.79153327946049589769 0.71912405002717583180 1.02930576307184451501 0.41740630436583853191 0.41740630436583853191 6.00000000000000000000 [[2, 0, 1, 0, 1, 0, 1, 0, 0, 1], [0.4174063043... 100.00000000000000000000 0.00000000000000000000 6.00000000000000000000 0.41740630436583853191 2.97732164356297479557 2021-01-01 00:00:00+00:00
2 2020-01-01T00:00:00+00:00 0.39260556661190376682 2.61960828202497308936 1.30133326455075293282 1.97999989986419677734 2.57663973350043917776 0.62324224034026320940 0.97894313322725490245 0.39260556661190376682 0.39260556661190376682 6.00000000000000000000 [[2, 0, 1, 0, 1, 0, 1, 0, 0, 1], [0.3926055666... 100.00000000000000000000 0.00000000000000000000 6.00000000000000000000 0.39260556661190376682 2.61960828202497308936 2020-01-01 00:00:00+00:00
3 2019-01-01T00:00:00+00:00 0.52185597867942956629 2.67803361303974618579 1.40968879498951937812 1.97999989986419677734 2.79118367291892877091 0.66793838483582601562 0.95320635699237210581 0.52185597867942956629 0.52185597867942956629 6.00000000000000000000 [[1, 1, 1, 0, 0, 1, 0, 1, 0, 1], [0.5218559786... 100.00000000000000000000 0.00000000000000000000 6.00000000000000000000 0.52185597867942956629 2.67803361303974618579 2019-01-01 00:00:00+00:00
4 2018-01-01T00:00:00+00:00 0.55289134985276822132 2.62495718225180807437 1.42760922946317037052 1.97999989986419677734 2.82666613138228051127 0.58242521861862195376 1.08680679995756301892 0.55289134985276822132 0.55289134985276822132 6.00000000000000000000 [[1, 1, 1, 0, 0, 1, 1, 0, 0, 1], [0.5528913498... 100.00000000000000000000 0.00000000000000000000 6.00000000000000000000 0.55289134985276822132 2.62495718225180807437 2018-01-01 00:00:00+00:00 
# Figure size: 10 is width, 5 is height
fig = plt.figure(figsize=(10, 5))

# Sort by date
df = df.sort_values(by="datetime")

# Choose a stat to plot: min, mean, max, median, etc.
which_stat = "mean"

plt.plot(
    [d[0:4] for d in df["datetime"]], # X-axis: sorted datetime
    df[which_stat], # Y-axis: maximum CO₂
    color="#6AB547", # Line color in hex format
    linestyle="-", # Line style
    linewidth=2, # Line width
    label=f"{items[0].assets[asset_name].title}", # Legend label
)

# Display legend
plt.legend()

# Insert label for the X-axis
plt.xlabel("Years")

# Insert label for the Y-axis
plt.ylabel("tC/km²/year")

# Insert title for the plot
plt.title(f"{which_stat.capitalize()} {items[0].assets[asset_name].title} for {aoi_name} (2010-2022)")

# Add data citation
plt.text(
    min([d[0:4] for d in df["datetime"]]),           # X-coordinate of the text
    df[which_stat].min(),                  # Y-coordinate of the text
    # Text to be displayed
    f"Source: {collection.title}",      #example text            
    fontsize=9,                             # Font size
    horizontalalignment="left",              # Horizontal alignment
    verticalalignment="top",                 # Vertical alignment
    color="blue",                            # Text color
)


# Plot the time series
plt.show()

Summary

In this notebook we have successfully completed the following steps for the STAC collection for the GOSAT-based Top-down Total and Natural Methane Emissions dataset.

  1. Install and import the necessary libraries
  2. Fetch the collection from STAC collections using the appropriate endpoints
  3. Count the number of existing granules within the collection
  4. Map the methane emission levels
  5. Generate zonal statistics for the area of interest (AOI)

If you have any questions regarding this user notebook, please contact us using the feedback form.

Back to top