CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes

Global, monthly 1 degree resolution methane emission estimates from microbial, fossil and pyrogenic sources derived using inverse modeling, version 2023.
Author

Siddharth Chaudhary, Vishal Gaur

Published

November 15, 2024

Access this Notebook

You can launch this notebook in the US GHG Center JupyterHub by clicking the link below. If you are a new user, you should first sign up for the hub by filling out this request form and providing the required information.

Access the CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes notebook in the US GHG Center JupyterHub.

Table of Contents

Data Summary and Application

  • Spatial coverage: Global
  • Spatial resolution: 1° x 1°
  • Temporal extent: January 1998 - December 2021
  • Temporal resolution: Monthly
  • Unit: Grams of methane per square meter per year
  • Utility: Climate Research

For more, visit the CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes data overview page.

Approach

  1. Identify available dates and temporal frequency of observations for the given collection using the GHGC API /stac endpoint. The collection processed in this notebook is the CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes Data product.
  2. Pass the STAC item into the raster API /collections/{collection_id}/items/{item_id}/tilejson.jsonendpoint.
  3. Using folium.plugins.DualMap, we will visualize two tiles (side-by-side), allowing us to compare time points.
  4. After the visualization, we will perform zonal statistics for a given polygon.

About the Data

CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes

Surface methane (CH₄) emissions are derived from atmospheric measurements of methane and its ¹³C carbon isotope content. Different sources of methane contain different ratios of the two stable isotopologues, ¹²CH₄ and ¹³CH₄. This makes normally indistinguishable collocated sources of methane, say from agriculture and oil and gas exploration, distinguishable. The National Oceanic and Atmospheric Administration (NOAA) collects whole air samples from its global cooperative network of flasks (https://gml.noaa.gov/ccgg/about.html), which are then analyzed for methane and other trace gasses. A subset of those flasks are also analyzed for ¹³C of methane in collaboration with the Institute of Arctic and Alpine Research at the University of Colorado Boulder. Scientists at the National Aeronautics and Space Administration (NASA) and NOAA used those measurements of methane and ¹³C of methane in conjunction with a model of atmospheric circulation to estimate emissions of methane separated by three source types, microbial, fossil and pyrogenic.

For more information regarding this dataset, please visit the CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes data overview page.

Terminology

Navigating data via the GHGC API, you will encounter terminology that is different from browsing in a typical filesystem. We’ll define some terms here which are used throughout this notebook. - catalog: All datasets available at the /stac endpoint - collection: A specific dataset, e.g. CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes - item: One granule in the dataset, e.g. one monthly file of methane inverse fluxes - asset: A variable available within the granule, e.g. microbial, fossil, or pyrogenic methane fluxes - STAC API: SpatioTemporal Asset Catalogs - Endpoint for fetching metadata about available datasets - Raster API: Endpoint for fetching data itself, for imagery and statistics

Install the Required Libraries

Required libraries are pre-installed on the GHG Center Hub. If you need to run this notebook elsewhere, please install them with this line in a code cell:

%pip install requests folium rasterstats pystac_client pandas matplotlib –quiet

# Import the following libraries
# For fetching from the Raster API
import requests
# For making maps
import folium
import folium.plugins
from folium import Map, TileLayer
# For talking to the STAC API
from pystac_client import Client
# For working with data
import pandas as pd
# For making time series
import matplotlib.pyplot as plt
# For formatting date/time data
import datetime
# Custom functions for working with GHGC data via the API
import ghgc_utils

Query the STAC API

STAC API Collection Names

Now, you must fetch the dataset from the STAC API by defining its associated STAC API collection ID as a variable. The collection ID, also known as the collection name, for the CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes dataset is ct-ch4-monthgrid-v2023.*

**You can find the collection name of any dataset on the GHGC data portal by navigating to the dataset landing page within the data catalog. The collection name is the last portion of the dataset landing page’s URL, and is also listed in the pop-up box after clicking “ACCESS DATA.”*

# Provide the STAC and RASTER API endpoints
# The endpoint is referring to a location within the API that executes a request on a data collection nesting on the server.

# The STAC API is a catalog of all the existing data collections that are stored in the GHG Center.
STAC_API_URL = "https://earth.gov/ghgcenter/api/stac"

# The RASTER API is used to fetch collections for visualization
RASTER_API_URL = "https://earth.gov/ghgcenter/api/raster"

# The collection name is used to fetch the dataset from the STAC API. First, we define the collection name as a variable
# Name of the collection for CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes 
collection_name = "ct-ch4-monthgrid-v2023"
# Fetch the collection from the STAC API using the appropriate endpoint
# The 'pystac_client' library enables an HTTP request
catalog = Client.open(STAC_API_URL)
collection = catalog.get_collection(collection_name)

# Print the properties of the collection to the console
collection

Examining the contents of our collection under the temporal variable, we see that the data is available from January 1998 to December 2021. By looking at the dashboard:time density, we observe that the data is periodic with monthly time density.

#%%time
#items = list(collection.get_items())  # Convert the iterator to a list
#print(f"Found {len(items)} items")
>>>>>>> remote <modified: >
# The search function lets you search for items within a specific date/time range
search = catalog.search(
    collections=collection_name,
    datetime=['2010-01-01T00:00:00Z','2010-12-31T00:00:00Z']
)
# Take a look at the items we found
for item in search.item_collection():
    print(item)
<Item id=ct-ch4-monthgrid-v2023-201012>
<Item id=ct-ch4-monthgrid-v2023-201011>
<Item id=ct-ch4-monthgrid-v2023-201010>
<Item id=ct-ch4-monthgrid-v2023-201009>
<Item id=ct-ch4-monthgrid-v2023-201008>
<Item id=ct-ch4-monthgrid-v2023-201007>
<Item id=ct-ch4-monthgrid-v2023-201006>
<Item id=ct-ch4-monthgrid-v2023-201005>
<Item id=ct-ch4-monthgrid-v2023-201004>
<Item id=ct-ch4-monthgrid-v2023-201003>
<Item id=ct-ch4-monthgrid-v2023-201002>
<Item id=ct-ch4-monthgrid-v2023-201001>
# Examine the first item in the collection
# Keep in mind that a list starts from 0, 1, 2... therefore items[0] is referring to the first item in the list/collection
items = search.item_collection()
items[0]
# Restructure our items into a dictionary where keys are the datetime items
# Then we can query more easily by date/time, e.g. "2020"
items_dict = {item.properties["start_datetime"][:7]: item for item in items}
# Before we go further, let's pick which asset to focus on for the remainder of the notebook.
# We'll focus on microbial sources of CH4 fluxes, so our asset of interest is:
asset_name = "microbial"

Creating Maps using Folium

You will now explore changes in the microbial CH₄ flux for two different dates/times. You will visualize the outputs on a map using folium.

Fetch Imagery from Raster API

Here we get information from the Raster API which we will add to our map in the next section.

# Specify two date/times that you would like to visualize, using the format of items_dict.keys()
dates = ["2010-07","2010-01"]

Below, we use some statistics of the raster data to set upper and lower limits for our color bar. These are saved as the rescale_values, and will be passed to the Raster API in the following step(s).

# Extract collection name and item ID for the first date
observation_date_1 = items_dict[dates[0]]
collection_id = observation_date_1.collection_id
item_id = observation_date_1.id
# Select relevant asset (microbial CH4 emissions)
object = observation_date_1.assets[asset_name]
raster_bands = object.extra_fields.get("raster:bands", [{}])
# Print the raster bands' information
raster_bands
[{'scale': 1.0,
  'nodata': -9999.0,
  'offset': 0.0,
  'sampling': 'area',
  'data_type': 'float64',
  'histogram': {'max': 352.1198378793238,
   'min': 0.0,
   'count': 11,
   'buckets': [64535, 213, 31, 13, 3, 2, 1, 1, 0, 1]},
  'statistics': {'mean': 0.9793962770298661,
   'stddev': 5.309472357044403,
   'maximum': 352.1198378793238,
   'minimum': 0.0,
   'valid_percent': 100.0}}]
# Use mean, scaled stddev, and minimum to generate an appropriate color bar range.
rescale_values = {
    "max": raster_bands[0]['statistics']['mean'] + 2.5*raster_bands[0]['statistics']['stddev'],
    "min": raster_bands[0]['statistics']['minimum'],
}

print(rescale_values)
{'max': 14.253077169640875, 'min': 0.0}

Now, you will pass the item id, collection name, asset name, and the rescale values to the Raster API endpoint, along with a colormap. This step is done twice, one for each date/time you will visualize, and tells the Raster API which collection, item, and asset you want to view, specifying the colormap and colorbar ranges to use for visualization. The API returns a JSON with information about the requested image. Each image will be referred to as a tile.

# Choose a colormap for displaying the data
# Make sure to capitalize per Matplotlib standard colormap names
# For more information on Colormaps in Matplotlib, please visit https://matplotlib.org/stable/users/explain/colors/colormaps.html
color_map = "PuRd"
# Make a GET request to retrieve information for your first date/time
observation_date_1_tile = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map.lower()}"
    f"&rescale={rescale_values['min']},{rescale_values['max']}"
).json()

# Print the properties of the retrieved granule to the console
observation_date_1_tile
{'tilejson': '2.2.0',
 'version': '1.0.0',
 'scheme': 'xyz',
 'tiles': ['https://earth.gov/ghgcenter/api/raster/collections/ct-ch4-monthgrid-v2023/items/ct-ch4-monthgrid-v2023-201007/tiles/WebMercatorQuad/{z}/{x}/{y}@1x?assets=microbial&color_formula=gamma+r+1.05&colormap_name=purd&rescale=0.0%2C14.253077169640875'],
 'minzoom': 0,
 'maxzoom': 24,
 'bounds': [-180.0, -90.0, 180.0, 90.0],
 'center': [0.0, 0.0, 0]}
# Repeat the above for your second date/time
# Note that we do not calculate new rescale_values for this tile, because we want date tiles 1 and 2 to have the same colorbar range for best visual comparison.
observation_date_2 = items_dict[dates[1]]
# Extract collection name and item ID
collection_id = observation_date_2.collection_id
item_id = observation_date_2.id

observation_date_2_tile = requests.get(
    f"{RASTER_API_URL}/collections/{collection_id}/items/{item_id}/tilejson.json?"
    f"&assets={asset_name}"
    f"&color_formula=gamma+r+1.05&colormap_name={color_map.lower()}"
    f"&rescale={rescale_values['min']},{rescale_values['max']}"
).json()

# Print the properties of the retrieved granule to the console
observation_date_2_tile
{'tilejson': '2.2.0',
 'version': '1.0.0',
 'scheme': 'xyz',
 'tiles': ['https://earth.gov/ghgcenter/api/raster/collections/ct-ch4-monthgrid-v2023/items/ct-ch4-monthgrid-v2023-201001/tiles/WebMercatorQuad/{z}/{x}/{y}@1x?assets=microbial&color_formula=gamma+r+1.05&colormap_name=purd&rescale=0.0%2C14.253077169640875'],
 'minzoom': 0,
 'maxzoom': 24,
 'bounds': [-180.0, -90.0, 180.0, 90.0],
 'center': [0.0, 0.0, 0]}

Generate Map

First, we’ll define the Area of Interest (AOI) as a GEOJSON. This will be visualized as a filled polygon on the map.

# The AOI is currently set to Eastern Canada, North America.
aoi = {
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {},
      "geometry": {
        "coordinates": [
          [
            # [longitude, latitude]
            [-106.81091327586626,58.13717287115446],  # Southwest Bounding Coordinate
            [-106.81091327586626,46.689085955377266], # Southeast Bounding Coordinate
            [-84.5565048510494,46.689085955377266],   # Northeast Bounding Coordinate
            [-84.5565048510494,58.13717287115446],    # Northwest Bounding Coordinate
            [-106.81091327586626,58.13717287115446]   # Closing the polygon at the Southwest Bounding Coordinate
          ]
        ],
        "type": "Polygon"
      }
    }
  ]
}

We will use the DualMap format from folium to visualize our two dates side-by-side for the area of interest. Below we add the desired layers to our map and add markers to identify the date/times shown.

# Initialize the map, specifying the center of the map and the starting zoom level.
# 'folium.plugins' allows mapping side-by-side via 'DualMap'
# Map is centered on the position specified by "location=(lat,lon)"
map_ = folium.plugins.DualMap(location=(52, -95.3), zoom_start=4)

# Define the first map layer using the tile fetched for the first date
# The TileLayer library helps in manipulating and displaying raster layers on a map
map_layer_observation_date_1 = TileLayer(
    tiles=observation_date_1_tile["tiles"][0], # Path to retrieve the tile
    attr="GHG", # Set the attribution
    opacity=0.8, # Adjust the transparency of the layer
    name=f"{dates[0]} {items[0].assets['microbial'].title}",
    overlay=True,
)
# Add the first layer to the Dual Map
# This will appear on the left side, specified by 'm1'
map_layer_observation_date_1.add_to(map_.m1)


# Define the second map layer using the tile fetched for the second date
map_layer_observation_date_2 = TileLayer(
    tiles=observation_date_2_tile["tiles"][0], # Path to retrieve the tile
    attr="GHG", # Set the attribution
    opacity=0.8, # Adjust the transparency of the layer
    name=f"{dates[1]} {items[0].assets['microbial'].title}",
    overlay=True,
)
# Add the second layer to the Dual Map
# This will appear on the right side, specified by 'm2'
map_layer_observation_date_2.add_to(map_.m2)

# Display AOI on both maps
folium.GeoJson(aoi, name="Eastern Canada, North America").add_to(map_)

# Add a layer control to switch between map layers
folium.LayerControl(collapsed=False).add_to(map_)

# Add colorbar
# We can use 'generate_html_colorbar' from the 'ghgc_utils' module 
# to create an HTML colorbar representation.
legend_html = ghgc_utils.generate_html_colorbar(color_map,rescale_values,label='g CH4/m2/year',dark=True)

# Add colorbar to the map
map_.get_root().html.add_child(folium.Element(legend_html))

# Visualize the Dual Map
map_
Make this Notebook Trusted to load map: File -> Trust Notebook

This visualization effectively illustrates the difference in microbial activity in warm vs. cold temperatures, which vary most widely at high latitudes.

Calculate Zonal Statistics

To perform zonal statistics, first we need to create a polygon. In this use case we are creating a polygon in Eastern Canada, North America.

# Give your AOI a name to be used in your time series plot later on.
aoi_name='Eastern Canada'
# This AOI is defined as a GEOJSON.
aoi = {
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {},
      "geometry": {
        "coordinates": [
          [
            # [longitude, latitude]
            [-106.81091327586626,58.13717287115446],  # Southwest Bounding Coordinate
            [-106.81091327586626,46.689085955377266], # Southeast Bounding Coordinate
            [-84.5565048510494,46.689085955377266],   # Northeast Bounding Coordinate
            [-84.5565048510494,58.13717287115446],    # Northwest Bounding Coordinate
            [-106.81091327586626,58.13717287115446]   # Closing the polygon at the Southwest Bounding Coordinate
          ]
        ],
        "type": "Polygon"
      }
    }
  ]
}
# Quick Folium map to visualize this AOI
map_ = folium.Map(location=(53, -96.5), zoom_start=3)
# Add AOI to map
folium.GeoJson(aoi, name=aoi_name).add_to(map_)
# Add data layer to visualize number of grid cells within AOI
map_layer_observation_date_1.add_to(map_)
# Add a quick colorbar
legend_html = ghgc_utils.generate_html_colorbar(color_map,rescale_values,label='g CH4/m2/year',dark=True)
map_.get_root().html.add_child(folium.Element(legend_html))
map_
Make this Notebook Trusted to load map: File -> Trust Notebook

Generate the statistics for the AOI using a function from the ghgc_utils module, which fetches the data and its statistics from the Raster API.

%%time

# Statistics will be returned as a Pandas DataFrame
df = ghgc_utils.generate_stats(items,aoi,url=RASTER_API_URL,asset=asset_name)
Generating stats...
Done!
CPU times: user 100 ms, sys: 17.6 ms, total: 118 ms
Wall time: 7.66 s
# Take a look at the first 5 rows of our statistics DataFrame
df.head(5)
datetime min max mean count sum std median majority minority unique histogram valid_percent masked_pixels valid_pixels percentile_2 percentile_98 date
0 2010-12-01T00:00:00+00:00 0.00000000000000000000 6.82134329057177346556 0.57121562215132426488 259.83999633789062500000 148.42466516794601716356 0.87483299220655952322 0.15977512022801718117 0.00000000000000000000 0.00000000036842792514 284.00000000000000000000 [[219, 28, 36, 10, 2, 1, 1, 1, 0, 1], [0.0, 0.... 100.00000000000000000000 0.00000000000000000000 299.00000000000000000000 0.00000000000000000000 2.77342200516181325298 2010-12-01 00:00:00+00:00
1 2010-11-01T00:00:00+00:00 0.00000000000000000000 9.18063230321920364929 0.48526060207306331851 259.83999633789062500000 126.09011306558737430805 0.85094422073117337302 0.15373060116246106688 0.00000000000000000000 0.00000000035909511554 284.00000000000000000000 [[245, 39, 7, 2, 3, 1, 0, 0, 0, 2], [0.0, 0.91... 100.00000000000000000000 0.00000000000000000000 299.00000000000000000000 0.00000000000000000000 2.95262395920586362408 2010-11-01 00:00:00+00:00
2 2010-10-01T00:00:00+00:00 0.00000000000000000000 20.97541986485276765961 1.30173648352961701669 259.83999633789062500000 338.24320311323430132688 2.93905303511565607621 0.25056334584628725537 0.00000000000000000000 0.00000000035415695592 284.00000000000000000000 [[260, 17, 9, 2, 3, 2, 2, 2, 0, 2], [0.0, 2.09... 100.00000000000000000000 0.00000000000000000000 299.00000000000000000000 0.00000000000000000000 12.13802520760731518124 2010-10-01 00:00:00+00:00
3 2010-09-01T00:00:00+00:00 0.00000000000000000000 66.99361532484425652001 5.67575875359129611297 259.83999633789062500000 1474.78913374791295609612 10.38348145192236593459 1.19664200489307481767 0.00000000000000000000 0.00000000033616082800 284.00000000000000000000 [[234, 23, 19, 6, 5, 3, 3, 2, 2, 2], [0.0, 6.6... 100.00000000000000000000 0.00000000000000000000 299.00000000000000000000 0.00000000000000000000 41.57290663092389593203 2010-09-01 00:00:00+00:00
4 2010-08-01T00:00:00+00:00 0.00000000000000000000 97.63207641317480067755 10.64292497187927821756 259.83999633789062500000 2765.45758571755641241907 16.33012417584687270278 1.61103687945000451087 0.00000000000000000000 0.00000000029630847828 284.00000000000000000000 [[193, 45, 33, 7, 5, 7, 4, 2, 0, 3], [0.0, 9.7... 100.00000000000000000000 0.00000000000000000000 299.00000000000000000000 0.00000000000000000000 64.69942926410929828762 2010-08-01 00:00:00+00:00

Time-Series Analysis

Let’s look at the maximum methane flux within our AOI over the full time range of the dataset. The code below generates a time series plot from the table above.

# Figure size: 20 representing the width, 10 representing the height
df = df.sort_values(by="datetime")
fig = plt.figure(figsize=(10,5))

# Change 'which_stat' below if you would rather look at a different statistic, like minimum or mean.
which_stat = "mean"

plt.plot(
    [d[0:7] for d in df["datetime"]], # X-axis: sorted datetime
    df[which_stat], # Y-axis: maximum CH4 flux
    color="red", # Line color
    linestyle="-", # Line style
    linewidth=2, # Line width
    label=f"{items[0].assets[asset_name].title}", # Legend label
)

# Display legend
plt.legend()

# Insert label for the X-axis
plt.xlabel("Years")

# Insert label for the Y-axis
plt.ylabel("g CH₄/m²/year")
plt.xticks(rotation = 90)

# Show once-annual tick marks
#interval = 12
#ticks = range(0, len(df), interval)  # Tick positions
#plt.xticks(ticks, [d[0:7] for d in df["datetime"]][list(ticks)])  # Use the corresponding datetime values
plt.xticks([d[0:7] for d in df['datetime'][:]][::2])

# Insert title for the plot
plt.title(f"{which_stat.capitalize()} Monthly Flux from {items[0].assets[asset_name].title} sources within {aoi_name} AOI (1998-2021)")

# Add data citation
plt.text(
    [d[0:7] for d in df['datetime']][0],           # X-coordinate of the text
    df[which_stat].min(),                  # Y-coordinate of the text

    # Text to be displayed
    f"Source: {collection.title}",                  
    fontsize=8,                             # Font size
    horizontalalignment="left",              # Horizontal alignment
    verticalalignment="top",                 # Vertical alignment
    color="blue",                            # Text color
)


# Plot the time series
plt.show()

This is a visual representation of the seasonal cycle of microbial methane emissions in Eastern Canada over the span of 22 years!

Summary

In this notebook we have successfully completed the following steps for the STAC collection for the CarbonTracker-CH₄ Isotopic Methane Inverse Fluxes dataset: 1. Install and import the necessary libraries 2. Fetch the collection from STAC collections using the appropriate endpoints 3. Count the number of existing granules within the collection 4. Map and compare the CH₄ inverse fluxes over an area of interest for two distinctive dates/times 5. Create a table that displays the minimum, maximum, and sum of the CH₄ inverse fluxes for a specified region 6. Generate a time-series graph of the CH₄ inverse fluxes for a specified region

If you have any questions regarding this user notebook, please contact us using the feedback form.

Back to top