IMPACT Brings Even More to AGU 2023

IMPACT Unofficial
8 min readDec 1, 2023

--

The 2023 American Geophysical Union conference, Wide. Open. Science. (December 11–15), will feature a number of IMPACT researchers. Their presentations, posters, and eLightning sessions will cover a range of topics including: satellite data processing and analysis; utilization of various frameworks and tools for Earth observation data; cloud-based data exploration, visualization, and analysis; environmental applications and climate research using Earth observation data; and improvement of data quality through validation frameworks and profiling tools. See below for when and where details.

CSDA @AGU2023

Posters

Publishing MAXAR® Data to NASA Earthdata Cloud using the Cumulus Framework

IN13B-0568: Monday, Dec. 11, 2023, 14:10–18:30 PST

The poster discusses NASA’s efforts to enhance accessibility to MAXAR satellite imagery data through cloud-based architecture and data pipelines within the Earthdata Cloud. Utilizing the NASA Commercial Smallsat Data Acquisition Program, the petabyte-scale MAXAR data archive is being migrated to AWS cloud and incorporated into the ESDIS Earthdata Cloud environment using the Cumulus framework. This initiative aims to improve access for the broader NASA scientific community.

SREDD: Framework to facilitate MLopS related to discovery of commercial satellite data of multiple resolution

GC11E-0887: Monday, Dec. 11, 2023, 08:30–12:50 PST

This poster presents Super Resolution Event Detection Dashboard (SREDD), an architecture designed to automate the ML model inference process. The architecture leverages Amazon S3, Elastic Cluster Service (ECS), Managed Workflows for Apache Airflow (MWAA), and other cloud-based services that efficiently handle geospatial data management. We collected daily Planet imagery of 20 lakes across the USA, Canada and Europe. Attention U-Net was used to train over 15 lakes on four bands, Red, Green, Blue, and NIR, and tested over five lakes.

eLightning

Planet Utilities: A Python Package for Efficient Processing and Analysis of PlanetScope Satellite Data

IN24B-10: Tuesday, Dec. 12, 2023, 16:27–16:30 PST

This session introduces “Planet Utilities,” a Python package aimed at simplifying the processing and analysis of high-resolution PlanetScope satellite data. It offers a range of functions for metadata extraction, data reading, image plotting, reprojection, supervised classification, data validation, and feature extraction. The package accommodates users of varying expertise levels, providing both a user-friendly interface for beginners and customization options for advanced users, supported by extensive examples and tutorials. Overall, it serves as a valuable tool for researchers, students, and practitioners in fields like environmental monitoring and land cover classification, fostering collaboration and data-driven discoveries through its open-source nature on GitHub.

Presentations

Evaluation of Computer Vision Based Flood Mapping using High Resolution Satellite Imagery

H11A-04: Monday, Dec. 11, 2023, 09:00–09:10 PST

Computer vision techniques with high-resolution satellite imagery are pivotal for swift and precise mapping of flood-hit areas, offering an alternative to time-consuming traditional remote sensing methods. Initially, a Machine Learning (ML) model was developed and trained on a single flood event, showcasing its effectiveness in predicting affected areas on a small dataset. Now, an updated investigation involves refining the ML model using a larger training dataset encompassing about 5000 samples from multiple major flood events, aiming to assess its performance across diverse spatial and temporal conditions. This evaluation includes comprehensive accuracy assessments based on machine learning and remote sensing standards, alongside metrics like processing speeds and scalability.

NASA’s Commercial Smallsat Data Acquisition (CSDA) Program Data System Services

IN34B-05: Wednesday, Dec. 13, 2023, 16:40–16:50 PST

The NASA Commercial Smallsat Data Acquisition (CSDA) Program aims to acquire and manage commercial satellite data that aligns with NASA’s Earth sciences missions. They’ve developed a scalable data system ensuring efficient management of acquired commercial data, focusing on improving end user services for data access and information about archived products. The presentation outlines recent additions to the CSDA data, enhancements to search and download functions via the Smallsat Data Explorer, integration with the Earth Observing System Data and Information System, efforts for long-term preservation of commercial datasets, and new user services for better dissemination of acquired data information.

GHG @AGU2023

Poster

Utilization of Multi-Agency Collaboration and a Comprehensive and Modern Data Management Approach to Achieve an Impactful Greenhouse Data Exploration and Analysis System

IN41C-0612: Thursday, Dec. 14, 2023, 08:30–12:50 PST

The complexity of data management is amplified in multi-agency environments, necessitating collaborative efforts to develop a tailored data governance strategy. The NASA MSFC Information System Development and Integration team is constructing a multi-agency data system for the U.S. Greenhouse Gas Center, aiming to offer visualization and analysis tools for authoritative greenhouse gas data from partners like EPA, NASA, NOAA, and NIST. Their efforts involve enhancing metadata consistency, accommodating unique stewardship needs, employing modern data handling methods, ensuring open access, and implementing a governance framework with automated checks to maintain data integrity and compliance with standards.

HLS @AGU2023

Poster

An Update on NASA’s Harmonized Landsat and Sentinel-2 Project

IN43C-0644: Thursday, Dec. 14, 2023, 14:10–18:30 PST

The HLS project harmonizes Landsat and Sentinel-2 data into interchangeable 30-meter datasets (L30 & S30) to meet the need for more frequent satellite revisit times. NASA initiated the release of global HLS products in 2021, offering access via cloud-based services and direct download from LP DAAC, accumulating a sizable 10-year archive starting from 2013 totaling nearly 4 PB in size. This poster updates the HLS product status, showcases its applications, and outlines future plans and derived products catering to user requirements.

eLightning

Assessment of Atmospheric Correction Methods on Harmonized Landsat and Sentinel-2 (HLS) Products

EP34B-06: Wednesday, Dec. 13, 2023, 16:17–16:20 PST

The HLS project generates global surface reflectance data from Landsat-8/9 and Sentinel-2A/2B satellites with high temporal frequency. It utilizes the LaSRC algorithm for atmospheric correction, currently reliant on MODIS data, but transitioning to VIIRS due to MODIS reaching end-of-life. This presentation assesses the impact of newer LaSRC versions and VIIRS data on HLS surface reflectance, comparing it to alternative atmospheric correction algorithms across diverse test sites globally.

Presentation

The Harmonized Landsat and Sentinel-2 Version 2.0 Surface Reflectance Data Set

IN42B-05: Thursday, Dec. 14, 2023, 11:02–11:12 PST

The Harmonized Landsat and Sentinel-2 (HLS) version 2.0 dataset is now complete, encompassing global land areas excluding Antarctica and utilizing measurements from Landsat OLI and Sentinel-2 MSI. This collaborative project between NASA and USGS ensures 30-meter surface reflectance data comparability by applying algorithms like LaSRC for reflectance derivation, Fmask for cloud detection, and adjustments for spectral and view angle differences. Evaluation using same-day observation pairs from globally distributed sites reveals atmospheric correction’s impact on reducing reflectance differences, highlighting directional characteristics uncovered by this correction, while subsequent BRDF correction effectively minimizes between-sensor differences, and bandpass adjustment has a limited effect.

VEDA @AGU2023

Poster

Advancing Climate Research Through the U.S. Greenhouse Gas Center and VEDA Open Science Platform

SY21C-0824: Tuesday, Dec. 12, 2023, 08:30–12:50 PST

The U.S. Greenhouse Gas Center (GHG Center) emerges as a collaborative hub integrating data from agencies like the EPA, NASA, NIST, and NOAA to enable informed decision making in combating climate change. This initiative emphasizes seamless interagency collaboration, open data accessibility, and user-friendly tools facilitated by the VEDA open-science platform. The GHG Center’s beta version, powered by VEDA, centralizes and curates greenhouse gas data, emphasizing dataset curation, validation, and user-friendly features for effective implementation and strategic decision making.

NASA Kiosk Demo

Visualization, Exploration, and Data Analysis (VEDA): A Pathfinder System to Support Open Science

Demo Kiosk 1: Wednesday, Dec. 13, 2023, 10:00–11:00 PST

VEDA enables discovery, accessibility, and visualization of NASA Earth science data for a broad user community via its visualization and exploration dashboard, the VEDA spatiotemporal asset catalog (STAC), visualization and configuration APIs, as well as an open analytical hub for advanced researchers to perform collaborative analysis and research on cloud-based datasets using cloud computing. All the services provided by VEDA are designed to lower the barrier to entry for science enthusiasts and new researchers, support the transition of legacy workflows to a modernized cloud-based workflow, and improve the efficiency of scientific research.

Presentations

An Open Platform for Cloud-Based, Cloud-Optimized Data Exploration, Visualization, and Analysis

IN11A-02: Monday, Dec. 11, 2023, 08:45–08:55 PST

NASA’s VEDA platform, designed for Earth science data exploration, offers a suite of services like visualization dashboards, JupyterHub for analysis, and a data catalog, supporting various user expertise levels. Utilizing cloud-native tools and adhering to STAC and OGC standards, VEDA facilitates efficient data access via APIs and promotes interoperability with similar platforms. Its modular, open-source design has attracted initiatives like the U.S. Greenhouse Gas Center and IEEE GRSS, showcasing adaptability for specific data needs and aiming to engage a broad user and development community in Earth science exploration.

Lowering the Barrier to Entry for NASA Earth Science Data in the Cloud

ED21A-02: Tuesday, Dec. 12, 2023, 08:43–08:53 PST

NASA’s Earth Science Data Systems (ESDS) face challenges due to growing archive volumes, projected to exceed 600 PB by 2029, further complicated by data migration to the cloud. The VEDA platform aims to simplify user access to this data by abstracting complexities, offering dynamic visualization compatible with GIS services, and providing data stories with Jupyter notebooks for both novice and experienced users. With features like a dashboard for exploration and a JupyterHub for cloud-based execution, VEDA aims to enable educational workshops, support science in the cloud, and promote open science through accessible data exploration and analysis.

Enhancing Data Quality: Leveraging Profiling Tools and Validation Frameworks

IN21A-06: Tuesday, Dec. 12, 2023, 09:20–09:30 PST

NASA’s VEDA platform aims to democratize access to scientific datasets by focusing on cloud-optimized data, intending to simplify access to this new data paradigm. Converting legacy data to cloud-optimized formats, essential for platforms like VEDA, can challenge existing validation workflows, prompting a need for automated validation techniques. The presentation will discuss automated validation methods, employing tools like pyQuARC and Great Expectations, crucial for ensuring data quality and metadata compliance with STAC specifications, supporting data standardization and interoperability among cloud-based platforms like VEDA.

Next-Gen Zarr Web Map Visualization (Invited)

IN42A-01: Thursday, Dec. 14 2023, 10:20–10:30 PST

This presentation demonstrates the importance of visualizing Earth science data in web browsers and highlights challenges due to the scale of geospatial data and the demand for fast rendering. As Zarr gained popularity for large-scale data analysis, users sought browser-based visualization tools, leading to the creation of two options: a dynamic tile server and a direct client. The authors contributed to both approaches, offering insights and testing results in a “Zarr Visualization Cookbook” to help users understand tradeoffs, preprocessing requirements, and performance testing for delivering Zarr data in web browsers, aiming to enhance the adoption of this format for understanding large-scale environmental data.

eoAPI: an open source, configurable and innovative cyberinfrastructure for delivering the SpatioTemporal Asset Catalog (STAC) and data services

INV53A-06: Friday, Dec. 15, 2023, 15:30–15:42 PST

This presentation introduces eoAPI, an open-source, cloud-computing infrastructure focusing on Earth observation data, combining STAC data ingestion, hosting, querying services, raster, and vector services. It powers various applications like NASA’s VEDA Open Science Platform and AWS ASDI’s data catalog, leveraging the STAC specification for data discovery, visualization, machine learning model training, and Earth data science. While numerous open tools facilitate working with EO data at scale, the flexibility of STAC poses challenges in selecting interoperable projects; eoAPI simplifies this by offering a pre-built, opinionated blueprint for a cloud-native EO infrastructure, making it accessible to both STAC-curious and serious users.

VEDA: Empowering Open Science for Earth Observation Data

INV51D-13: Friday, Dec. 15, 2023, 16:00–16:25 PST

NASA’s VEDA platform embodies open science by promoting inclusivity and accessibility in scientific research through its commitment to openness and integration of community-standard open-source software. VEDA’s innovative components, like STAC storage, OGC-standard dynamic tiling, and a customizable dashboard, enhance data discovery and analytics, while its cloud-native approach ensures efficient data access and exploration. The presentation aims to demonstrate the seamless transition from the dashboard to the JupyterHub environment, showcasing how researchers can engage in open science by exploring, visualizing, and analyzing VEDA datasets through accessible APIs, highlighting VEDA’s versatility beyond its core platform.

--

--

IMPACT Unofficial

This is the unofficial blog of the Interagency Implementation and Advanced Concepts Team.