Skip to content

ejoliet/asdf-firefly

Repository files navigation

asdf-firefly

asdf-firefly is a command-line tool for inspecting Roman Space Telescope ASDF files, navigating their metadata tree, previewing tabular data, and opening FITS-compatible image data in Firefly.

ASDF, the Advanced Scientific Data Format, stores hierarchical metadata and binary arrays in a versioned scientific data container. This tool is designed for quick terminal exploration of Roman WFI products before handing visual inspection to Firefly's browser-based FITS image and table viewers.

Features

  • Navigate an ASDF file's header and metadata tree from the terminal.
  • Search, filter, and print selected metadata paths.
  • Discover arrays, tables, and FITS-like image products inside a Roman ASDF file.
  • Operate on local ASDF files, HTTP(S) ASDF URLs, and S3 ASDF URIs without requiring a manual pre-download.
  • Preview tabular data as aligned terminal output, CSV, ECSV, Parquet, or VOTable.
  • Extract image-like arrays to temporary FITS files.
  • Open extracted FITS images in Firefly for interactive visualization.
  • Open exported tables in Firefly for sorting, filtering, plotting, and row inspection.

Installation

pip install asdf-firefly

Install remote-object-store support when working with S3-hosted Roman products:

pip install "asdf-firefly[remote]"

Install Roman extension support to reduce ASDF warnings and enable mission-specific tagged objects:

pip install "asdf-firefly[roman]"

For development:

git clone https://github.com/example/asdf-firefly.git
cd asdf-firefly
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Use the same Python environment for pip and asdf-firefly. If the console script is in a different environment than the dependencies, commands can fail with ModuleNotFoundError, such as No module named 'asdf'. Check with:

which python
which pip
which asdf-firefly
python -m pip show asdf asdf-firefly

If a private package index rejects credentials, install from PyPI explicitly:

python -m pip install asdf astropy numpy platformdirs rich -i https://pypi.org/simple
python -m pip install -e . --no-build-isolation

Quick Start

Inspect the top-level ASDF metadata:

asdf-firefly info roman-wfi-product.asdf

Browse the ASDF tree:

asdf-firefly tree roman-wfi-product.asdf

Show a specific metadata path:

asdf-firefly get roman-wfi-product.asdf /meta/instrument

Validate an ASDF file:

asdf-firefly validate roman-wfi-product.asdf

List table-like objects:

asdf-firefly tables roman-wfi-product.asdf

Preview a table in the terminal:

asdf-firefly table show roman-wfi-product.asdf /catalog --limit 20

Open an image-like array in Firefly:

asdf-firefly image open roman-wfi-product.asdf /data --firefly

The FILE.asdf argument in commands may be a local path, an HTTP(S) URL, or an S3 URI when remote support is installed:

asdf-firefly info s3://roman-products/wfi/roman-wfi-product.asdf
asdf-firefly tree https://archive.example.org/roman/roman-wfi-product.asdf --depth 2

Command Overview

All commands that accept FILE.asdf accept the same input source forms:

  • Local paths such as roman-wfi-product.asdf.
  • HTTP(S) URLs supported directly by ASDF Python.
  • S3 URIs such as s3://bucket/key.asdf when the remote extra is installed.

Remote sources are opened through seekable file-like objects where possible so metadata inspection and bounded previews do not force full array or table downloads. Commands that export or open data in Firefly may read the selected array or table because those commands explicitly require data materialization.

info

Print a compact summary of an ASDF file.

asdf-firefly info FILE.asdf

Example output:

File: roman-wfi-product.asdf
ASDF standard: 1.6.0
Tree keys: asdf_library, history, meta, data, dq, err, catalog
Arrays: 3
Tables: 1
FITS-like images: 1
External blocks: 0

tree

Display the ASDF metadata tree.

asdf-firefly tree FILE.asdf

Useful options:

asdf-firefly tree FILE.asdf --depth 3
asdf-firefly tree FILE.asdf --arrays
asdf-firefly tree FILE.asdf --types
asdf-firefly tree FILE.asdf --find exposure

get

Print a value at a tree path.

asdf-firefly get FILE.asdf /meta/telescope
asdf-firefly get FILE.asdf /meta --format json
asdf-firefly get FILE.asdf /meta/instrument --format yaml

Paths use slash-separated ASDF tree keys. Array and table values are summarized by default to avoid dumping large binary data to the terminal.

validate

Validate an ASDF file and report schema or extension problems.

asdf-firefly validate FILE.asdf

Useful options:

asdf-firefly validate FILE.asdf --strict
asdf-firefly validate FILE.asdf --format json

Roman products may emit warnings about missing ASDF extension packages, such as roman_datamodels, asdf-astropy, or gwcs. Basic metadata inspection can still work with raw Python fallback objects, but richer Roman, WCS, coordinate, and unit behavior requires the roman extra.

tables

List table-like objects discovered in the ASDF tree.

asdf-firefly tables FILE.asdf

Example output:

Path              Rows    Columns    Type
/catalog          4382    14         astropy.table.Table
/dq_summary       128     6          structured ndarray

table show

Preview tabular data in the terminal.

asdf-firefly table show FILE.asdf /catalog

Useful options:

asdf-firefly table show FILE.asdf /catalog --columns ra,dec,flux
asdf-firefly table show FILE.asdf /catalog --where "flux > 10"
asdf-firefly table show FILE.asdf /catalog --limit 50
asdf-firefly table show FILE.asdf /catalog --sort flux --descending

table export

Export tabular data for use in other tools.

asdf-firefly table export FILE.asdf /catalog catalog.csv

Supported output formats:

  • csv
  • ecsv
  • fits
  • parquet
  • votable

The output format is inferred from the filename unless --format is provided.

asdf-firefly table export FILE.asdf /catalog catalog.vot --format votable

table open

Export a table to a temporary file and open it in Firefly.

asdf-firefly table open FILE.asdf /catalog --firefly

If the table includes sky coordinates such as ra and dec, Firefly can use them for plotting and image overlays when paired with a loaded image.

images

List image-like arrays that can be converted to FITS.

asdf-firefly images FILE.asdf

Example output:

Path             Shape          Dtype      WCS
/data            4096 x 4096    float32    yes
/err             4096 x 4096    float32    yes

image export

Export an image-like ASDF array to FITS.

asdf-firefly image export FILE.asdf /data science.fits

Useful options:

asdf-firefly image export FILE.asdf /data science.fits --hdu SCI
asdf-firefly image export FILE.asdf /data science.fits --include-wcs
asdf-firefly image export FILE.asdf /data science.fits --overwrite

image open

Export an image-like ASDF array to a temporary FITS file and open it in Firefly.

asdf-firefly image open FILE.asdf /data --firefly

Use a local Firefly server:

asdf-firefly image open FILE.asdf /data --firefly-url http://localhost:8080/firefly

Use an existing Firefly deployment:

asdf-firefly image open FILE.asdf /data --firefly-url https://irsa.ipac.caltech.edu/applications/Firefly

Header Navigation

ASDF does not use a FITS-style flat card header as its native metadata model. Instead, metadata is stored in a hierarchical YAML tree. asdf-firefly treats this tree as the navigable header.

Common header commands:

asdf-firefly tree FILE.asdf --depth 2
asdf-firefly get FILE.asdf /meta
asdf-firefly get FILE.asdf /history
asdf-firefly tree FILE.asdf --find wcs

For ASDF files that contain embedded FITS metadata, use:

asdf-firefly fits-header FILE.asdf
asdf-firefly fits-header FILE.asdf --hdu SCI

Firefly Integration

Firefly is a web-based astronomy visualization framework from IPAC. It provides interactive FITS and HiPS image visualization, tabular data display, and plotting.

asdf-firefly does not require ASDF support inside Firefly. Instead, it bridges ASDF to Firefly by exporting compatible temporary products:

  • Image-like arrays are exported as FITS.
  • Tables are exported as VOTable, FITS table, CSV, or another supported table format.
  • WCS metadata is preserved when available.
  • Temporary products can be retained with --keep-temp.

Example:

asdf-firefly image open roman-wfi-product.asdf /data --firefly --keep-temp

Configuration

Create a configuration file at ~/.config/asdf-firefly/config.toml:

[firefly]
url = "http://localhost:8080/firefly"
keep_temp = false

[display]
max_rows = 50
max_width = 120

[export]
overwrite = false

[remote]
anonymous = false
block_size = "default"

Environment variables override configuration values:

export ASDF_FIREFLY_URL=http://localhost:8080/firefly
export ASDF_FIREFLY_KEEP_TEMP=1

Examples

Find all Roman WCS-related paths:

asdf-firefly tree roman-wfi-product.asdf --find wcs

Preview Roman observation metadata:

asdf-firefly get roman-wfi-product.asdf /meta/observation --format yaml

Show a compact catalog preview:

asdf-firefly table show roman-wfi-product.asdf /catalog --columns ra,dec,mag --limit 10

Export a source catalog:

asdf-firefly table export roman-wfi-product.asdf /catalog sources.ecsv

Open a science image in Firefly:

asdf-firefly image open roman-wfi-product.asdf /data --firefly

Open a source catalog in Firefly:

asdf-firefly table open roman-wfi-product.asdf /catalog --firefly

Exit Codes

Code Meaning
0 Command completed successfully.
1 General runtime error.
2 Invalid command-line usage.
3 ASDF file could not be opened or validated.
4 Requested ASDF path was not found.
5 Requested object cannot be converted to the requested output format.
6 Firefly could not be reached or launched.

Development Notes

The intended implementation stack is:

  • asdf for ASDF reading and validation.
  • asdf-astropy, gwcs, and roman-datamodels for optional Roman tagged-object support.
  • astropy for tables, FITS export, WCS handling, and units.
  • rich for terminal tree and table rendering.
  • fsspec and s3fs for optional S3 URI access.
  • Python standard-library argparse for the initial CLI scaffold.
  • Firefly for browser-based FITS image and table visualization.

See DEVELOPER.md for architecture notes and IMPLEMENTATION_PLAN.md for the staged build plan.

Contributing

Contributions should preserve the core workflow: inspect ASDF safely, summarize large data by default, export explicit intermediate products, and hand visualization to Firefly. See CONTRIBUTING.md for setup, testing, issue triage, and pull request expectations.

Roadmap

  • Interactive terminal tree navigation.
  • Schema-aware metadata validation summaries.
  • Multi-extension FITS export from grouped ASDF arrays.
  • Firefly session bundles containing image, catalog, and overlay state.
  • Authenticated archive URL profiles beyond baseline HTTP(S) and S3 object access.
  • Plugin hooks for Roman mission-specific ASDF data models.

References

About

ASDF file explorer

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages