ASDF - Advanced Scientific Data Format

asdf is a tool for reading and writing Advanced Scientific Data Format (ASDF) files.

The Advanced Scientific Data Format (ASDF) is a next-generation interchange format for scientific data. This package contains the Python implementation of the ASDF Standard. More information on the ASDF Standard itself can be found here.

The ASDF format has the following features:

• A hierarchical, human-readable metadata format (implemented using YAML)

• Numerical arrays are stored as binary data blocks which can be memory mapped. Data blocks can optionally be compressed.

• The structure of the data can be automatically validated using schemas (implemented using JSON Schema)

• Native Python data types (numerical types, strings, dicts, lists) are serialized automatically

• ASDF can be extended to serialize custom data types

Note

This is the Advanced Scientific Data Format - if you are looking for the Adaptable Seismic Data Format, go here: http://seismic-data.org/

Getting Started

• Installation
  • Requirements
  • Installing with pip
  • Installing with conda
  • Building from source
  • Running the tests
• Overview
  • Hello World
  • Creating Files
  • Reading Files
• Core Features
  • Data Model
  • Array Data
  • Using extensions
  • Schema validation
  • Versioning and Compatibility
  • External References
  • Saving history entries
  • Saving ASDF in FITS
  • Rendering ASDF trees
  • Searching the ASDF tree
  • Searching Schema information
• Configuration
  • AsdfConfig and you
  • Config options
  • Additional AsdfConfig features
• Command line tool

Extending ASDF

• Common use cases
  • Validate an ASDF tree against a schema
  • Serialize a new type
  • Support a new block compressor
• URIs in ASDF
  • URI vs URL
  • The asdf:// URI scheme
  • Entities identified by URI
• ASDF schemas
  • Anatomy of a schema
  • Checking schema syntax
  • Testing validation
  • See also:
• Resources and resource mappings
  • Installing resources via AsdfConfig
  • The DirectoryResourceMapping class
  • Installing resources via entry points
• Converters
  • The Converter interface
  • A simple example
  • Multiple tags
  • Reference cycles
  • Entry point performance considerations
• Extensions
  • Extension features
  • Installing an extension
  • Populating an extension from a manifest
• Extension manifests
  • Anatomy of a manifest
  • Validating a manifest
• Binary block compressors
  • The Compressor interface
  • Entry point performance considerations
• Deprecated extension API
  • An Example
  • Defining custom types
  • Creating custom schemas
  • Adding custom validators
  • Defining custom extension classes
  • Packaging custom extensions
  • Testing custom schemas

API Documentation

• User API
• Developer API

Developer Overview

Currently a work in progress. Intended to give an overview of how the various parts of ASDF interact and which modules do what and how.

• High level overview of the basic ASDF library
• Tag Versioning and You

Resources

• Contributing
• Change Log
• Citation

See also

• The Advanced Scientific Data Format (ASDF) standard.

• asdf Python package distribution on pypi

Index

• Index

• Module Index

• Search Page