Turn Scientific Data Formats Into netCDF-Ready Intelligence

The Problem

Scientific data is often locked in specialized file formats that require custom parsers, brittle conversion pipelines, and duplicated tooling. Researchers waste time writing format-specific readers instead of doing science.

What NEP Adds to netCDF

NEP builds on two extension points already present in netCDF-C:

• HDF5 filter plugins for per-variable lossless compression (LZ4 and BZIP2).
• The NC_Dispatch layer for read-only User-Defined Format (UDF) readers.

Because NEP exposes foreign formats as ordinary netCDF datasets, existing analysis tools, visualization pipelines, and AI/ML workflows can work with DICOM images, protein structures, GRIB2 weather grids, and planetary data without custom parsers. New read-only or read-write readers can be added for additional formats.

NEP is built and maintained by our Senior Engineer, a co-author of the netCDF C/Fortran/C++ libraries and former GRIB/BUFR libraries team lead at NOAA.

Core Capabilities

• LZ4 compression – high-throughput, near-real-time workflows
• BZIP2 compression – maximum ratio for long-term archives
• UDF readers – open DICOM, PDB/mmCIF, GRIB2, GeoTIFF, FITS, CDF, and PDS4 through nc_open()
• Standard API – no changes to existing C, Fortran, or Python netCDF code
• CF-style attributes – native metadata translated to netCDF attributes where possible
• Open source – CMake, Autotools, Spack, and Conda builds available