HDF5 documents and links Introduction to HDF5 HDF5 User Guide | And in this document, the HDF5 Reference Manual H5 H5A H5D H5E H5F H5G H5I H5P H5R H5S H5T H5Z Tools Datatypes

# H5Z: Filter and Compression Interface

## Filter and Compression API Functions

These functions enable the user to configure new filters for the local environment.

- H5Zfilter_avail

- H5Zregister

- H5Zunregister

The FORTRAN90 Interfaces:
In general, each FORTRAN90 subroutine performs exactly the same task as the corresponding C function.

- h5zfilter_avail_f

- h5zunregister_f

HDF5 supports a filter pipeline that provides the capability for standard and customized raw data processing during I/O operations. HDF5 is distributed with a small set of standard filters such as compression (gzip and a shuffling algorithm) and error checking (Fletcher32 checksum). For further flexibility, the library allows a user application to extend the pipeline through the creation and registration of customized filters.

As mentioned above, one set of filters distributed with HDF5 provides built-in methods for raw data compression. The flexibility of the filter pipeline implementation enables the definition of additional filters by a user application. A filter
— is associated with a dataset when the dataset is created,
— can be used only with chunked data (ie., datasets stored in the `H5D_CHUNKED` storage layout), and
— is applied independently to each chunk of the dataset.

The HDF5 library does not support filters for contiguous datasets because of the difficulty of implementing random access for partial I/O. Compact dataset filters are not supported because it would not produce significant results.

See The Dataset Interface (H5D) in the HDF5 User's Guide for further information regarding data compression.

Name: H5Zfilter_avail
Signature:
  herr_t `H5Zfilter_avail`(H5Z_filter_t `filter`)
Purpose:
  Determines whether a filter is available.
Description:
  `H5Zfilter_avail` determines whether the filter specified in `filter` is available to the application.
Parameters:

H5Z_filter_t `filter`
  IN: Filter identifier.

Returns:
  Returns a non-negative value if successful; otherwise returns a negative value.
Fortran90 Interface: h5zfilter_avail_f

      SUBROUTINE h5zfilter_avail_f(filter, status, hdferr)
        IMPLICIT NONE
        INTEGER, INTENT(IN)  :: filter     ! Filter
                                           ! Valid values are:
                                           !    H5Z_FILTER_DEFLATE_F
                                           !    H5Z_FILTER_SHUFFLE_F
                                           !    H5Z_FILTER_FLETCHER32_F
        LOGICAL, INTENT(OUT) :: status     ! Flag indicating whether
                                           ! filter is availble:
                                           !    .TRUE.
                                           !    .FALSE.
      END SUBROUTINE h5zfilter_avail_f

Name: H5Zregister
Signature:
  herr_t `H5Zregister`(const H5Z_class_t `filter_class`) )
Purpose:
  Registers new filter.
Description:
  `H5Zregister` registers a new filter with the HDF5 library.

  Making a new filter available to an application is a two-step process. The first step is to write the three filter callback functions described below: `can_apply_func`, `set_local_func`, and `filter_func`. This call to `H5Zregister`, registering the filter with the library, is the second step. The `can_apply_func` and `set_local_func` fields can be set to `NULL` if they are not required for the filter being registered.

  `H5Zregister` accepts a single parameter, the `filter_class` data structure, which is defined as follows:

             typedef struct H5Z_class_t {
                 H5Z_filter_t filter_id;
                 const char  *comment;
                 H5Z_can_apply_func_t can_apply_func;
                 H5Z_set_local_func_t set_local_func;
                 H5Z_func_t filter_func;
             } H5Z_class_t;

  `filter_id` is the identifier for the new filter. This is a user-defined value between `H5Z_FILTER_RESERVED` and `H5Z_FILTER_MAX`, both of which are defined in the HDF5 source file `H5Zpublic.h`.

  `comment` is used for debugging, may contain a descriptive name for the filter, and may be the null pointer.

  `can_apply_func`, described in detail below, is a user-defined callback function which determines whether the combination of the dataset creation property list values, the datatype, and the dataspace represent a valid combination to apply this filter to.

  `set_local_func`, described in detail below, is a user-defined callback function which sets any parameters that are specific to this dataset, based on the combination of the dataset creation property list values, the datatype, and the dataspace.

  `filter_func`, described in detail below, is a user-defined callback function which performs the action of the filter.

  The statistics associated with a filter are not reset by this function; they accumulate over the life of the library.

  The callback functions
  Before `H5Zregister` can link a filter into an application, three callback functions must be defined as described in the HDF5 Library header file `H5Zpublic.h`.

  The can apply callback function is defined as follows:
  typedef herr_t (*`H5Z_can_apply_func_t`) (hid_t `dcpl_id`, hid_t `type_id`, hid_t `space_id`)

  Before a dataset is created, the can apply callbacks for any filters used in the dataset creation property list are called with the dataset's dataset creation property list, `dcpl_id`, the dataset's datatype, `type_id`, and a dataspace describing a chunk, `space_id`, (for chunked dataset storage).

  This callback must determine whether the combination of the dataset creation property list settings, the datatype, and the dataspace represent a valid combination to which to apply this filter. For example, an invalid combination may involve the filter not operating correctly on certain datatypes, on certain datatype sizes, or on certain sizes of the chunk dataspace.

  This callback can be the `NULL` pointer, in which case the library will assume that the filter can be applied to a dataset with any combination of dataset creation property list values, datatypes, and dataspaces.

  The can apply callback function must return a positive value for a valid combination, zero for an invalid combination, and a negative value for an error.

  The set local callback function is defined as follows:
  typedef herr_t (*`H5Z_set_local_func_t`) (hid_t `dcpl_id`, hid_t `type_id`, hid_t `space_id`)

  After the can apply callbacks are checked for a new dataset, the set local callback functions for any filters used in the dataset creation property list are called. These callbacks receive `dcpl_id`, the dataset's private copy of the dataset creation property list passed in to `H5Dcreate` (i.e. not the actual property list passed in to `H5Dcreate`); `type_id`, the datatype identifier passed in to `H5Dcreate`, which is not copied and should not be modified; and `space_id`, a dataspace describing the chunk (for chunked dataset storage), which should also not be modified.

  The set local callback must set any filter parameters that are specific to this dataset, based on the combination of the dataset creation property list values, the datatype, and the dataspace. For example, some filters perform different actions based on different datatypes, datatype sizes, numbers of dimensions, or dataspace sizes.

  The set local callback may be the `NULL` pointer, in which case, the library will assume that there are no dataset-specific settings for this filter.

  The set local callback function must return a non-negative value on success and a negative value for an error.

  The filter operation callback function, defining the filter's operation on the data, is defined as follows: typedef size_t (*`H5Z_func_t`) (unsigned int `flags`, size_t `cd_nelmts`, const unsigned int `cd_values[]`, size_t `nbytes`, size_t *`buf_size`, void **`buf`)

  The parameters `flags`, `cd_nelmts`, and `cd_values` are the same as for the function `H5Pset_filter`. The one exception is that an additional flag, `H5Z_FLAG_REVERSE`, is set when the filter is called as part of the input pipeline.

  The parameter `*buf` points to the input buffer which has a size of `*buf_size` bytes, `nbytes` of which are valid data.

  The filter should perform the transformation in place if possible. If the transformation cannot be done in place, then the filter should allocate a new buffer with `malloc()` and assign it to `*buf`, assigning the allocated size of that buffer to `*buf_size`. The old buffer should be freed by calling `free()`.

  If successful, the filter operation callback function returns the number of valid bytes of data contained in `*buf`. In the case of failure, the return value is `0` (zero) and all pointer arguments are left unchanged.

Note:
  The `H5Zregister` interface is substantially revised from the HDF5 Release 1.4.x series. The `H5Z_class_t` struct and the set local and can apply callback functions first appeared in HDF5 Release 1.6.
Parameters:

const H5Z_class_t `filter_class`
  IN: Struct containing filter-definition information.

Returns:
  Returns a non-negative value if successful; otherwise returns a negative value.
Fortran90 Interface:
  None.

Name: H5Zunregister
Signature:
  herr_t `H5Zunregister`(H5Z_filter_t `filter`)
Purpose:
  Unregisters a filter.
Description:
  `H5Zunregister` unregisters the filter specified in `filter`.

  After a call to `H5Zunregister`, the filter specified in `filter` will no longer be available to the application.

Parameters:

H5Z_filter_t `filter`
  IN: Identifier of the filter to be unregistered.

Returns:
  Returns a non-negative value if successful; otherwise returns a negative value.
Fortran90 Interface: h5zunregister_f

      SUBROUTINE h5zunregister_f(filter, hdferr)
        IMPLICIT NONE
        INTEGER, INTENT(IN)  :: filter  ! Filter; one of the possible values:
                                        !    H5Z_FILTER_DEFLATE_F
                                        !    H5Z_FILTER_SHUFFLE_F
                                        !    H5Z_FILTER_FLETCHER32_F
        INTEGER, INTENT(OUT) :: hdferr  ! Error code
                                        ! 0 on success, and -1 on failure
      END SUBROUTINE h5zunregister_f

HDF5 documents and links Introduction to HDF5 HDF5 User Guide | And in this document, the HDF5 Reference Manual H5 H5A H5D H5E H5F H5G H5I H5P H5R H5S H5T H5Z Tools Datatypes