Previous topic

Masking and null values

Next topic

Supported Formats

This Page

Custom reading/writingΒΆ

One of the new features introduced in ATpy 0.9.2 is the ability for users to write their own read/write functions and register them with ATpy. A read or write function needs to satisfy the following requirements:

  • The first argument should be a Table instance (in the case of a single table reader/writer) or a TableSet instance (in the case of a table set reader/writer)

  • The function can take any other arguments, with the exception of the keyword arguments verbose and type.

  • The function should not return anything, but rather should operate directly on the table or table set instance passed as the first argument

  • If the file format supports masking/null values, the function should take into account that there are two ways to mask values (see Masking and null values). The Table instance has a _masked attribute that specifies whether the user wants a Table with masked arrays, or with a null value. The function should take this into account. For example, in the built-in FITS reader, the table is populated with add_column in the following way:

    if self._masked:
        self.add_column(name, data, unit=columns.units[i], \
            mask=data==columns.nulls[i])
    else:
        self.add_column(name, data, unit=columns.units[i], \
            null=columns.nulls[i])
    

The reader/writer function can then fill the table by using the Table methods described in Full API for Table class (for a single table reader/writer) or Full API for TableSet class (for a table set reader/writer). In particular, a single table reader will likely contain calls to add_column, while a single table writer will likely contain references to the data attribute of Table.

Once a custom function is available, the user can register it using one of the four ATpy functions:

  • atpy.register_reader: Register a reader function for single tables
  • atpy.register_set_reader: Register a reader function for table sets
  • atpy.register_writer: Register a writer function for single tables
  • atpy.register_set_writer: Register a writer function for tables sets

The API for these functions is of the form (ttype, function, override=True/False), where ttype is the code name for the format (like the build-in fits, vo, ipac, or sql types), function is the actual function to use, and override allows the user to override existing definitions (for example to provide an improved ipac reader).

For example, if a function is defined for reading HDF5 tables, which we can call hdf5.read, then one would first need to register this function after importing atpy:

>>> import atpy
>>> atpy.register_reader('hdf5', hdf5.read)

This type can then be used when reading in a table:

>>> t = atpy.Table('mytable.hdf5', type='hdf5')

It is also possible to register extensions for a specific type using atpy.register_extensions. This function expects a table type and a list of file extensions to associate with it. For example, by setting:

>>> atpy.register_extensions('hdf5', ['hdf5', 'hdf'])

One can then read in an HDF5 table without specifying the type:

>>> t = atpy.Table('mytable.hdf5')

We encourage users to send us examples of reader/writer functions for various formats, and would be happy in future to include readers and writers for commonly used formats in ATpy.