Skip to content

Configuration: Data Handlers

BinaryFileWritingDataHandler

This Data Handler class writes serialized data into a file. The name of the file is generated by joining a string identifying the rank of the process writing the file, with a number that increases progressively with each file written by the same process. A suffix is appended to the resulting string. Optionally, a prefix can be prepended to the file name.

Configuration Parameters for Hdf5Serializer

  • file_prefix (str): This parameter is optional. It defines a prefix that is prepended to the name of each file written by the Data Handler. The default value is "" (an empty string, no prefix). Example: processed_

  • file_suffix (str): This parameter is optional. It defines the suffix (extension) for the files written by the Data Handler. The default value is "h5" (i.e: an ".h5" extension is used for the files). Example: .data

  • write_directory (str): This parameter is optional. It defines the directory where the Data Handler writes the files. The default value of this parameter is the current working directory.

BinaryFileWritingDataHandler

This Data Handler sends data through a socket, to be consumed by an external application. The handler can create sockets using either the NNG or the ZMQ library. Currently, Sockets created by this handler are blocking: if nothing receives the data on the other side of the socket, the data flow through the handler stops.

Configuration Parameters for Hdf5Serializer

  • urls (list of str): This parameter defines the list of URIS where the Data Handler opens its sockets. Each URI must be in a format compatible with the ZMQ and NNG libraries, depending on which one is used by the handler. Additionally, this parameter must always have the form of a list, even if just one URI is provided

  • role (str): This parameter specifies if the Data Handler should create a socket or connect to an already existing one. The parameter can only have two values: server, (to have the handler create the socket) or client (to connect to an existing socket. The server role corresponds to sockets performing a bind operation with the ZMQ library, or a listen operation with the NNG library. The client role corresponds instead to socket performing a connect operation with ZMQ or a dial operation with NNG.

  • library (str): This parameter dictates which library the handler uses to manage the socket. The parameter can only have the values zmq or nng to use the ZMQ or NNG libraries respectively

  • socket_type (str): This parameter determines the type of socket that the handler creates. Currently only the PUSH socket type is supported and this parameter can only take the value push