Module pqreader#

This module contains functions to load and decode files from PicoQuant hardware.

The main functions to decode PicoQuant files (PTU, HT3, PT3, T3R) are respectively:

These functions return the arrays timestamps (also called macro-time or timetag), detectors (or channel), nanotimes (also called micro-time or TCSPC time) and an additional metadata dict.

Other lower level functions are:

  • ptu_reader() to load metadata and raw t3 records from PTU files

  • ht3_reader() to load metadata and raw t3 records from HT3 files

  • pt3_reader() to load metadata and raw t3 records from PT3 files

  • process_pturecords() to decode t2 and t3 records based on spec.

  • process_t3records_t3rfile() to decode the t3 records for t3r files.

Deprecated functions:

  • process_t3records() to decode the t3 records and return timestamps (after overflow correction), detectors and TCSPC nanotimes.

  • process_t2records() to decode the t2 records and return timestamps (after overflow correction) and detectors.

The functions performing overflow/rollover correction can take advantage of numba, if installed, to significanly speed-up the processing.

List of functions#

High-level functions to load and decode several PicoQuant file formats:

phconvert.pqreader.load_ptu(filename, return_marker=False, ovcfunc='auto')#

Load data from a PicoQuant .ptu file.

Parameters:

  • filename (string) – the path of the PTU file to be loaded.

  • return_marker (bool) – whether or not to return information

  • markers (on) –

  • ovcfunc (string) – ‘auto’, ‘base’, ‘numba’, ‘loop’ ‘none’, or callable, which type of overflow function to use. ‘auto’ will use the fastest available function ‘base’ will use the numpy parrallelized version, ‘numba’ will used the numba accelerated version (requires numba to be installed), ‘loop’ will use the non-numba accelerated of loop based function, ‘none’ will not applay any sort of correction, thus allowing inspectection and direct manipulation of marker/ overflow photons after the fact.

Returns:

np.ndarray[np.uint64]

Array of timestamps of all photons/markers in data

detectorsnp.ndarray[np.uint8]

Array of detectors of all photons/markers in data

nanotimes np.ndarray[np.uint16]

Array of nanotimes of all photons/markers in data

metadict

Dictionary of tags from the beginning of ptu file. Tags with clear interpretation extracted to outer level of dictionary. dictionary with metadata containing the keys ‘timestamps_unit’, ‘nanotimes_unit’, ‘acquisition_duration’ and ‘tags’. The data in the PTU file header is returned as a dictionary of “tags”. Each item in the dictionary has ‘idx’, ‘type’, ‘value’ and ‘offset’ keys. Some tags also have a ‘data’ key. Use _ptu_print_tags() to print the tags as an easy-to-read table.

marker_idsnp.ndarray[np.uint8]

array of the ids of markers (i.e. records that are not photons)

Return type:

timestamps

phconvert.pqreader.load_ht3(filename, ovcfunc=None)#

Load data from a PicoQuant .ht3 file.

Parameters:

  • filename (string) – the path of the HT3 file to be loaded.

  • ovcfunc (function or None) – function to use for overflow/rollover correction of timestamps. If None, it defaults to the fastest available implementation for the current machine.

Returns:

A tuple of timestamps, detectors, nanotimes (integer arrays) and a dictionary with metadata containing at least the keys ‘timestamps_unit’ and ‘nanotimes_unit’.

phconvert.pqreader.load_pt3(filename, ovcfunc=None)#

Load data from a PicoQuant .pt3 file.

Parameters:

  • filename (string) – the path of the PT3 file to be loaded.

  • ovcfunc (function or None) – function to use for overflow/rollover correction of timestamps. If None, it defaults to the fastest available implementation for the current machine.

Returns:

A tuple of timestamps, detectors, nanotimes (integer arrays) and a dictionary with metadata containing at least the keys ‘timestamps_unit’ and ‘nanotimes_unit’.

phconvert.pqreader.load_phu(filename)#

Load data from a PicoQuant .phu file.

Parameters:

filename (string) – the path of the PHU file to be loaded.

Returns:

A tuple of histograms, histogram resolution, and tags. The latter is an dictionary of tags contained in the file header. Each item in the dictionary has ‘idx’, ‘type’, ‘value’ and ‘offset’ keys. Some tags also have a ‘data’ key. Use _ptu_print_tags() to print the tags as an easy-to-read table.

Low-level functions#

These functions are the building blocks for loading and decoding the different file formats:

phconvert.pqreader.ptu_reader(filename)#

Read the header and unprocessed t2 or t3 records from ptu file

Parameters:

filename (str) – Ptu file to read.

Raises:

ValueError – Proplem with specification definition, as defined in _ptu_rec_map, raise issue on github.com/Photon-HDF5/phconvert/issues .

Returns:

  • records (numpy.ndarray[uint32]) – Unprocessed ptu records as a numpy array.

  • spec (dict) – dictionary of spec tags infered from ptu version/record type.

  • tags (dict) – Header tags of ptu file.

phconvert.pqreader.ht3_reader(filename)#

Load raw t3 records and metadata from an HT3 file.

phconvert.pqreader.pt3_reader(filename)#

Load raw t3 records and metadata from a PT3 file.

phconvert.pqreader.process_t3records(t3records, time_bit=10, dtime_bit=15, ch_bit=6, special_bit=True, ovcfunc=None)#

Extract the different fields from the raw t3records array.

The input array of t3records is an array of “records” (a C struct). It packs all the information of each detected photons. This function decodes the different fields and returns 3 arrays containing the timestamps (i.e. macro-time or number of sync, few-ns resolution), the nanotimes (i.e. the micro-time or TCSPC time, ps resolution) and the detectors.

t3records have these fields (in little-endian order):

| Optional special bit | detectors | nanotimes | timestamps |
  MSB                                                   LSB

Bit allocation of these fields, starting from the MSB:

  • special bit: 1 bit if special_bit = True (default), else no special bit.

  • channel: default 6 bit, (argument ch_bit), detector or special marker

  • nanotimes: default 15 bit (argument dtime_bit), nanotimes (TCSPC time)

  • timestamps: default 10 bit, (argument time_bit), the timestamps (macro-time)

Timestamps: The returned timestamps are overflow-corrected, and therefore should be monotonically increasing. Each overflow event is marked by a special detector (or a special bit) and this information is used for the correction. These overflow “events” are not removed in the returned arrays resulting in spurious detectors. This choice has been made for safety (you can always go and check where there was an overflow) and for efficiency (removing a few elements requires allocating a new array that is potentially expensive for big data files). Under normal usage the additional detectors take negligible space and can be safely ignored.

Parameters:

  • t3records (array) – raw array of t3records as saved in the PicoQuant file.

  • time_bit (int) – number of bits in the t3record used for timestamps (or macro-time).

  • dtime_bit (int) – number of bits in the t3record used for the nanotime (TCSPC time or micro-time)

  • ch_bit (int) – number of bits in the t3record used for the detector number.

  • special_bit (bool) – if True the t3record contains a special bit for overflow correction. This special bit will become the MSB in the returned detectors array. If False, it assumes no special bit in the t3record.

  • ovcfunc (function or None) – function to perform overflow correction of timestamps. If None use the default function. The default function is the numba-accelerated version is numba is installed otherwise it is function using plain numpy.

Returns:

A 3-element tuple containing the following 1D arrays (all of the same length):

  • detectors (arrays of uint8): detector number. When special_bit = True the highest bit in detectors will be the special bit.

  • timestamps (array of int64): the macro-time (or number of sync) of each photons after overflow correction. Units are specified in the file header.

  • nanotimes (array of uint16): the micro-time (TCSPC time), i.e. the time lag between the photon detection and the previous laser sync. Units (i.e. the bin width) are specified in the file header.

phconvert.pqreader.process_t2records(t2records, time_bit=25, ch_bit=6, special_bit=True, ovcfunc=None)#

Extract the different fields from the raw t2records array.

The input array of t2records is an array of “records” (a C struct). It packs all the information of each detected photons. This function decodes the different fields and returns 2 arrays containing the timestamps (also called macro-time or timetag) and the detectors (or channel).

t2records have these fields (in little-endian order):

| Optional special bit | detectors |  timestamps |
  MSB                                        LSB

  • special bit: 1 bit if special_bit = True (default), else no special bit.

  • channel: default 6 bit, (argument ch_bit), detector or special marker

  • timestamps: default 25 bit, (argument time_bit), the timestamps (macro-time)

The returned timestamps are overflow-corrected, and therefore should be monotonically increasing. Each overflow event is marked by a special detector (or a special bit) and this information is used for the correction. These overflow “events” are not removed in the returned arrays resulting in spurious detectors. This choice has been made for safety (you can always go and check where there was an overflow) and for efficiency (removing a few elements requires allocating a new array that is potentially expensive for big data files). Under normal usage the additional detectors take negligible space and can be safely ignored.

Parameters:

  • t2records (array) – raw array of t2records as saved in the PicoQuant file.

  • time_bit (int) – number of bits in the t2record used for timestamps

  • ch_bit (int) – number of bits in the t2record used for the detector number.

  • special_bit (bool) – if True the t2record contains a special bit for overflow correction or external markers. This special bit will become the MSB in the returned detectors array. If False, it assumes no special bit in the t2record.

  • ovcfunc (function or None) – function to perform overflow correction of timestamps. If None use the default function. The default function is the numba-accelerated version if numba is installed otherwise it is function using plain numpy.

Returns:

A 2-element tuple containing the following 1D arrays (all of the same length):

  • timestamps (array of int64): the macro-time (or number of sync) of each photons after overflow correction. Units are specified in the file header.

  • detectors (arrays of uint8): detector number. When special_bit = True the highest bit in detectors will be the special bit.