pixie16.read.list_mode package

pixie16.read.list_mode.list_mode_data_reader module

Read list mode data

The file provides a python based list mode data parser.

The input can either come from a data stream or from one or several binary files (in case a large binary data set is split across multiple files).

We keep all the data in numpy arrays, whic are either memmaped or pre-allocated, to make data access fast.

exception pixie16.read.list_mode.list_mode_data_reader.EmptyError

Bases: Exception

We reached the end of the last file.

class pixie16.read.list_mode.list_mode_data_reader.Event(channel: int = -1, crate: int = -1, slot: int = -1, timestamp: float = -1.0, CFD_fraction: int = -1, energy: int = -1, trace: int = -1, CFD_error: int = -1, pileup: int = -1, trace_flag: int = -1, Esum_trailing: int = -1, Esum_leading: int = -1, Esum_gap: int = -1, baseline: float = -1.0, QDCSum0: int = -1, QDCSum1: int = -1, QDCSum2: int = -1, QDCSum3: int = -1, QDCSum4: int = -1, QDCSum5: int = -1, QDCSum6: int = -1, QDCSum7: int = -1, ext_timestamp: float = -1.0, chunk_timestamp: float = -1)

Bases: Struct

Data storage for a single even that has attribute access.

This is similar to a namedtuple, but the msgspec implemenation is faster.

CFD_error: int
CFD_fraction: int
Esum_gap: int
Esum_leading: int
Esum_trailing: int
QDCSum0: int
QDCSum1: int
QDCSum2: int
QDCSum3: int
QDCSum4: int
QDCSum5: int
QDCSum6: int
QDCSum7: int
baseline: float
channel: int
chunk_timestamp: float
crate: int
energy: int
ext_timestamp: float
pileup: int
slot: int
timestamp: float
to_list()
trace: int
trace_flag: int
class pixie16.read.list_mode.list_mode_data_reader.FileReader(files: Iterable[Path])

Bases: object

Read binary data from multiple files.

Use mmap for fast data access to binary data on disc. The data can be distributed across several files.

This class should be used as a context manager: >>> files = (“file1.bin”, “file2.bin”) >>> with FileReader(files) as r: >>> r.read(100)

or one needs to call self.open_files() and self.close_files() manually.

close_files()
open_files()
read(size: int) → bytes

Retrieve the next size bytes from the files and advance the file position.

Parameters:

size – How many bytes should be returned.

Raises:

EmptyError – When data of the requested size is not available in the data stream.

exception pixie16.read.list_mode.list_mode_data_reader.LeftoverBytesError

Bases: Exception

To be raised if a partial event is left in the byte stream/file.

class pixie16.read.list_mode.list_mode_data_reader.ListModeDataReader(reader: FileReader | StreamReader)

Bases: object

iterevents()

Will raise an exception to exit loop.

pop() → Event

Pop the top event from queue

pop_all()
class pixie16.read.list_mode.list_mode_data_reader.StreamReader(initial_buffer_size=25000000)

Bases: object

Handle streamed binary data.

Provides the same interface as FileReader and can be used as a direct replacement.

New data can be added using the put method.

Internally we store the data in a np.array of np.uint32.

We make the array large enough to hold the data and have the buffer grow and shrink as needed.

This is similar to a ring buffer, but we just allocate twice the buffer size memory and then keep track of the left and right position in the buffer and shift the data to the far left every now and then.

adjust_and_shift_buffer() → None

Create a new buffer and copy the old buffer to beginning.

This also takes care of changing the size of the buffer

put(input: numpy.ndarray) → None

Add data to the buffer.

Also increase/decrease the buffer size as needed.

read(size: int) → numpy.ndarray

Return size 32-bit words from the buffer as a numpy array.

pixie16.read.list_mode.list_mode_data_reader.events_from_files_generator(files)

Yields events from a list of binary files

pixie16.read.list_mode.list_mode_data_reader.read_list_mode_data(files, buffer_size=1000000000)

Loads a list of binary files into a pandas DataFrame

pixie16.read.list_mode.list_mode_data_reader.read_list_mode_data_as_events(files, buffer_size=1000000000, max_size=None)

Returns a list of at most max_size events from files

pixie16.read.list_mode.list_mode_data_reader.sort_events_by_channel(events, channel_list=None)

Sort events by channel.

Takes the output of read_list_mode_data_as_events and sorts them into a dictionary that has the channel number as key and as value a list of events (still including the channel number).

Parameters:

  • events – List of events as returned by read_list_mode_data_as_events

  • channel_list – Optional list of channels to include. This can help to reduce the amount of data.

pixie16.read.list_mode.list_mode_data_reader.to_tuple(obj)