pottu::Pipeline Class Reference

Pipeline capsules the pottu-stage pipeline loop and allows running it multithreaded. More...

#include <Pipeline.hpp>

Collaboration diagram for pottu::Pipeline:

Classes
struct	BuiltPipeline

Public Member Functions
void	setSource (DataSource *stage) noexcept
	Sets the data source. More...
void	setTimeStamper (StageRawToStampedRaw *stage) noexcept
	Sets the time stamper. More...
void	setDetectorEventGenerator (StageStampedRawToDetectorEvent *stage) noexcept
	Sets the detector event generator. More...
void	add (StageDetectorEvent *stage) noexcept
	Adds a normal processing stage. More...
void	add (StageDetectorEventToDetectorEvent *stage) noexcept
	Adds a normal processing stage which has an internal detector event buffer. More...
void	add (StageDetectorEventFinal *stage) noexcept
	Adds a processing stage which consumes the data. More...
void	run (std::size_t num_threads=1)
	Runs the defined pipeline until the data source has no more data. More...
void	_buildSingleThreadedPipeline ()
void	_buildMultiThreadedPipeline (std::size_t num_lines)
void	_runSingleThreaded ()

Public Attributes
DataSource *	_source {nullptr}
StageRawToStampedRaw *	_stamper {nullptr}
StageStampedRawToDetectorEvent *	_deteventgen {nullptr}
std::vector< std::variant< StageDetectorEvent *, StageDetectorEventToDetectorEvent *, StageDetectorEventFinal * > >	_stagesRest
std::unique_ptr< BuiltPipeline >	_builtPipeline

Detailed Description

Pipeline capsules the pottu-stage pipeline loop and allows running it multithreaded.

This pipeline object is a quite simple. It is intended to be used in a typical sort which includes pipeline like:

• FileReader or any other data source derived from DataSource. Use method setSource() to set.
• TimeStamper or any other StageRawToStampedRaw object. Use method setTimeStamper() to set.
• DetectorEventGenerator or any other StageStampedRawToDetectorEvent object. Use method setDetectorEventGenerator() to set.
• Any number of
  • StageDetectorEvent objects or
  • StageDetectorEventToDetectorEvent objects. Use method add() to set these.

The order of processing StageDetectorEvent objects and StageDetectorEventToDetectorEvent objects is the same as that those were added into the pipeline. The very first stages are fixed and can be set in any moment when the pipeline is not running.

Missing of any of the first three stages will throw an exception if tried to run.

To run a pipeline multithreaded, define the preprocessor name POTTU_ENABLE_MULTITHREADING=1. One can do this for example by adding target_compile_definitions( yoursort PRIVATE POTTU_ENABLE_MULTITHREADING=1 [OTHER DEFINES...] ) in the CMakeLists.txt file.

Member Function Documentation

add() [1/3]

void pottu::Pipeline::add ( StageDetectorEvent *  stage )

inlinenoexcept

Adds a normal processing stage.

Currently there is no interface to modify the array of added stages except adding more of stages. If really required then modify _stagesRest vector directly and reset the unique_ptr _builtPipeline.

add() [2/3]

void pottu::Pipeline::add ( StageDetectorEventFinal *  stage )

inlinenoexcept

Adds a processing stage which consumes the data.

If multiple final stages are added then the data vector is copied for these.

add() [3/3]

void pottu::Pipeline::add ( StageDetectorEventToDetectorEvent *  stage )

inlinenoexcept

Adds a normal processing stage which has an internal detector event buffer.

Currently there is no interface to modify the array of added stages except adding more of stages. If really required then modify _stagesRest vector directly and reset the unique_ptr _builtPipeline.

run()

void pottu::Pipeline::run ( std::size_t  num_threads = 1 )

inline

Runs the defined pipeline until the data source has no more data.

Parameters

num_threads

Number of threads to use to run the pipeline. Special value zero tries to detect the maximum number of concurrent threads in the system. If fails in that then uses 4 threads.

The actual pipeline and the required buffers to carry the data between stages are created when this function is called first time or if new stages have been added or if this method is called with different thread number.

If more data has been added (for example new data files) then run() can be called again.

setDetectorEventGenerator()

void pottu::Pipeline::setDetectorEventGenerator ( StageStampedRawToDetectorEvent *  stage )

inlinenoexcept

Sets the detector event generator.

Parameters

stage

Typically pointer to a DetectorEventGenerator object.

This overwrites the earlier set stage.

setSource()

void pottu::Pipeline::setSource ( DataSource *  stage )

inlinenoexcept

Sets the data source.

Parameters

stage

Typically pointer to a FileReader object.

This overwrites the earlier stage.

setTimeStamper()

void pottu::Pipeline::setTimeStamper ( StageRawToStampedRaw *  stage )

inlinenoexcept

Sets the time stamper.

Parameters

stage

Typically pointer to a TimeStamper object.

This overwrites the earlier set stage.

---

The documentation for this class was generated from the following file:

• /home/jhsaren/git/analysis/pottu/pottu/Pipeline.hpp