pemtk.fit._parallel

PEMtk wrappers for parallel processing for fitting

Currently wraps xyzpy’s run_combos for parallel functionality and data handling. See the xyzpy docs for details.

Other methods to be implemented.

13/09/21 v1 Basic wrapper with XYZPY implemented

Module Contents

Functions

multiFit(self[, nRange, parallel, num_workers, ...])

Basic wrapper for pemtk.fitClass.fit() for multiprocess execution.

fitPara([data, n, randomizeParams, seedParams])

Wrap self.fit() for XYZPY runner.
pemtk.fit._parallel.multiFit(self, nRange=[0, 10], parallel=True, num_workers=None, randomizeParams=True, seedParams=None)[source]

Basic wrapper for pemtk.fitClass.fit() for multiprocess execution.

Run a batch of fits in parallel, and return results to main class structure.

Currently wraps xyzpy’s run_combos for parallel functionality and data handling. See the xyzpy docs for details.

Note: full set of results currently returned as an Xarray DataSet, then sorted back to base class as self.data[n] (integer n). In future may just want to use Xarray return directly?

Parameters:

  • nRange (list) – Fit indexers. Set [nStart, nStop], full run will be set as list(range(nRange[0],nRange[1])). TODO: more flexibility here, and auto.

  • parallel (bool, default = True) – Run fit jobs in parallel? Note - in testing this seemed to be ignored?

  • num_workers (int, default = None) – Number of cores to use if parallel job. Currently set to default to ~90% of mp.cpu_count()

  • randomizeParams (bool, default = True) – Randomize seed parameters per fit?

  • seedParams (int, default = None) – NOT IMPLEMENTED, but will provide an option to seed fits from a previous result.

pemtk.fit._parallel.fitPara(data=None, n=None, randomizeParams=True, seedParams=None)[source]

Wrap self.fit() for XYZPY runner.