PEMtk fitting setup & batch run demo

01/09/22

Outline of this notebook:

Use setup_fit_demo.py script to load data and setup fitting environment.
Run a batch of fits…
- 1. run batch
  - Serial with self.fit() method.
  - Parallel fitting routine with self.multiFit() method. This uses XYZpy library for quick parallelization.
- 1. load a batch
Exploring the results… see the “analysis” notebook for more.

Versions

06/06/21 v1
22/08/22 v2
01/09/22 v2b

v2b updated version including:

Data IO with self.writeFitData() method.
fitting now supports additional backends, these are set in self.fitOpts. For a list of available backends run self.backends(). Currently this implements support for AF or MF fitting routines.

Setup

[1]:

# Import & set paths
import pemtk
from pemtk.fit.fitClass import pemtkFit

from pathlib import Path

# Path for demo script
demoPath = Path(pemtk.__file__).parent.parent/Path('demos','fitting')

# Run demo script to configure workspace
%run {demoPath/"setup_fit_demo.py"}

*** ePSproc installation not found, setting for local copy.

* Setting plotter defaults with epsproc.basicPlotters.setPlotters(). Run directly to modify, or change options in local env.

* Set Holoviews with bokeh.

C:\Users\femtolab\.conda\envs\ePSdev\lib\site-packages\xyzpy\plot\xyz_cmaps.py:6: MatplotlibDeprecationWarning:
The revcmap function was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use Colormap.reversed() instead.
  return LinearSegmentedColormap(name, cm.revcmap(cmap._segmentdata))

*** Setting up demo fitting workspace and main `data` class object...
For more details see https://pemtk.readthedocs.io/en/latest/fitting/PEMtk_fitting_basic_demo_030621-full.html
To use local source code, pass the parent path to this script at run time, e.g. "setup_fit_demo ~/github"


* Loading packages...

* Set Holoviews with bokeh.

* Loading demo matrix element data from D:\code\github\ePSproc\data\photoionization\n2_multiorb...

*** Job subset details
Key: subset
No 'job' info set for self.data[subset].

*** Job orb6 details
Key: orb6
Dir D:\code\github\ePSproc\data\photoionization\n2_multiorb, 1 file(s).
{   'batch': 'ePS n2, batch n2_1pu_0.1-50.1eV, orbital A2',
    'event': ' N2 A-state (1piu-1)',
    'orbE': -17.096913836366,
    'orbLabel': '1piu-1'}

*** Job orb5 details
Key: orb5
Dir D:\code\github\ePSproc\data\photoionization\n2_multiorb, 1 file(s).
{   'batch': 'ePS n2, batch n2_3sg_0.1-50.1eV, orbital A2',
    'event': ' N2 X-state (3sg-1)',
    'orbE': -17.341816310545997,
    'orbLabel': '3sg-1'}


* Loading demo ADM data from D:\code\github\ePSproc\data\alignment\N2_ADM_VM_290816.mat...

* Subselecting data...
Subselected from dataset 'orb5', dataType 'matE': 36 from 11016 points (0.33%)
Subselected from dataset 'pol', dataType 'pol': 1 from 3 points (33.33%)
Subselected from dataset 'ADM', dataType 'ADM': 52 from 14764 points (0.35%)

* Calculating AF-BLMs...
Subselected from dataset 'sim', dataType 'AFBLM': 195 from 195 points (100.00%)

*Setting  up fit parameters (with constraints)...
Set 6 complex matrix elements to 12 fitting params, see self.params for details.

C:\Users\femtolab\.conda\envs\ePSdev\lib\site-packages\pandas\core\generic.py:3939: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._update_inplace(obj)

Auto-setting parameters.

name	value	initial value	min	max	vary	expression
m_PU_SG_PU_1_n1_1_1	1.78461575	1.784615753610107	1.0000e-04	5.00000000	True
m_PU_SG_PU_1_1_n1_1	1.78461575	1.784615753610107	1.0000e-04	5.00000000	False	m_PU_SG_PU_1_n1_1_1
m_PU_SG_PU_3_n1_1_1	0.80290495	0.802904951323892	1.0000e-04	5.00000000	True
m_PU_SG_PU_3_1_n1_1	0.80290495	0.802904951323892	1.0000e-04	5.00000000	False	m_PU_SG_PU_3_n1_1_1
m_SU_SG_SU_1_0_0_1	2.68606212	2.686062120382649	1.0000e-04	5.00000000	True
m_SU_SG_SU_3_0_0_1	1.10915311	1.109153108617096	1.0000e-04	5.00000000	True
p_PU_SG_PU_1_n1_1_1	-0.86104140	-0.8610414024232179	-3.14159265	3.14159265	False
p_PU_SG_PU_1_1_n1_1	-0.86104140	-0.8610414024232179	-3.14159265	3.14159265	False	p_PU_SG_PU_1_n1_1_1
p_PU_SG_PU_3_n1_1_1	-3.12044446	-3.1204444620772467	-3.14159265	3.14159265	True
p_PU_SG_PU_3_1_n1_1	-3.12044446	-3.1204444620772467	-3.14159265	3.14159265	False	p_PU_SG_PU_3_n1_1_1
p_SU_SG_SU_1_0_0_1	2.61122920	2.611229196458127	-3.14159265	3.14159265	True
p_SU_SG_SU_3_0_0_1	-0.07867828	-0.07867827542158025	-3.14159265	3.14159265	True



*** Setup demo fitting workspace OK.

Run fits

For this notebook, we’ll either (a) run a batch of fits or (b) load sample data.

With the current codebase, running multiple fits will default to using the same basis set, and output results sequentially to the main self.data dictionary.

Update 20/08/22: fitting now supports additional backends, these are set in self.fitOpts. For a list of available backends run self.backends(). Currently this implements support for AF or MF fitting routines.

Benchmarks for an AMD Threadripper 2950X (16 core) system.

(a) Run a batch

(1) serial execution

Either:

Manually with a loop.
With self.multiFit() method, although this is optimised for parallel execution (see below).

[2]:

import time

start = time.time()

# Maual execution
for n in range(0,100):
    data.randomizeParams()
    data.fit()

end = time.time()
print((end - start)/60)

# Or run with self.multiFit(parallel = False)
# data.multiFit(nRange = [0,100], parallel = False)

30.742977670828502

[3]:

# We now have 100 fit results
data.data.keys()

[3]:

dict_keys(['subset', 'orb6', 'orb5', 'ADM', 'pol', 'sim', 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99])

[4]:

# Quick data dump
# TODO: better save routine (json/h5).
# Now wrapped in self.writeFitData(), see https://github.com/phockett/PEMtk/issues/6 and docs at https://epsproc.readthedocs.io/en/dev/dataStructures/ePSproc_dataStructures_IO_demo_280622.html

# import pickle
# with open('dataDump_100fitTests_130621.pickle', 'wb') as handle:
#     pickle.dump(data.data, handle, protocol=pickle.HIGHEST_PROTOCOL)

# Data IO with self.writeFitData()
# This will default to the working dir and set a data-stamped file name if nothing is passed.
data.writeFitData()

[4]:

WindowsPath('D:/code/python/jupyterNotebooks/dev/PEMtk/fitting/dataDump_010922_16-53-04.pickle')

(2) parallel execution

Updated version including parallel fitting routine with self.multiFit() method.

This currently uses the XYZpy library for quick parallelization, although there is some additional setup overhead in the currently implementation due to class init per fit batch. The default aims to set ~90% CPU usage, based on core-count.

[5]:

data.multiFit(nRange = [0,100])

  0%|                                                                                          | 0/100 [00:00<?, ?it/s]C:\Users\femtolab\.conda\envs\ePSdev\lib\site-packages\joblib\externals\loky\process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.
  "timeout or by a memory leak.", UserWarning
100%|################################################################################| 100/100 [11:09<00:00,  6.70s/it]

(b) Load a batch of fit runs

Load sample data for analysis instead of running fits. Note this can be run minimally without the full setup routines above, using the commented-out cell below to init a blank object.

(The demo file(s) are available in demos/fitting.)

[6]:

# If running from scratch, create a blank object first
# # Init blank object
from pemtk.fit.fitClass import pemtkFit
data = pemtkFit()

[7]:

# Load sample dataset
# Full path to the file may be required here, in demos/fitting
# import pickle

# with open('dataDump_100fitTests_10t_randPhase_130621.pickle', 'rb') as handle:
#     data.data = pickle.load(handle)

# Data IO with self.writeFitData()
# This will default to the working dir
data.loadFitData('dataDump_100fitTests_10t_randPhase_130621.pickle')

Read data from D:\code\python\jupyterNotebooks\dev\PEMtk\fitting\dataDump_100fitTests_10t_randPhase_130621.pickle with pickle.

[8]:

data.data.keys()

[8]:

dict_keys(['subset', 'orb6', 'orb5', 'ADM', 'pol', 'sim', 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99])

Exploring a fit result

Each result contains a set of fit results:

[9]:

nFit = 11
data.data[nFit].keys()

[9]:

dict_keys(['AFBLM', 'residual', 'results'])

Here ‘results’ is an lmFit object, containing various outputs, including the final paramter set and fit statistics, which can be inspected directly. (See the basic demo notebook for more.)

[10]:

data.data[nFit]['results']

[10]:

Fit Statistics

fitting method	leastsq
# function evals	493
# data points	195
# variables	8
chi-square	1.3656e-04
reduced chi-square	7.3029e-07
Akaike info crit.	-2747.48342
Bayesian info crit.	-2721.29942

Variables

name	value	standard error	relative error	initial value	min	max	vary	expression
m_PU_SG_PU_1_n1_1	1.58289364	0.00420180	(0.27%)	0.9969358394309723	1.0000e-04	5.00000000	False	m_PU_SG_PU_1_1_n1
m_PU_SG_PU_1_1_n1	1.58289364	0.00420180	(0.27%)	0.9969358394309723	1.0000e-04	5.00000000	True
m_PU_SG_PU_3_n1_1	1.15075298	0.00591749	(0.51%)	0.6501789011457593	1.0000e-04	5.00000000	False	m_PU_SG_PU_3_1_n1
m_PU_SG_PU_3_1_n1	1.15075298	0.00591749	(0.51%)	0.6501789011457593	1.0000e-04	5.00000000	True
m_SU_SG_SU_1_0_0	2.71014401	0.00245323	(0.09%)	0.5256290247418078	1.0000e-04	5.00000000	True
m_SU_SG_SU_3_0_0	1.04870406	0.00589304	(0.56%)	0.3431948628999326	1.0000e-04	5.00000000	True
p_PU_SG_PU_1_n1_1	1.22200072	28873.8615	(2362835.06%)	0.22504503901184436	-3.14159265	3.14159265	False	p_PU_SG_PU_1_1_n1
p_PU_SG_PU_1_1_n1	1.22200072	28873.8615	(2362835.07%)	0.22504503901184436	-3.14159265	3.14159265	True
p_PU_SG_PU_3_n1_1	-1.09376359	28873.8574	(2639862.74%)	0.2384618778656229	-3.14159265	3.14159265	False	p_PU_SG_PU_3_1_n1
p_PU_SG_PU_3_1_n1	-1.09376359	28873.8574	(2639862.73%)	0.2384618778656229	-3.14159265	3.14159265	True
p_SU_SG_SU_1_0_0	-2.28911439	28873.8581	(1261354.97%)	0.2118753318675165	-3.14159265	3.14159265	True
p_SU_SG_SU_3_0_0	0.90233526	28873.8543	(3199903.15%)	0.5407367643463352	-3.14159265	3.14159265	True

Correlations (unreported correlations are < 0.100)

p_PU_SG_PU_3_1_n1	p_SU_SG_SU_1_0_0	1.0000
p_PU_SG_PU_1_1_n1	p_PU_SG_PU_3_1_n1	1.0000
p_PU_SG_PU_1_1_n1	p_SU_SG_SU_1_0_0	1.0000
p_PU_SG_PU_1_1_n1	p_SU_SG_SU_3_0_0	1.0000
p_PU_SG_PU_3_1_n1	p_SU_SG_SU_3_0_0	1.0000
p_SU_SG_SU_1_0_0	p_SU_SG_SU_3_0_0	1.0000
m_PU_SG_PU_1_1_n1	m_PU_SG_PU_3_1_n1	-0.9519
m_SU_SG_SU_1_0_0	m_SU_SG_SU_3_0_0	-0.8360
m_PU_SG_PU_1_1_n1	p_SU_SG_SU_3_0_0	-0.5513
m_PU_SG_PU_1_1_n1	p_PU_SG_PU_1_1_n1	-0.5513
m_PU_SG_PU_1_1_n1	p_PU_SG_PU_3_1_n1	-0.5513
m_PU_SG_PU_1_1_n1	p_SU_SG_SU_1_0_0	-0.5513
m_PU_SG_PU_3_1_n1	p_SU_SG_SU_3_0_0	0.5195
m_PU_SG_PU_3_1_n1	p_PU_SG_PU_1_1_n1	0.5195
m_PU_SG_PU_3_1_n1	p_PU_SG_PU_3_1_n1	0.5195
m_PU_SG_PU_3_1_n1	p_SU_SG_SU_1_0_0	0.5195
m_SU_SG_SU_3_0_0	p_SU_SG_SU_1_0_0	-0.3544
m_SU_SG_SU_3_0_0	p_PU_SG_PU_1_1_n1	-0.3544
m_SU_SG_SU_3_0_0	p_PU_SG_PU_3_1_n1	-0.3544
m_SU_SG_SU_3_0_0	p_SU_SG_SU_3_0_0	-0.3544
m_SU_SG_SU_1_0_0	p_PU_SG_PU_1_1_n1	0.3413
m_SU_SG_SU_1_0_0	p_SU_SG_SU_1_0_0	0.3413
m_SU_SG_SU_1_0_0	p_PU_SG_PU_3_1_n1	0.3413
m_SU_SG_SU_1_0_0	p_SU_SG_SU_3_0_0	0.3413
m_PU_SG_PU_3_1_n1	m_SU_SG_SU_1_0_0	-0.1329

The best fit results are set in an Xarray, keyed by AFBLM.

[11]:

data.data[nFit]['AFBLM']

For further analysis & batch results, see the “analysis” notebook.

Versions

[12]:

import scooby
scooby.Report(additional=['epsproc', 'pemtk', 'xarray', 'jupyter'])

[12]:

Thu Sep 01 17:04:18 2022 Eastern Daylight Time
OS	Windows	CPU(s)	32	Machine	AMD64
Architecture	64bit	RAM	63.9 GB	Environment	Jupyter
Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)]
epsproc	1.3.2-dev	pemtk	0.0.1	xarray	0.15.0
jupyter	Version unknown	numpy	1.18.1	scipy	1.3.0
IPython	7.12.0	matplotlib	3.3.1	scooby	0.5.6
Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191125 for Intel(R) 64 architecture applications

[13]:

# Check current Git commit for local ePSproc version
from pathlib import Path
!git -C {Path(ep.__file__).parent} branch
!git -C {Path(ep.__file__).parent} log --format="%H" -n 1

* dev
  master
  numba-tests
815a839f3c8c4995532d6b5f87997b8ba2bb12eb

[14]:

# Check current remote commits
!git ls-remote --heads https://github.com/phockett/ePSproc
# !git ls-remote --heads git://github.com/phockett/epsman

92c661789a7d2927f2b53d7266f57de70b3834fa        refs/heads/dependabot/pip/notes/envs/envs-versioned/mistune-2.0.3
fe1e9540c7b91fe571f60562acd31d8e489d491e        refs/heads/dependabot/pip/notes/envs/envs-versioned/nbconvert-6.5.1
815a839f3c8c4995532d6b5f87997b8ba2bb12eb        refs/heads/dev
1c0b8fd409648f07c85f4f20628b5ea7627e0c4e        refs/heads/master
69cd89ce5bc0ad6d465a4bd8df6fba15d3fd1aee        refs/heads/numba-tests
ea30878c842f09d525fbf39fa269fa2302a13b57        refs/heads/revert-9-master

[15]:

# Check current Git commit for local PEMtk version
import pemtk
from pathlib import Path
!git -C {Path(pemtk.__file__).parent} branch
!git -C {Path(pemtk.__file__).parent} log --format="%H" -n 1

  master
* mfFittingDev
c1be71d7cbb7d000370d9d28dbeaaca48b9e8bb0

[16]:

# Check current remote commits
!git ls-remote --heads https://github.com/phockett/PEMtk
# !git ls-remote --heads git://github.com/phockett/epsman

1059c4ebbd637467c0b6c37a95a51b684be0d298        refs/heads/master
7eda41b192a0b9a214bef2925becc99f716bf59a        refs/heads/mfFittingDev

[ ]: