ctrl_bps_parsl is a BPS plugin implemented with Parsl. Originally written by James Chiang for DESC as gen3_workflow, it has been refactored for general use, and will be adopted by the Subaru Prime Focus Spectrograph (PFS) and the Merian survey. For more details, see the package README.
I propose that ctrl_bps_parsl be included in the lsst_bps_plugins metapackage, which will make it readily available to everyone who installs lsst_distrib. The advantages are:
1. ctrl_bps_parsl is currently the only plugin that supports typical HPC clusters with Slurm/PBS/LSF schedulers. Users of the Gen2 packages ctrl_pool and pipe_drivers will find ctrl_bps_parsl to be the appropriate tool under Gen3.
2. ctrl_bps_parsl is currently the only plugin that supports operating BPS on the local machine. This allows developers to test BPS workflows without access to a cluster, and it opens the possibility of developing a simple continuous integration (CI) test suite for BPS.
One concern has to do with support, since there are currently no LSST-DM developers with first-hand experience using it. I intend to support ctrl_bps_parsl since we need it for PFS. Beyond that, I believe the code is written to LSST-DM standards (including GHA checks with black and mypy, among others), and LSST-DM developers should have little difficulty finding their way around. DESC's Computing Infrastructure team includes a Parsl developer in case knowledge from or fixes to Parsl are required.
Adding ctrl_bps_parsl to lsst_bps_plugins could be done either with or without adding the parsl package to the rubin-env conda environment. parsl is available from conda-forge. Given the possibility of using ctrl_bps_parsl for CI, I think it would be best to go ahead and add it now. This would make ctrl_bps_parsl useful without any additional effort, and therefore allow use of BPS to the regular LSST-DM user without any additional setup.