Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Data Release Production
-
Labels:None
-
Team:Data Facility
-
Urgent?:No
Description
The BPS htcondor plugin will require the htcondor python bindings to function at S3DF (and elsewhere) . From examining
https://anaconda.org/conda-forge/htcondor
https://anaconda.org/conda-forge/htcondor/files
it appears that a suitable distribution for Python 3.10 should be available.
If this looks appropriate, requesting that the htcondor python bindings be included in 'the stack'.
Attachments
Activity
I do not think that the execute nodes actually need access to the htcondor python bindings from conda-forge.
If the bindings are in the base rubin-env environment, that would help in some broad cases, like users running at other sites conveniently having them in place, or more exotic scaling scenarios where parts of workflow run on a compute node. But for the initial base case of running at s3df, I do not think access on the execute nodes is needed.
Greg Daues
added a comment - I do not think that the execute nodes actually need access to the htcondor python bindings from conda-forge.
If the bindings are in the base rubin-env environment, that would help in some broad cases, like users running at other sites conveniently having them in place, or more exotic scaling scenarios where parts of workflow run on a compute node. But for the initial base case of running at s3df, I do not think access on the execute nodes is needed. This seems to be done. Perhaps we can mark this ticket as done/complete.
[daues@sdfrome001 ~]$ source /sdf/group/rubin/sw/loadLSST.bash |
(lsst-scipipe-5.1.0) [daues@sdfrome001 ~]$ setup lsst_distrib -t w_2023_05 |
(lsst-scipipe-5.1.0) [daues@sdfrome001 ~]$ python3 |
Python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:26:04) [GCC 10.4.0] on linux |
Type "help", "copyright", "credits" or "license" for more information. |
>>> import htcondor |
>>>
|
Greg Daues
added a comment - This seems to be done. Perhaps we can mark this ticket as done/complete.
[daues @sdfrome001 ~]$ source /sdf/group/rubin/sw/loadLSST.bash
(lsst-scipipe- 5.1 . 0 ) [daues @sdfrome001 ~]$ setup lsst_distrib -t w_2023_05
(lsst-scipipe- 5.1 . 0 ) [daues @sdfrome001 ~]$ python3
Python 3.10 . 8 | packaged by conda-forge | (main, Nov 22 2022 , 08 : 26 : 04 ) [GCC 10.4 . 0 ] on linux
Type "help" , "copyright" , "credits" or "license" for more information.
>>> import htcondor
>>>
This should really be an RFC, but I will try to get it into rubin-env 5.0.0 anyway.
My understanding is that this needs to be part of the user's conda environment for both submission and execution and as a result needs to be in the base rubin-env environment rather than rubin-env-rsp or rubin-env-developer. Is that correct?