Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ctrl_bps
-
Story Points:1
-
Epic Link:
-
Team:Data Facility
-
Urgent?:No
Description
Discussion on slack about better names for outCollection led to outputRun. https://lsstc.slack.com/archives/C01FBUGM2CV/p1633359652426800?thread_ts=1633359502.425800&cid=C01FBUGM2CV Change it in the default values and anywhere in bps code or documentation. Add code to check if outCollection is set, throw an error and tell user to "Rename outCollection to outputRun" (https://lsstc.slack.com/archives/C2JPT1KB7/p1633362884486400)

Attachments
Activity
Looks good to me. Can be merged to main after addressing Eli's and Sergey's comments (if any).
Mikolaj Kowalik
added a comment - Looks good to me. Can be merged to main after addressing Eli's and Sergey's comments (if any). I've fixed the documentation formatting that Eli pointed out. Sergey Padolski (or maybe Hsin-Fang Chiang as the previous editor of the PanDA YAMLs), would either of you like to check the PanDA YAML changes in the PR before I merge it?
I am checking
Sergey Padolski
added a comment - I am checking I confirm that updated version is submitting and running fine: https://panda-doma.cern.ch/tasks/?taskname=u_spadolski_pipelines_check_20211006t144306z*
if the SW image version is set to lsstsqre/centos:7-stack-lsst_distrib-w_2021_40.
May be it worth to update the default value in pipelines_check_idf.yaml.
The old version of YAML leads to the exception:
raise KeyError("outCollection is deprecated. Replace all outCollection references with outputRun.") |
KeyError: 'outCollection is deprecated. Replace all outCollection references with outputRun.' |
As expected.
Sergey Padolski
added a comment - I confirm that updated version is submitting and running fine: https://panda-doma.cern.ch/tasks/?taskname=u_spadolski_pipelines_check_20211006t144306z*
if the SW image version is set to lsstsqre/centos:7-stack-lsst_distrib-w_2021_40.
May be it worth to update the default value in pipelines_check_idf.yaml.
The old version of YAML leads to the exception:
raise KeyError( "outCollection is deprecated. Replace all outCollection references with outputRun." )
KeyError: 'outCollection is deprecated. Replace all outCollection references with outputRun.'
As expected. Sergey Padolski I need help finding that to fix as I cannot find any other instances of outCollection in that ticket branch:
$ git clone https://github.com/lsst/ctrl_bps |
Cloning into 'ctrl_bps'... |
remote: Enumerating objects: 2386, done. |
remote: Counting objects: 100% (1019/1019), done. |
remote: Compressing objects: 100% (396/396), done. |
remote: Total 2386 (delta 668), reused 860 (delta 569), pack-reused 1367 |
Receiving objects: 100% (2386/2386), 558.59 KiB | 0 bytes/s, done. |
Resolving deltas: 100% (1192/1192), done. |
|
|
$ cd ctrl_bps/
|
|
|
$ git checkout tickets/DM-32027 |
Branch tickets/DM-32027 set up to track remote branch tickets/DM-32027 from origin. |
Switched to a new branch 'tickets/DM-32027' |
|
|
$ grep -i -r outCol *
|
doc/changes/DM-32027.misc.rst:Change ``outCollection`` in submit YAML to ``outputRun``. |
python/lsst/ctrl/bps/drivers.py: if "outCollection" in config: |
python/lsst/ctrl/bps/drivers.py: raise KeyError("outCollection is deprecated. Replace all outCollection references with outputRun.") |
tests/test_drivers.py: def testDeprecatedOutCollection(self):
|
tests/test_drivers.py: with self.assertRaisesRegex(KeyError, "outCollection"): |
tests/test_drivers.py: _init_submission_driver({"payload": {"outCollection": "bad"}}) |
|
|
$ grep -i -r out_col *
|
$
|
Talked with Sergei. It's not a missed outCollection. It's that the weekly container should also be updated to match. Another request was to add a comment asking users to update it.
Sergey Padolski: Please review the changes in the PanDA files to make sure I haven't broken something.
Eli Rykoff: I added just a little bit to the quickstart.rst about the payload section. Please feel free to comment if you have time on that (or any part of this change).
And asking Mikolaj Kowalik to look over everything as the official review.