Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15771

Remove unused remnants of pex_policy

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ctrl_orca
    • Labels:
      None
    • Story Points:
      1
    • Team:
      Data Facility

      Description

      Remove remnants of pex_policy, which was long ago shifted to used pex_config.

      The main things being gotten rid of are:

      Commands:

      killcondor.py - used the old style pex_policy job description to identify and kill jobs.
      writeNodeList.py - this was required in an early version of ctrl_orca to enumerate the nodes used (in pbs jobs, I believe).
      ProvenanceRecorder.py - This was used to make database records for policy files used in a run. This was in addition to the provenance information that used to be recorded via the provenance package, which is also obsolete, since it also used pex_policy. I don't think a replacement was ever made for that package.

      lsst/ctrl/orca:

      PolicyUtils.py - used to get all file names listed in a policy

      The ups/ctrl_orca.table needs to have pex_policy removed as a required setup.

        Attachments

          Issue Links

            Activity

            No builds found.
            spietrowicz Steve Pietrowicz created issue -
            spietrowicz Steve Pietrowicz made changes -
            Field Original Value New Value
            Risk Score 0
            tjenness Tim Jenness made changes -
            Watchers Steve Pietrowicz [ Steve Pietrowicz ] Jim Bosch, Steve Pietrowicz [ Jim Bosch, Steve Pietrowicz ]
            tjenness Tim Jenness made changes -
            Link This issue is triggered by RFC-525 [ RFC-525 ]
            spietrowicz Steve Pietrowicz made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            spietrowicz Steve Pietrowicz added a comment -

            Can you please check the version in this ticket out and do a run using this? There were several commands deleted (which don't need to be tested, obviously), a "required" package that was removed, an object (PolicyUtils.py, which wasn't in use either) deleted, and a command line option removed.

            Just try this as you've regularly used ctrl_orca, (through ctrl_execute), and that should be a good enough test.

            Show
            spietrowicz Steve Pietrowicz added a comment - Can you please check the version in this ticket out and do a run using this? There were several commands deleted (which don't need to be tested, obviously), a "required" package that was removed, an object (PolicyUtils.py, which wasn't in use either) deleted, and a command line option removed. Just try this as you've regularly used ctrl_orca, (through ctrl_execute), and that should be a good enough test.
            spietrowicz Steve Pietrowicz made changes -
            Reviewers Hsin-Fang Chiang [ hchiang2 ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            With the ticket branch, running this following command gave me an error:

            runOrca.py  -c "processCcd.py /datasets/hsc/repo  --output /scratch/hchiang2/orca-output" -i /home/hchiang2/orca/test.input -e $EUPS_PATH -p lsstvc -N <my-node-set> 
            

            Traceback (most recent call last):
              File "/home/hchiang2/stack/ctrl_orca/bin/orca.py", line 83, in <module>
                orca.repository = parser.opts.repository
            AttributeError: 'Values' object has no attribute 'repository'
            

            Running with the master seemed to go through. Any ideas?

            Show
            hchiang2 Hsin-Fang Chiang added a comment - With the ticket branch, running this following command gave me an error: runOrca.py -c "processCcd.py /datasets/hsc/repo --output /scratch/hchiang2/orca-output" -i /home/hchiang2/orca/test.input -e $EUPS_PATH -p lsstvc -N <my-node-set> Traceback (most recent call last): File "/home/hchiang2/stack/ctrl_orca/bin/orca.py", line 83, in <module> orca.repository = parser.opts.repository AttributeError: 'Values' object has no attribute 'repository' Running with the master seemed to go through. Any ideas?
            Hide
            spietrowicz Steve Pietrowicz added a comment -

            The reason this is happening it was looking for a default for the command line option that no longer exists. I've removed it now; the interfaces for the objects default this to None throughout the rest of the code, which is exactly how this was being invoked before (if you don't use the -r option, which runOrca.py does not). The rest of the code retains this, in case we ever want to put Config file directories in place at a later date.

            Please pull a new version of this branch, and try it again.

            Show
            spietrowicz Steve Pietrowicz added a comment - The reason this is happening it was looking for a default for the command line option that no longer exists. I've removed it now; the interfaces for the objects default this to None throughout the rest of the code, which is exactly how this was being invoked before (if you don't use the -r option, which runOrca.py does not). The rest of the code retains this, in case we ever want to put Config file directories in place at a later date. Please pull a new version of this branch, and try it again.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            The new version works fine!  

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The new version works fine!  
            hchiang2 Hsin-Fang Chiang made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            spietrowicz Steve Pietrowicz made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            spietrowicz Steve Pietrowicz made changes -
            Story Points 1
            swinbank John Swinbank made changes -
            Team Data Facility [ 12219 ]

              People

              Assignee:
              spietrowicz Steve Pietrowicz
              Reporter:
              spietrowicz Steve Pietrowicz
              Reviewers:
              Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, Jim Bosch, Steve Pietrowicz, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.