Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21212

Update existing cp_pipe tasks to pipelineTasks

    XMLWordPrintable

    Details

    • Story Points:
      40
    • Epic Link:
    • Sprint:
      DRP F19-6 (Nov)
    • Team:
      Data Release Production

      Description

      The new tasks for cp_pipe will need to be ready to work with the gen3 butler/as pipelineTasks in the near future, which likely requires some sort of plan.  This ticket is to remind everyone of that fact.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23576 Create ci_lsst variant for gen3

          • Invalid
          FF-depends on

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-20069 Finishing writing the basic version of the PTC task

          • Done
          is FF-depended by

          Improvement - An improvement or enhancement to an existing feature or task. DM-22302 cp_pipe combineHeaders does not construct date endpoints correctly

          • To Do

          Improvement - An improvement or enhancement to an existing feature or task. DM-22521 enable partial image reads in cp_pipe combine to avoid memory issues

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-22534 Implement cpSkyTask

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-22527 Migrate gen3 utility scripts to obs_base

          • Invalid
          is parent task of

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21786 Update PTC task to pipelineTasks (gen 3 butler)

          • Done
          mitigates

          Improvement - An improvement or enhancement to an existing feature or task. DM-10293 Bad exposure scales doesn't tell which exposures are bad

          • Invalid

            Activity

            Ascending order - Click to sort in descending order
            6 older comments
            Hide
            Permalink
            mfisherlevine Merlin Fisher-Levine added a comment -

            I can help discuss this after the AuxTel run if necessary, but I'm sure Chris doesn't really need me for that.

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - I can help discuss this after the AuxTel run if necessary, but I'm sure Chris doesn't really need me for that.
            Hide
            Permalink
            czw Christopher Waters added a comment -

            I believe I have sorted out the flat issue, and will be getting this back to active review once I have the code cleaned/commits rebased/documentation updated/etc.

            A notebook containing the validation of the new gen3 flats and a quick discussion of changes is available here: https://github.com/czwa/notebooks/blob/master/20200527-lsst_gen3_flat_scaling/flatscale.20200527.ipynb

            Show
            czw Christopher Waters added a comment - I believe I have sorted out the flat issue, and will be getting this back to active review once I have the code cleaned/commits rebased/documentation updated/etc. A notebook containing the validation of the new gen3 flats and a quick discussion of changes is available here:  https://github.com/czwa/notebooks/blob/master/20200527-lsst_gen3_flat_scaling/flatscale.20200527.ipynb
            Hide
            Permalink
            czw Christopher Waters added a comment -

            I would like to re-request review on this ticket.  The algorithm now matches the gen2 implementation scaling between detectors.  The code should now be better documented and clearer to follow than it was before.

            Show
            czw Christopher Waters added a comment - I would like to re-request review on this ticket.  The algorithm now matches the gen2 implementation scaling between detectors.  The code should now be better documented and clearer to follow than it was before.
            Hide
            Permalink
            price Paul Price added a comment -

            I think the only comments I have are minor, so go ahead and merge after cleaning up.

            The commit history is not orthogonal, and I would urge you to put some effort into cleaning this up before merging. Squash commits dealing with the same new code, and split commits that contain multiple functional changes. It might be easiest to simply toss all your commits and start crafting new ones from your changes. Let me know if you need help on doing any of that.

            Show
            price Paul Price added a comment - I think the only comments I have are minor, so go ahead and merge after cleaning up. The commit history is not orthogonal, and I would urge you to put some effort into cleaning this up before merging. Squash commits dealing with the same new code, and split commits that contain multiple functional changes. It might be easiest to simply toss all your commits and start crafting new ones from your changes. Let me know if you need help on doing any of that.
            Hide
            Permalink
            czw Christopher Waters added a comment -

            https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32080/pipeline/

            Show
            czw Christopher Waters added a comment - https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32080/pipeline/

              People

              Assignee:
              czw Christopher Waters
              Reporter:
              czw Christopher Waters
              Reviewers:
              Nate Lust, Paul Price
              Watchers:
              Andrés Alejandro Plazas Malagón, Christopher Waters, Merlin Fisher-Levine, Paul Price, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.