Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12450

Implement RFC-407: improve interface for clobbering vs. reusing outputs

    Details

    • Story Points:
      2
    • Sprint:
      DRP F17-5, DRP F17-6
    • Team:
      Data Release Production

      Description

      See RFC for description.

      IMO, this has bitten too many DRP team members recently to just put it on the backlog; I think it's really costing us time and money. Even if the RFC needs to stew for a while before merging, I'd like to get at least a partial implementation on a branch while it's fresh in my mind.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            An implementation of the proposal currently up on RFC-407 has been pushed on the branch for this ticket. It has not been fully tested. This ticket will stay "In Progress" until the RFC closes, and it's possible I'll add an attempt to address problem (3) on the RFC as well.

            Show
            jbosch Jim Bosch added a comment - An implementation of the proposal currently up on RFC-407 has been pushed on the branch for this ticket. It has not been fully tested. This ticket will stay "In Progress" until the RFC closes, and it's possible I'll add an attempt to address problem (3) on the RFC as well.
            Hide
            jbosch Jim Bosch added a comment -

            Changes are ready for review. I'd like to split it up as follows:

            Paul Price, while I don't want to burden you with a formal review while traveling, since you're the main user of the clobber/overwrite options, I want to make sure you have a chance to weigh in here as well. I'm most concerned that the new behavior (described on RFC-407) is acceptable to you; you're welcome but not expected to verify that the code here accomplishes that. If you do want to take a look but won't be able to do so immediately, just let me know so I can hold off on the merge until you have a chance.

            Finally, Nate Lust should note that I've added a doDetection config option to multibandDriver that is now required to enable re-running detection. I believe this will only be needed when adding fake sources at the coadd stage, and it should be included in any instructions for doing so.

            I will be writing a community post announcing these changes and describing their usage once we're ready to merge. I think it's premature to do that now.

            Show
            jbosch Jim Bosch added a comment - Changes are ready for review. I'd like to split it up as follows: Nate Pease : daf_persistence Hsin-Fang Chiang : pipe_base, pipe_drivers, pipe_tasks (recommend approaching them in that order) Paul Price , while I don't want to burden you with a formal review while traveling, since you're the main user of the clobber/overwrite options, I want to make sure you have a chance to weigh in here as well. I'm most concerned that the new behavior (described on RFC-407 ) is acceptable to you; you're welcome but not expected to verify that the code here accomplishes that. If you do want to take a look but won't be able to do so immediately, just let me know so I can hold off on the merge until you have a chance. Finally, Nate Lust should note that I've added a doDetection config option to multibandDriver that is now required to enable re-running detection. I believe this will only be needed when adding fake sources at the coadd stage, and it should be included in any instructions for doing so. I will be writing a community post announcing these changes and describing their usage once we're ready to merge. I think it's premature to do that now.
            Hide
            npease Nate Pease added a comment -

            daf_persistence looks fine to me.

            Show
            npease Nate Pease added a comment - daf_persistence looks fine to me.
            Hide
            price Paul Price added a comment -

            Sounds good to me. I'm glad you've found a way to reduce these long-annoying problems.

            Show
            price Paul Price added a comment - Sounds good to me. I'm glad you've found a way to reduce these long-annoying problems.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Looks good to me. Two comments on the PRs.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Looks good to me. Two comments on the PRs.
            Hide
            jbosch Jim Bosch added a comment -

            Merged to master.

            Show
            jbosch Jim Bosch added a comment - Merged to master.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                Hsin-Fang Chiang, Nate Pease
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, Nate Pease, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: