Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-285

Revise front-end for warping

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      The warping implementation (currently MakeCoaddTempExpTask) relies on a SelectImagesTask to sort out the inputs. Unfortunately, this selection is performed inside the MakeCoaddTempExp.run method, which means MakeCoaddTempExpTask gets parallelised over patches rather than over individual warps (unless the user has some additional mechanics for setting the command-lines, essentially doing the selection themselves), which is inefficient. The SelectImagesTask hierarchy grew up as a MySQL database query before other things got tacked on to support butler-based selection, and could use simplification.

      I propose to move the selection of images into the TaskRunner.getTargetList method, which will allow for parallelisation over individual warps. I will also refactor the SelectImagesTask hierarchy and add a sub-class to perform selection of images using the spatial database introduced in DM-3472.

      There is a SelectSdssImagesTask in obs_sdss that I propose to drop because it relies on a particular database that I don't have access to and cannot recreate, and that code probably hasn't been used in a while. If someone objects to dropping it, I will attempt to modify it to conform with the SelectImagesTask changes, but testing and fixing it will have to be the responsibility of whoever wants to keep it.

      While I'm modifying the warping front-end, I also propose to move the config parameters that are operational in nature (e.g., doWrite, doOverwrite) out of the Config. Their presence in the Config causes trouble because there are legitimate reasons for changing their values between command-line incantations, but doing so requires adding --clobber-config which should only be required for changes of an algorithmic nature.

        Attachments

          Issue Links

            Activity

            price Paul Price created issue -
            price Paul Price made changes -
            Field Original Value New Value
            Link This issue blocks DM-5766 [ DM-5766 ]
            rowen Russell Owen made changes -
            Watchers Andy Salnikov, Hsin-Fang Chiang, Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Yusra AlSayyad [ Andy Salnikov, Hsin-Fang Chiang, Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Yusra AlSayyad ] Andy Salnikov, Hsin-Fang Chiang, Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Russell Owen, Yusra AlSayyad [ Andy Salnikov, Hsin-Fang Chiang, Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Russell Owen, Yusra AlSayyad ]
            price Paul Price made changes -
            Status Proposed [ 10805 ] Adopted [ 10806 ]
            price Paul Price made changes -
            Link This issue is triggering DM-5766 [ DM-5766 ]
            tjenness Tim Jenness made changes -
            Risk Score 0
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-15236 [ DM-15236 ]
            yusra Yusra AlSayyad made changes -
            Link This issue is triggering DM-17028 [ DM-17028 ]
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 19628 ]
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 19761 ]
            yusra Yusra AlSayyad made changes -
            Resolution Done [ 10000 ]
            Status Adopted [ 10806 ] Implemented [ 11105 ]

              People

              Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Watchers:
              Andy Salnikov, Hsin-Fang Chiang, Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Russell Owen, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.