Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23139

Check whether the gen3 butler pre-filter template inputs and defer their loading if not

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_diffim
    • Labels:
      None
    • Templates:
    • Story Points:
      1
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      As of the currently available limited ci_hsc gen3 repo we have a very limited number of template images, however a production environment gen3 collection may contain many templates, covering large sky areas.

       

      As a follow up of the initial gen3 support DM-22541 in ImageDifferenceTask, check whether the number of template inputs can potentially be huge in a production environment or not because of gen3 automatic filtering on patch spatial information and science image coordinate matching.

       

      If all available template images are potentially loaded as task input, add support for deferred loading of the template coadd images to avoid unnecessary I/O and save resources. Load only at a later stage, if input belongs to a relevant skymap patch. If the coadd images loaded as inputs are pre-filtered at the middleware level then we perhaps do not need to implement any cautious approach in ImageDifferenceTask.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gkovacs Gabor Kovacs
                Reporter:
                gkovacs Gabor Kovacs
                Reviewers:
                John Swinbank
                Watchers:
                Gabor Kovacs, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel