Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30046

Investigate memory usage of ForcedPhotCcd

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_base
    • Labels:

      Description

      DM-29776 revealed that the Gen3 version of ForcedPhotCcdTask uses much more memory than ForcedPhotCoadd, which doesn't make sense given that (for HSC) detectors are half the size of patches and we don't need HeavyFootprints or replace-with-noise in ForcedPhotCcd.

      Make sure:

      • we're using deferred loading instead of grabbing all overlapping patch catalogs at once;
      • we're not not loading HeavyFootprints unnecessarily;
      • we're not doing anything else stupidly memory-hungry.

      If this ticket fixes the problem, please make sure the new requestMemory values are communicated to operators and the rest of the team.

        Attachments

          Issue Links

            Activity

            Hide
            mschmitz Morgan Schmitz added a comment - - edited

            Thanks Arun!

            This merge actually only addresses the first of Jim's three points in the ticket description; if this does not solve the issue, I'll file another follow-up ticket.

            (Changes done at pair coding with Jim Bosch and Nate Lust)

            Show
            mschmitz Morgan Schmitz added a comment - - edited Thanks Arun! This merge actually only addresses the first of Jim's three points in the ticket description; if this does not solve the issue, I'll file another follow-up ticket. (Changes done at pair coding with Jim Bosch and Nate Lust )
            Hide
            kannawad Arun Kannawadi added a comment -

            Looks good. It might be safer to run ci_hsc also on Jenkins. Not sure if Jim/Nate said it was unnecessary.

            Show
            kannawad Arun Kannawadi added a comment - Looks good. It might be safer to run ci_hsc also on Jenkins. Not sure if Jim/Nate said it was unnecessary.

              People

              Assignee:
              mschmitz Morgan Schmitz
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Arun Kannawadi
              Watchers:
              Arun Kannawadi, Jim Bosch, Morgan Schmitz
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.