Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31839

Reduce memory usage in MatchPessemisticB

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_astrom
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      AP F21-4 (September), AP F21-5 (October)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Breaking up DM-29515 into smaller tickets. This ticket is for implementing the first part of the plan laid out in DM-31312.

        Attachments

          Issue Links

            Activity

            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Sure. I already check the results with the CI dataset and found they came out the same. Even going to far as to make sure the arrays in the input later functions were the same.

            I'll show how much the memory usage is reduced by by checking the difference in how much is stored in the look up arrays in `self` before and after the change. Sound good?

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Sure. I already check the results with the CI dataset and found they came out the same. Even going to far as to make sure the arrays in the input later functions were the same. I'll show how much the memory usage is reduced by by checking the difference in how much is stored in the look up arrays in `self` before and after the change. Sound good?
            Hide
            lauren Lauren MacArthur added a comment -

            Sounds perfect, thanks!

            Show
            lauren Lauren MacArthur added a comment - Sounds perfect, thanks!
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Ran through the HiTS and HSC dataset from the ap_verify ci datasets. For HiTS the static of memory stored in lookup tables. is reduced by a factor of ~2.5. For HSC the reduction is a factor of 2.375

            Overall the memory stored in self of the class is reduced by a factor of ~2.5 when including all variables stored in the class's `self`.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Ran through the HiTS and HSC dataset from the ap_verify ci datasets. For HiTS the static of memory stored in lookup tables. is reduced by a factor of ~2.5. For HSC the reduction is a factor of 2.375 Overall the memory stored in self of the class is reduced by a factor of ~2.5 when including all variables stored in the class's `self`.
            Hide
            lauren Lauren MacArthur added a comment -

            Thanks for the numbers...an improvement, indeed.  I've left a few minor comments on the PR, but otherwise, LGTM.

            Show
            lauren Lauren MacArthur added a comment - Thanks for the numbers...an improvement, indeed.  I've left a few minor comments on the PR, but otherwise, LGTM.
            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Jenkins after review: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35117/pipeline

              People

              Assignee:
              cmorrison Chris Morrison [X] (Inactive)
              Reporter:
              cmorrison Chris Morrison [X] (Inactive)
              Reviewers:
              Lauren MacArthur
              Watchers:
              Chris Morrison [X] (Inactive), Eric Bellm, Ian Sullivan, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.