Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15921

Excessive memory usage in MatchPessimisticB

    Details

    • Templates:
    • Team:
      Alert Production

      Description

      Hayden Smotherman, always an effective discoverer of bugs, ran into some decam fields that consistently crashed due to excessive memory usage, using >100GB on astro-lsst-01. We reran the failing chip in pdb and broke in when the memory usage suddenly began climbing, which corresponded to PessimisticPatternMatcherB._build_distances_and_angles(). This function creates several N x N arrays, where N is the number of objects in the reference catalog. In the failing case there were 37k reference objects (from Gaia) covering the 2k x 4k sensor, so the arrays total about 50 GB. We did not investigate the additional memory usage beyond this. We did double check that the ref cat sources sent to the matcher were correct and did not include any extra sources beyond the normal chip buffer area. By eye, the field appears dense but not nearly as crowded as something like the DECAPS area. Visit 439117 ccd 7 of the Allen NEO data was our test case.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ctslater Colin Slater
                Watchers:
                Andrew Connolly, Chris Morrison, Colin Slater, Eric Bellm, Hayden Smotherman, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel