Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9926

Reduce memory footprint of meas_mosaic read

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_mosaic
    • Labels:
    • Templates:
    • Team:
      Data Release Production

      Description

      I am exhausting the memory using meas_mosaic during read. Usually we think of meas_mosaic as using too much memory during calculation, but I'm hitting the limit during read:

      Mosaic INFO: Reading catalogs ...
      Mosaic INFO: Use 28 cores for reading source catalog
      slurmstepd: error: Job 43298 exceeded memory limit (150023884 > 131072000), being killed
      slurmstepd: error: Exceeded job memory limit
      slurmstepd: error: *** JOB 43298 ON perseus-r5c4n2 CANCELLED AT 2017-03-23T22:32:38 ***
      

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  price Paul Price
                  Reporter:
                  price Paul Price
                  Reviewers:
                  Lauren MacArthur
                  Watchers:
                  Lauren MacArthur, Paul Price, Yusra AlSayyad
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel