Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25210

Fix psfex regression in w18

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: psfex
    • Labels:
      None
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      DRP metrics regressions in w19 turned out to be the result of multiple issues. After fixing the overscan issue, we noticed the the metrics still did not return to their w14 baseline.

      The lingering problem was isolated to an interaction between psfex and the new conda env. I don't understand exactly why, but stepping through pdb with a w18 and w17 showed that despite giving psfex the same inputs we were getting different answers here:

      > /software/lsstsw/stack_20200504/stack/miniconda3-4.7.12-2deae7a/Linux64/meas_extensions_psfex/19.0.0-
      2-gd82b0d5+3/python/lsst/meas/extensions/psfex/psfexPsfDeterminer.py(385)determinePsf()
      -> psfex.makeit(fields, sets)
      

      And inspection of fields yielded segfaults in w18.

      *To reproduce:*

        singleFrameDriver.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB/ --rerun private/yusra/RC2/w22reg/hybrid --id visit=1208 ccd=89 --cores 1 --config processCcd.isr.doWrite=True
      

      For iter 1 you should see singleFrameDriver.processCcd.charImage.measurePsf INFO: PSF determination using 65/70 stars as a success marker.

      [yusra@lsst-dev01 w22reg]$ diff w17_ccd89.log  w18_ccd89.log  | grep PSF
      < singleFrameDriver.processCcd.charImage.measurePsf INFO: PSF determination using 65/70 stars.
      < singleFrameDriver.processCcd.charImage INFO: iter 1; PSF sigma=1.23, dimensions=(41, 41); median background=2387.68
      > singleFrameDriver.processCcd.charImage.measurePsf INFO: PSF determination using 45/70 stars.
      > singleFrameDriver.processCcd.charImage INFO: iter 1; PSF sigma=1.11, dimensions=(41, 41); median background=2387.68
      < singleFrameDriver.processCcd.charImage.measurePsf INFO: PSF determination using 64/74 stars.
      < singleFrameDriver.processCcd.charImage INFO: iter 2; PSF sigma=1.23, dimensions=(41, 41); median background=2387.75
      > singleFrameDriver.processCcd.charImage.measurePsf INFO: PSF determination using 50/74 stars.
      > singleFrameDriver.processCcd.charImage INFO: iter 2; PSF sigma=1.11, dimensions=(41, 41); median background=2387.82
      

      For more info see team debugging June 2 2020 here: https://lsstc.slack.com/archives/C4JQP6FRS/p1591113943137300

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              ktl Kian-Tat Lim
              Reporter:
              yusra Yusra AlSayyad
              Reviewers:
              Tim Jenness
              Watchers:
              John Swinbank, Kian-Tat Lim, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: