Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27159

measurePhotonTransferCurve.py fails on LSSTCam

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: cp_pipe
    • Labels:
      None

      Description

      I ran a PTC task on BOT run 12606, the first time we have the full LSSTCam.  Six CCDs failed to return results.  I am tracking down the reasons. Four CCDs failed for the same reason, R01_S20 (det 6), R11_S21 (det 43), R23_S10 (det 102), R43_S11 (det 184).  The error given in 'RuntimeError: dictionary changed size during iteration'.  below is a command line that reproduces the error, and the error string.

      Command line:

      measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_12606 --rerun /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_12606 --id detector=6 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216^3020100800218^3020100800219^3020100800221^3020100800222     -c maxMeanSignal=120000 ptcFitType=EXPAPPROXIMATION doPhotodiode=False -j 1

      Error:

      Traceback (most recent call last):
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/cp_pipe/20.0.0-18-g541646f+c4459f5943/bin/measurePhotonTransferCurve.py", line 27, in <module>
          MeasurePhotonTransferCurveTask.parseAndRun()
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_base/20.0.0-19-gcdd82e7+6f5ab6e0f6/python/lsst/pipe/base/cmdLineTask.py", line 610, in parseAndRun
          resultList = taskRunner.run(parsedCmd)
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_base/20.0.0-19-gcdd82e7+6f5ab6e0f6/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
          resultList = list(mapFunc(self, targetList))
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/20.0.0-30-g1d38f5b5+80082c6d28/python/lsst/pipe/tasks/getRepositoryData.py", line 57, in __call__
          result = task.runDataRef(dataRefList)
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_base/20.0.0-19-gcdd82e7+6f5ab6e0f6/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/cp_pipe/20.0.0-18-g541646f+c4459f5943/python/lsst/cp/pipe/ptc.py", line 254, in runDataRef
          expPairs = self.makePairs(dataRefList)
        File "/software/lsstsw/stack_20200922/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/cp_pipe/20.0.0-18-g541646f+c4459f5943/python/lsst/cp/pipe/ptc.py", line 411, in makePairs
          for (key, value) in flatPairs.items():
      RuntimeError: dictionary changed size during iteration
      

        Attachments

          Issue Links

            Activity

            Hide
            cslage Craig Lage added a comment -

            One thing I've noticed here is that the CCDs that failed for this issue appeared to find an odd number of postISRCCD images.  So it appears that it must have found one member of the flat pair, but not the other.  But this shouldn't cause it to fail, it should continue and generate the PTC with the flat pairs that it finds.

            Show
            cslage Craig Lage added a comment - One thing I've noticed here is that the CCDs that failed for this issue appeared to find an odd number of postISRCCD images.  So it appears that it must have found one member of the flat pair, but not the other.  But this shouldn't cause it to fail, it should continue and generate the PTC with the flat pairs that it finds.
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Looks good, though I found one other bug outside the PR, which is being added on to this now.

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Looks good, though I found one other bug outside the PR, which is being added on to this now.
            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32846/

              People

              Assignee:
              plazas Andrés Alejandro Plazas Malagón
              Reporter:
              cslage Craig Lage
              Reviewers:
              Merlin Fisher-Levine
              Watchers:
              Andrés Alejandro Plazas Malagón, Christopher Waters, Craig Lage, Merlin Fisher-Levine
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: