Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35569

ip_isr doEmpiricalReadNoise fails if an amplifier is fully masked

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_isr
    • Labels:
      None
    • Story Points:
      2
    • Epic Link:
    • Team:
      Data Release Production

      Description

      If an amplifier is fully masked due to defects, it has no overscan result generated.  As this overscan result is used when the doEmpiricalReadNoise option is true, the task raises.  A fully masked amplifier has no need of empirical read noise, and so shouldn't need to raise.

      This likely should still raise if the overscan result is missing for another reason (doOverscan=False or something similar).

        Attachments

          Activity

          Hide
          mrawls Meredith Rawls added a comment -

          Thanks for filing this. I'll add that a single detector failing in this way also causes downstream ISR tasks for other detectors to fail, which seems like a bug. I would be OK with the task raising for just the fully masked detector, but not causing cascading failures for all the other detectors.

          Show
          mrawls Meredith Rawls added a comment - Thanks for filing this. I'll add that a single detector failing in this way also causes downstream ISR tasks for other detectors to fail, which seems like a bug. I would be OK with the task raising for just the fully masked detector, but not causing cascading failures for all the other detectors.
          Hide
          czw Christopher Waters added a comment -

          My expectation is that this should fail ISR for all bad detectors, and that all ISR tasks and cp_pipe tasks that do not involve that detector should continue without problems, so this is concerning.  I'm copying the example command from slack so that it doesn't get lost.

          pipetask run -d "exposure IN (288847, 288848, 288849, 288850, 288851, 288852, 288853, 288854, 288855, 288856, 288857) AND instrument='DECam'" -b /repo/main -i DECam/defaults,DECam/calib/DM-34710,u/mrawls/DM-34710-hits-crosstalk-sources -o u/put-your-own-repo-here-plz -p $CP_PIPE_DIR/pipelines/DarkEnergyCamera/cpFlat.yaml

          Show
          czw Christopher Waters added a comment - My expectation is that this should fail ISR for all bad detectors, and that all ISR tasks and cp_pipe tasks that do not involve that detector should continue without problems, so this is concerning.  I'm copying the example command from slack so that it doesn't get lost. pipetask run -d "exposure IN (288847, 288848, 288849, 288850, 288851, 288852, 288853, 288854, 288855, 288856, 288857) AND instrument='DECam'" -b /repo/main -i DECam/defaults,DECam/calib/DM- 34710 ,u/mrawls/DM- 34710 -hits-crosstalk-sources -o u/put-your-own-repo-here-plz -p $CP_PIPE_DIR/pipelines/DarkEnergyCamera/cpFlat.yaml
          Hide
          czw Christopher Waters added a comment -

          My test case for this was the command:

           pipetask run -d "exposure IN (288847) AND detector=2 AND instrument='DECam'" -b /sdf/group/rubin/repo/main -i DECam/defaults,DECam/calib/DM-34710,u/mrawls/DM-34710-hits-crosstalk-sources -o u/czw/DM-35569/test02 -p $CP_PIPE_DIR/pipelines/DarkEnergyCamera/cpFlat.yaml​

          which fails on main, but completes with the changes.

           

          Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/37275/pipeline

          Show
          czw Christopher Waters added a comment - My test case for this was the command: pipetask run -d "exposure IN (288847) AND detector=2 AND instrument='DECam'" -b /sdf/group/rubin/repo/main -i DECam/defaults,DECam/calib/DM- 34710 ,u/mrawls/DM- 34710 -hits-crosstalk-sources -o u/czw/DM- 35569 /test02 -p $CP_PIPE_DIR/pipelines/DarkEnergyCamera/cpFlat.yaml​ which fails on main, but completes with the changes.   Jenkins run:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/37275/pipeline
          Hide
          plazas Andrés Alejandro Plazas Malagón added a comment -

          Looks good to me. I just had a couple of comments about the unit test.

          Show
          plazas Andrés Alejandro Plazas Malagón added a comment - Looks good to me. I just had a couple of comments about the unit test.

            People

            Assignee:
            czw Christopher Waters
            Reporter:
            czw Christopher Waters
            Reviewers:
            Andrés Alejandro Plazas Malagón
            Watchers:
            Andrés Alejandro Plazas Malagón, Christopher Waters, Lee Kelvin, Meredith Rawls
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.