Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26316

Allow bias generation to retain overscan signal

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: cp_pipe, ip_isr
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Team:
      Data Release Production

      Description

      ComCam data has patterns in the biases along the x-axis (perpendicular to parallel transfers).  The fitted overscan is therefore not removing all of the signal it should.  Reversing the application of the overscan and bias correction should allow these patterns to be removed by an initial bias pass, allowing the overscan correction to remove the row-by-row signals.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          czw Christopher Waters added a comment -

          This runs reasonably cleanly through `ci_cpp_gen2`, with altered `constructCalib.py`

          config.isr.doOverscan = False
          		 
          config.isr.doAssembleCcd = False
          		 
          config.visitKeys = ['expId']  # noqa: F821
          		 
          config.isr.doBiasBeforeOverscan = True

          And with `-c isr.doBiasBeforeOverscan=True` added to the `runIsr.py` call.

          Repeating on comCam data also appears successful:

           constructBias.py --batch-type none \
          		 
              /project/shared/comCam/_parent/ \
          		 
              --rerun /project/czw/overscan.20200814/biasGen/ \
          		 
              --id expId=2020072800038..2020072800044 detector=0 \
          		 
              --longlog -j 4 \
          		 
              -c isr.doCrosstalk=False \
          		 
              -C /home/czw/dev/DM-23302/ci_cpp_gen2/DATA-26316/constructCalib.py \
          		 
              -C /home/czw/dev/DM-23302/ci_cpp_gen2/DATA-26316/constructBias.pymkdir /project/czw/overscan.20200814/calibs/ingestCalibs.py /project/shared/comCam/_parent/ \
          		 
              /project/czw/overscan.20200814/biasGen/bias/*/*.fits \
          		 
              --validity 9999 \
          		 
              --calib /project/czw/overscan.20200814/calibs/ \
          		 
              --mode=linkrunIsr.py /project/shared/comCam/_parent/ \
          		 
              --rerun /project/czw/overscan.20200814/proc/ \
          		 
              --calib /project/czw/overscan.20200814/calibs/ \
          		 
              --id expId=2020072800038 detector=0 \
          		 
              -c isr.doBias=True \
          		 
              -c isr.doBiasBeforeOverscan=True \
          		 
              -c isr.doFlat=False \
          		 
              -c isr.doDark=False \
          		 
              -c isr.doCrosstalk=False \
          		 
              -c isr.doDefect=False \
          		 
              --clobber-versions --clobber-config

          Jenkins is running: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32465/pipeline

          Show
          czw Christopher Waters added a comment - This runs reasonably cleanly through `ci_cpp_gen2`, with altered `constructCalib.py` config.isr.doOverscan = False config.isr.doAssembleCcd = False config.visitKeys = [ 'expId' ] # noqa: F821 config.isr.doBiasBeforeOverscan = True And with `-c isr.doBiasBeforeOverscan=True` added to the `runIsr.py` call. Repeating on comCam data also appears successful: constructBias.py --batch-type none \ /project/shared/comCam/_parent/ \ --rerun /project/czw/overscan. 20200814 /biasGen/ \ --id expId= 2020072800038 .. 2020072800044 detector= 0 \ --longlog -j 4 \ -c isr.doCrosstalk=False \ -C /home/czw/dev/DM- 23302 /ci_cpp_gen2/DATA- 26316 /constructCalib.py \ -C /home/czw/dev/DM- 23302 /ci_cpp_gen2/DATA- 26316 /constructBias.pymkdir /project/czw/overscan. 20200814 /calibs/ingestCalibs.py /project/shared/comCam/_parent/ \ /project/czw/overscan. 20200814 /biasGen/bias/*/*.fits \ --validity 9999 \ --calib /project/czw/overscan. 20200814 /calibs/ \ --mode=linkrunIsr.py /project/shared/comCam/_parent/ \ --rerun /project/czw/overscan. 20200814 /proc/ \ --calib /project/czw/overscan. 20200814 /calibs/ \ --id expId= 2020072800038 detector= 0 \ -c isr.doBias=True \ -c isr.doBiasBeforeOverscan=True \ -c isr.doFlat=False \ -c isr.doDark=False \ -c isr.doCrosstalk=False \ -c isr.doDefect=False \ --clobber-versions --clobber-config Jenkins is running:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32465/pipeline
          Hide
          Permalink
          rhl Robert Lupton added a comment -

          Looks good (and it works at NCSA).  One comment on isr.doBiasBeforeOverscan on PR

          Show
          rhl Robert Lupton added a comment - Looks good (and it works at NCSA).  One comment on isr.doBiasBeforeOverscan on PR

            People

            Assignee:
            czw Christopher Waters
            Reporter:
            czw Christopher Waters
            Reviewers:
            Robert Lupton
            Watchers:
            Christopher Waters, Robert Lupton
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.