Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30651

Switch DECam overscan correction to median per row

    XMLWordPrintable

    Details

      Description

      update Jan 2022

      Original ticket title: Investigate amp background level offsets in DECam postISRCCDs

      This ticket updates the DECam ISR overscan correction config parameter fitType='MEDIAN_PER_ROW'. A large body of evidence (see comments below) point towards this being a superior approach at handling overscan corrections with DECam. This also brings Science Pipelines data processing in-line with the DECam Community Pipeline, which also fits per row.

      Original ticket efforts by Kenneth Herner, original description below:

      If you scroll through here, the problem is immediately apparent

      https://lsst.ncsa.illinois.edu/~lskelvin/hits2014/

      Hat-tip to Lee Kelvin for noticing this problem and plotting the images. This feels very familiar to when ISR order of operations was inconsistent regarding when assembleCcd happens, but I don't remember the resolution.

      Possibly relevant and/or wrong: https://community.lsst.org/t/a-change-to-the-order-of-operations-for-isr/3060 

        Attachments

        1. merian_bias_01.png
          821 kB
          Lee Kelvin
        2. merian_bias_35.png
          909 kB
          Lee Kelvin
        3. merian_bias_45.png
          800 kB
          Lee Kelvin
        4. merian_flat_01.png
          2.97 MB
          Lee Kelvin
        5. merian_flat_05.png
          2.60 MB
          Lee Kelvin
        6. merian_flat_25.png
          3.07 MB
          Lee Kelvin
        7. merian_flat_35.png
          4.36 MB
          Lee Kelvin
        8. merian_flat_45.png
          2.66 MB
          Lee Kelvin
        9. merian_flat_55.png
          3.99 MB
          Lee Kelvin

          Issue Links

            Activity

            Hide
            kherner Kenneth Herner added a comment -

            The Saha Bulge calibs I created for PREOPS-597 also had overscan set to median-per-row, so they won't be affected. Otherwise the datasets I'm using for the diffim settings comparisons are actually HSC images. From my perspective, fire away.

             

            General comment: After discussing with Meredith Rawls, Lee Kelvin, and Ian Sullivan, the plan is to split this ticket up. The existing ticket will cover only this change to the overscan setting, and I will open a new ticket for doing the amp vs. detector investigation and assign that to myself.

            Show
            kherner Kenneth Herner added a comment - The Saha Bulge calibs I created for PREOPS-597 also had overscan set to median-per-row, so they won't be affected. Otherwise the datasets I'm using for the diffim settings comparisons are actually HSC images. From my perspective, fire away.   General comment: After discussing with Meredith Rawls , Lee Kelvin , and Ian Sullivan , the plan is to split this ticket up. The existing ticket will cover only this change to the overscan setting, and I will open a new ticket for doing the amp vs. detector investigation and assign that to myself.
            Hide
            lskelvin Lee Kelvin added a comment - - edited

            Thanks Meredith Rawls and Kenneth Herner, that's good to know!

            Link to Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35639/pipeline

            Build products used for Jenkins are: lsst_distrib lsst_ci ap_verify_ci_hits2015. If there's another package that should be added to the build products to test this DECam config change, please do let me know!

            Update: I have subsequently learned that adding ap_verify_ci_hits2015 to the list of build products should not make a difference - the ap_verify datasets are not scons-buildable. With that said, it shouldn't have any negative impact either, so the above Jenkins is still valid.

            Show
            lskelvin Lee Kelvin added a comment - - edited Thanks Meredith Rawls and Kenneth Herner , that's good to know! Link to Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35639/pipeline Build products used for Jenkins are: lsst_distrib lsst_ci ap_verify_ci_hits2015 . If there's another package that should be added to the build products to test this DECam config change, please do let me know! Update: I have subsequently learned that adding ap_verify_ci_hits2015 to the list of build products should not make a difference - the ap_verify datasets are not scons-buildable. With that said, it shouldn't have any negative impact either, so the above Jenkins is still valid.
            Hide
            kherner Kenneth Herner added a comment -

            Changes look fine; Jenkins running.

            Show
            kherner Kenneth Herner added a comment - Changes look fine; Jenkins running.
            Hide
            kherner Kenneth Herner added a comment -

            I created DM-33126 to continue work on the still-open question of level settings in cpFlatNormalization.

            Show
            kherner Kenneth Herner added a comment - I created  DM-33126 to continue work on the still-open question of level settings in cpFlatNormalization.
            Hide
            lskelvin Lee Kelvin added a comment -

            Jenkins picked up a couple of hard coded unit test values which also needed to be updated. I've updated these, pushed to the PR, and re-ran Jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35645/pipeline/47

            Show
            lskelvin Lee Kelvin added a comment - Jenkins picked up a couple of hard coded unit test values which also needed to be updated. I've updated these, pushed to the PR, and re-ran Jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35645/pipeline/47

              People

              Assignee:
              lskelvin Lee Kelvin
              Reporter:
              mrawls Meredith Rawls
              Reviewers:
              Kenneth Herner
              Watchers:
              Christopher Waters, Dino Bektesevic, Eric Bellm, Ian Sullivan, Kenneth Herner, Lee Kelvin, Meredith Rawls
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:

                  CI Builds

                  No builds found.