Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30651

Switch DECam overscan correction to median per row

    XMLWordPrintable

    Details

      Description

      update Jan 2022

      Original ticket title: Investigate amp background level offsets in DECam postISRCCDs

      This ticket updates the DECam ISR overscan correction config parameter fitType='MEDIAN_PER_ROW'. A large body of evidence (see comments below) point towards this being a superior approach at handling overscan corrections with DECam. This also brings Science Pipelines data processing in-line with the DECam Community Pipeline, which also fits per row.

      Original ticket efforts by Kenneth Herner, original description below:

      If you scroll through here, the problem is immediately apparent

      https://lsst.ncsa.illinois.edu/~lskelvin/hits2014/

      Hat-tip to Lee Kelvin for noticing this problem and plotting the images. This feels very familiar to when ISR order of operations was inconsistent regarding when assembleCcd happens, but I don't remember the resolution.

      Possibly relevant and/or wrong: https://community.lsst.org/t/a-change-to-the-order-of-operations-for-isr/3060 

        Attachments

        1. merian_bias_01.png
          merian_bias_01.png
          821 kB
        2. merian_bias_35.png
          merian_bias_35.png
          909 kB
        3. merian_bias_45.png
          merian_bias_45.png
          800 kB
        4. merian_flat_01.png
          merian_flat_01.png
          2.97 MB
        5. merian_flat_05.png
          merian_flat_05.png
          2.60 MB
        6. merian_flat_25.png
          merian_flat_25.png
          3.07 MB
        7. merian_flat_35.png
          merian_flat_35.png
          4.36 MB
        8. merian_flat_45.png
          merian_flat_45.png
          2.66 MB
        9. merian_flat_55.png
          merian_flat_55.png
          3.99 MB

          Issue Links

            Activity

            Hide
            lskelvin Lee Kelvin added a comment - - edited

            Thanks Meredith Rawls and Kenneth Herner, that's good to know!

            Link to Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35639/pipeline

            Build products used for Jenkins are: lsst_distrib lsst_ci ap_verify_ci_hits2015. If there's another package that should be added to the build products to test this DECam config change, please do let me know!

            Update: I have subsequently learned that adding ap_verify_ci_hits2015 to the list of build products should not make a difference - the ap_verify datasets are not scons-buildable. With that said, it shouldn't have any negative impact either, so the above Jenkins is still valid.

            Show
            lskelvin Lee Kelvin added a comment - - edited Thanks Meredith Rawls and Kenneth Herner , that's good to know! Link to Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35639/pipeline Build products used for Jenkins are: lsst_distrib lsst_ci ap_verify_ci_hits2015 . If there's another package that should be added to the build products to test this DECam config change, please do let me know! Update: I have subsequently learned that adding ap_verify_ci_hits2015 to the list of build products should not make a difference - the ap_verify datasets are not scons-buildable. With that said, it shouldn't have any negative impact either, so the above Jenkins is still valid.
            Hide
            kherner Kenneth Herner added a comment -

            Changes look fine; Jenkins running.

            Show
            kherner Kenneth Herner added a comment - Changes look fine; Jenkins running.
            Hide
            kherner Kenneth Herner added a comment -

            I created DM-33126 to continue work on the still-open question of level settings in cpFlatNormalization.

            Show
            kherner Kenneth Herner added a comment - I created  DM-33126 to continue work on the still-open question of level settings in cpFlatNormalization.
            Hide
            lskelvin Lee Kelvin added a comment -

            Jenkins picked up a couple of hard coded unit test values which also needed to be updated. I've updated these, pushed to the PR, and re-ran Jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35645/pipeline/47

            Show
            lskelvin Lee Kelvin added a comment - Jenkins picked up a couple of hard coded unit test values which also needed to be updated. I've updated these, pushed to the PR, and re-ran Jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35645/pipeline/47
            Hide
            lskelvin Lee Kelvin added a comment -

            Thank you again to Kenneth Herner for your review, and Robert Lupton for taking a closer look under-the-hood. I rebased this ticket onto main, and re-ran Jenkins to make sure nothing fails. Branch merged and deleted, thanks again all.

            Show
            lskelvin Lee Kelvin added a comment - Thank you again to Kenneth Herner for your review, and Robert Lupton for taking a closer look under-the-hood. I rebased this ticket onto main, and re-ran Jenkins to make sure nothing fails . Branch merged and deleted, thanks again all.

              People

              Assignee:
              lskelvin Lee Kelvin
              Reporter:
              mrawls Meredith Rawls
              Reviewers:
              Kenneth Herner
              Watchers:
              Christopher Waters, Dino Bektesevic, Eric Bellm, Ian Sullivan, Kenneth Herner, Lee Kelvin, Meredith Rawls
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.