Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23782

Exploratory Work in Q1 2020 for Amp-to-Amp Offset Issue

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • None
    • None
    • 12
    • Data Release Production
    • No

    Description

      This ticket covers work performed on the known HSC amp-to-amp offset in Q1 2020. Initial reporting ticket is DM-20303. Prior exploratory work is covered in DM-22804, and completed exploratory conclusions are shown in DM-23794. Code implementation in the stack is ongoing (at time of writing) in DM-24389.

      This ticket covers further exploratory work analysing the amp-to-amp offset issue in HSC data, attempting to determine its origin, constructing analysis plots, and testing various fixes.

      Many of the plots in this ticket are produced in the attached deltaoverscan.ipynb jupyter notebook. A summary of what has been found:

      • tested for variability in amp-to-amp offset as a function of observed band via delta overscan plots - all bands appear to exhibit similar/identical offsets (i.e., issue is not band specific).
      • tested for incorrect gain being applied per amp (performed reruns with flatfielding turned off to generate signal-variance plots, essentially a PTC) - gain appears to be being applied correctly.
      • potential for gain differential across an amp. gaincheck plots below are color-coded according to distance from readout corner. The magnitude of this effect is not large enough to explain the amp offset issues we see here and (to first order) would be cancelled by the flat field.
      • amp offset level appears consistent at ~1 ADU - this manifests as a relatively larger effect (i.e., more noticeable) for shorter exposures.
      • it appears that prescriptive approaches towards fixing this issue may not be forthcoming (although work is ongoing looking at non-linearity effects). Empirical per-amp or per-CCD approaches may be more practical in the short term.

      It was decided to pursue two empirical per-amp/per-CCD based approaches towards fixing this issue: per-amp background subtraction (i.e., piston corrections) and per-CCD amp-edge pattern continuity corrections. Although much of this work was exploratory, all results relevant to that work have been moved to DM-23794.

      All source material has been stored on GitHub at: https://github.com/leeskelvin/amp2amp

      Attachments

        1. ampconfig.svg
          163 kB
        2. ampreadout.svg
          53 kB
        3. ccd043_g_amp1_vs_amp1+0.png
          ccd043_g_amp1_vs_amp1+0.png
          131 kB
        4. ccd043_grizy_amp1_vs_amp1+0.png
          ccd043_grizy_amp1_vs_amp1+0.png
          148 kB
        5. ccd043_grizy_amp1_vs_amp1+1.png
          ccd043_grizy_amp1_vs_amp1+1.png
          147 kB
        6. ccd043_grizy_amp1_vs_amp1+2.png
          ccd043_grizy_amp1_vs_amp1+2.png
          148 kB
        7. ccd043_grizy_amp1_vs_amp1+3.png
          ccd043_grizy_amp1_vs_amp1+3.png
          148 kB
        8. ccd043_grizy_amp1_vs_amp1+4.png
          ccd043_grizy_amp1_vs_amp1+4.png
          147 kB
        9. ccd043_grizy_amp2_vs_amp1+0.png
          ccd043_grizy_amp2_vs_amp1+0.png
          153 kB
        10. ccd043_grizy_amp3_vs_amp1+0.png
          ccd043_grizy_amp3_vs_amp1+0.png
          159 kB
        11. ccd043_grizy_amp4_vs_amp1+0.png
          ccd043_grizy_amp4_vs_amp1+0.png
          166 kB
        12. ccd043_i_amp1_vs_amp1+0.png
          ccd043_i_amp1_vs_amp1+0.png
          145 kB
        13. ccd043_r_amp1_vs_amp1+0.png
          ccd043_r_amp1_vs_amp1+0.png
          136 kB
        14. ccd043_y_amp1_vs_amp1+0.png
          ccd043_y_amp1_vs_amp1+0.png
          144 kB
        15. ccd043_z_amp1_vs_amp1+0.png
          ccd043_z_amp1_vs_amp1+0.png
          150 kB
        16. ccd43visit37542data.svg
          1.46 MB
        17. ccd43visit37542overscan.svg
          120 kB
        18. deltaoverscan.ipynb
          2.08 MB
        19. exposuretimes.svg
          31 kB
        20. gaincheck-linear.png
          gaincheck-linear.png
          86 kB
        21. gaincheck-log.png
          gaincheck-log.png
          92 kB
        22. obsdate.svg
          34 kB
        23. skybg.svg
          41 kB

        Issue Links

          Activity

            dtaranu Dan Taranu added a comment -

            The notebook does indeed look like Eli's overscan plots. Nice work.

            My only minor complaint/request is to please make the entire figure's background white (vs default transparent outside plot region) via one of the suggestions in this stackoverflow post. Otherwise, dark axis/tick labels are illegible on a dark background.

            dtaranu Dan Taranu added a comment - The notebook does indeed look like Eli's overscan plots. Nice work. My only minor complaint/request is to please make the entire figure's background white (vs default transparent outside plot region) via one of the suggestions in this stackoverflow post . Otherwise, dark axis/tick labels are illegible on a dark background.
            lskelvin Lee Kelvin added a comment -

            Thanks Dan, I've updated the notebook with the recommendation from the link you mentioned. Cheers.

            lskelvin Lee Kelvin added a comment - Thanks Dan, I've updated the notebook with the recommendation from the link you mentioned. Cheers.

            People

              lskelvin Lee Kelvin
              lskelvin Lee Kelvin
              Dan Taranu
              Dan Taranu, Lee Kelvin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.