Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12979

y-band background subtraction

    XMLWordPrintable

    Details

    • Team:
      External

      Description

      HSC has had some features in the y-band images due to encoders in the instrument. Sogo Mineo has written some code to remove them (report and link to the code here), and we would like to include it in the pipeline.

        Attachments

          Issue Links

            Activity

            No builds found.
            price Paul Price created issue -
            Hide
            price Paul Price added a comment -

            price@pap-laptop:~/LSST/obs_subaru (tickets/DM-12979=) $ git sub
            commit 689fea06b5e8b492e798fcdfc67de1dd2bde4547 (HEAD -> tickets/DM-12979, origin/tickets/DM-12979)
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Dec 12 13:55:12 2017 -0500
             
                add y-band stray light removal
                
                LEDs in an encoder in HSC are producing stray light on the detectors,
                producing the 'Eye of Y-band' feature. It can be removed by
                subtracting open-shutter darks. However, because the pattern of stray
                light varies with rotator angle, many dark exposures are required.
                To reduce the data volume for the darks, the images have been
                compressed using wavelets.
                
                The code here (provided by Sogo Mineo of NAOJ and only cleaned up
                a bit) retrieves the appropriate dark, uncompresses it and uses it
                to remove the stray light from an exposure.
                
                Some code here (provided by Satoshi Kawanomoto) may not meet coding
                standards, but we are accepting it as legacy code. Unfortunately, no
                tests are available for this code yet.
             
             policy/HscMapper.yaml                              |   3 +
             python/lsst/obs/subaru/isr.py                      |  13 +
             python/lsst/obs/subaru/strayLight/__init__.py      |   2 +
             python/lsst/obs/subaru/strayLight/rotatorAngle.py  | 268 +++++++++++++
             .../obs/subaru/strayLight/waveletCompression.py    | 427 +++++++++++++++++++++
             python/lsst/obs/subaru/strayLight/yStrayLight.py   | 220 +++++++++++
             6 files changed, 933 insertions(+)
             
             
            price@pap-laptop:~/LSST/pipe_drivers (tickets/DM-12979=) $ git sub
            commit 388cf09cfb4c36866cb6754651e0587d9790b1e2 (HEAD -> tickets/DM-12979, origin/tickets/DM-12979)
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Wed Dec 13 10:34:39 2017 -0500
             
                add visualizeVisit
                
                This provides a facility to produce a binned image of the entire focal
                plane (butler product visit_camera).
             
             bin.src/visualizeVisit.py                  |   3 +
             python/lsst/pipe/drivers/skyCorrection.py  |  43 +-------
             python/lsst/pipe/drivers/visualizeVisit.py | 160 +++++++++++++++++++++++++++++
             3 files changed, 167 insertions(+), 39 deletions(-)
            

            Show
            price Paul Price added a comment - price@pap-laptop:~/LSST/obs_subaru (tickets/DM-12979=) $ git sub commit 689fea06b5e8b492e798fcdfc67de1dd2bde4547 (HEAD -> tickets/DM-12979, origin/tickets/DM-12979) Author: Paul Price <price@astro.princeton.edu> Date: Tue Dec 12 13:55:12 2017 -0500   add y-band stray light removal LEDs in an encoder in HSC are producing stray light on the detectors, producing the 'Eye of Y-band' feature. It can be removed by subtracting open-shutter darks. However, because the pattern of stray light varies with rotator angle, many dark exposures are required. To reduce the data volume for the darks, the images have been compressed using wavelets. The code here (provided by Sogo Mineo of NAOJ and only cleaned up a bit) retrieves the appropriate dark, uncompresses it and uses it to remove the stray light from an exposure. Some code here (provided by Satoshi Kawanomoto) may not meet coding standards, but we are accepting it as legacy code. Unfortunately, no tests are available for this code yet.   policy/HscMapper.yaml | 3 + python/lsst/obs/subaru/isr.py | 13 + python/lsst/obs/subaru/strayLight/__init__.py | 2 + python/lsst/obs/subaru/strayLight/rotatorAngle.py | 268 +++++++++++++ .../obs/subaru/strayLight/waveletCompression.py | 427 +++++++++++++++++++++ python/lsst/obs/subaru/strayLight/yStrayLight.py | 220 +++++++++++ 6 files changed, 933 insertions(+)     price@pap-laptop:~/LSST/pipe_drivers (tickets/DM-12979=) $ git sub commit 388cf09cfb4c36866cb6754651e0587d9790b1e2 (HEAD -> tickets/DM-12979, origin/tickets/DM-12979) Author: Paul Price <price@astro.princeton.edu> Date: Wed Dec 13 10:34:39 2017 -0500   add visualizeVisit This provides a facility to produce a binned image of the entire focal plane (butler product visit_camera).   bin.src/visualizeVisit.py | 3 + python/lsst/pipe/drivers/skyCorrection.py | 43 +------- python/lsst/pipe/drivers/visualizeVisit.py | 160 +++++++++++++++++++++++++++++ 3 files changed, 167 insertions(+), 39 deletions(-)
            Hide
            price Paul Price added a comment -

            Thanks for volunteering to review, Jim Bosch!

            Show
            price Paul Price added a comment - Thanks for volunteering to review, Jim Bosch !
            price Paul Price made changes -
            Field Original Value New Value
            Reviewers Jim Bosch [ jbosch ]
            Status To Do [ 10001 ] In Review [ 10004 ]
            Hide
            jbosch Jim Bosch added a comment -

            Looks good overall. Comments on the PRs.

            Show
            jbosch Jim Bosch added a comment - Looks good overall. Comments on the PRs.
            jbosch Jim Bosch made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            price Paul Price added a comment -


            Left: visit=12979 with the correction. Right: the same exposure without the correction.

            Note that the correction makes ccd=33 unusable (the CCD failed by the time the correction data was collected, so we mask it as BAD) and introduces a bad amplifier on ccd=43 (for a similar reason).

            Show
            price Paul Price added a comment - Left : visit=12979 with the correction. Right : the same exposure without the correction. Note that the correction makes ccd=33 unusable (the CCD failed by the time the correction data was collected, so we mask it as BAD ) and introduces a bad amplifier on ccd=43 (for a similar reason).
            price Paul Price made changes -
            Attachment dm-12979.png [ 30637 ]
            Hide
            sogo.mineo Sogo Mineo added a comment -

            python/lsst/obs/subaru/strayLight/yStrayLight.py:

                if detId in badAmps:
                    isBad = numpy.zeros_like(model, dtype=bool)
                    for ii in badAmps[detId]:
                        amp = exposure.getDetector()[ii]
                        box = amp.getBBox()
                        isBad[box.getMinY():box.getMaxY(), box.getMinX():box.getMaxX()] = True
            

            box.getMinY():box.getMaxY() should be box.getStartY():box.getEndY()
            because getEndY() returns the one-past-the-end value that is expected by
            python slices. So too with X.

            I think the other things are OK.

            Show
            sogo.mineo Sogo Mineo added a comment - python/lsst/obs/subaru/strayLight/yStrayLight.py: if detId in badAmps: isBad = numpy.zeros_like(model, dtype=bool) for ii in badAmps[detId]: amp = exposure.getDetector()[ii] box = amp.getBBox() isBad[box.getMinY():box.getMaxY(), box.getMinX():box.getMaxX()] = True box.getMinY():box.getMaxY() should be box.getStartY():box.getEndY() because getEndY() returns the one-past-the-end value that is expected by python slices. So too with X. I think the other things are OK.
            price Paul Price made changes -
            Description HSC has had some features in the y-band images due to encoders in the instrument. Sogo Mineo has written some code to remove them (report and link to the code [here|https://hscdata.mtk.nao.ac.jp/hsc_bin_dist/4fc0f523/]), and we would like to include it in the pipeline. HSC has had some features in the y-band images due to encoders in the instrument. [~sogo.mineo] has written some code to remove them (report and link to the code [here|https://hscdata.mtk.nao.ac.jp/hsc_bin_dist/4fc0f523/]), and we would like to include it in the pipeline.
            Hide
            price Paul Price added a comment -

            Thanks, I didn't know about those! (But it is getBeginY instead of getStartY.) Fixed the code, and testing before merging.

            Show
            price Paul Price added a comment - Thanks, I didn't know about those! (But it is getBeginY instead of getStartY .) Fixed the code, and testing before merging.
            Hide
            price Paul Price added a comment -

            Hey, that saves ccd=33! Thanks Sogo Mineo!

            Merged to master.

            Show
            price Paul Price added a comment - Hey, that saves ccd=33 ! Thanks Sogo Mineo ! Merged to master.
            price Paul Price made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            Show
            price Paul Price added a comment - Post on CLO: https://community.lsst.org/t/y-band-stray-light-correction-for-hsc/2517
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue is triggering RFC-426 [ RFC-426 ]
            price Paul Price made changes -
            Component/s pipe_drivers [ 13309 ]
            price Paul Price made changes -
            Link This issue is triggering DM-13092 [ DM-13092 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue relates to DM-13066 [ DM-13066 ]

              People

              Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, Paul Price, Sogo Mineo
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.