Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13146

Update HSC defects

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
    • Team:
      External

      Description

      Hisanori Furusawa writes:

      I have reviewed CCD defects set in the current
      obs_subaru, and updated them.
      The compiled defectRegistry and relevant files are uploaded as:

      https://www.adc.nao.ac.jp/people/~furusawa/hsc/s18a/defects/defects_20171228.tar.gz

      • Here, I tried to mask edge pixels of each CCD as we discussed in the
        telecon (see the attached figure; red boxes are edge masks),
      • and also masked out bad pixel clusters more carefully.
      • I took a conservative choice for bad amps, in the same
        manner as before, e.g., ccd=0 ch=0, ccd=43 cch=4, where I applied
        masks for the entire amp, without too much tuning up the masked areas
        and period.
      • Similarly, I have masked ccd=94 ch=2 for all periods, as this amp
        shows flat field error due to poor linearity correction with low-count
        levels.

      I am esting singleFrameDriver through coadding with these masks
      over the RC dataset, and will report you when I found something bad.
      But, in the meantime, I would be grateful if you could be prepared to
      update the defects for the next release cutout.

        Attachments

        1. coadd_g_rc4_rc3_defects_3.jpg
          377 kB
          Hisanori Furusawa
        2. warp_g_rc4_rc3_defects_3.jpg
          322 kB
          Hisanori Furusawa

          Issue Links

            Activity

            Hide
            furusawa.hisanori Hisanori Furusawa added a comment - - edited

            I have quickly compared warp & coadd images created by using the updated defects (on hscPipe 6.0-beta4) with those with previous defects (hscPipe 6.0-beta3). I found no clear problems in the result with the updated defects. 

            • warp in g-band (left: 6.0beta4+updated defects, right: 6.0beta3+default_defects)
            • coadd in g-band (left: 6.0beta4+updated defects, right: 6.0beta3+default_defects)

            The result with beta4 includes improvements by the updated Yusra's artifact rejection in some regions, too. I am checking some more images, and will report if I find anything that needs attention. This test was done using the existing CALIB (with previous defects), so we should regenerate CALIBs in the next beta release, to make sure the whole procedure from isr to coadd runs without failure.

            Show
            furusawa.hisanori Hisanori Furusawa added a comment - - edited I have quickly compared warp & coadd images created by using the updated defects (on hscPipe 6.0-beta4) with those with previous defects (hscPipe 6.0-beta3). I found no clear problems in the result with the updated defects.  warp in g-band (left: 6.0beta4+updated defects, right: 6.0beta3+default_defects) coadd in g-band (left: 6.0beta4+updated defects, right: 6.0beta3+default_defects) The result with beta4 includes improvements by the updated Yusra's artifact rejection in some regions, too. I am checking some more images, and will report if I find anything that needs attention. This test was done using the existing CALIB (with previous defects), so we should regenerate CALIBs in the next beta release, to make sure the whole procedure from isr to coadd runs without failure.
            Hide
            price Paul Price added a comment -

            Nate Lust, would you mind reviewing this please?

            pprice@tiger-sumire:~/LSST/obs/subaru (tickets/DM-13146=) $ git sub
            commit fc8a70f4b1d1074e020d18efd3ef99d25a05c1e9
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Thu Dec 28 13:10:35 2017 -0500
             
                hsc: update defects
                
                Updated defect lists provided by Furusawa-san, who writes:
                > - Here, I tried to mask edge pixels of each CCD as we discussed
                > in the telecon (see the attached figure; red boxes are edge masks),
                > - and also masked out bad pixel clusters more carefully.
                > - I took a conservative choice for bad amps, in the same
                > manner as before, e.g., ccd=0 ch=0, ccd=43 cch=4, where I applied
                > masks for the entire amp, without too much tuning up the masked
                > areas and period.
                > Similarly, I have masked ccd=94 ch=2 for all periods, as this amp
                > shows flat field error due to poor linearity correction with
                > low-count levels.
             
             hsc/defects/2013-01-31/defects.dat | 836 ++++++++++++++++++++++++++++++++++--
             hsc/defects/2014-04-03/defects.dat | 837 +++++++++++++++++++++++++++++++++++--
             hsc/defects/2014-06-01/defects.dat | 836 ++++++++++++++++++++++++++++++++++--
             hsc/defects/2014-09-01/defects.dat |  42 --
             hsc/defects/2016-04-01/defects.dat | 837 +++++++++++++++++++++++++++++++++++--
             hsc/defects/2016-11-22/defects.dat | 837 +++++++++++++++++++++++++++++++++++--
             hsc/defects/2016-12-23/defects.dat | 836 ++++++++++++++++++++++++++++++++++--
             7 files changed, 4802 insertions(+), 259 deletions(-)
            

            Show
            price Paul Price added a comment - Nate Lust , would you mind reviewing this please? pprice@tiger-sumire:~/LSST/obs/subaru (tickets/DM-13146=) $ git sub commit fc8a70f4b1d1074e020d18efd3ef99d25a05c1e9 Author: Paul Price <price@astro.princeton.edu> Date: Thu Dec 28 13:10:35 2017 -0500   hsc: update defects Updated defect lists provided by Furusawa-san, who writes: > - Here, I tried to mask edge pixels of each CCD as we discussed > in the telecon (see the attached figure; red boxes are edge masks), > - and also masked out bad pixel clusters more carefully. > - I took a conservative choice for bad amps, in the same > manner as before, e.g., ccd=0 ch=0, ccd=43 cch=4, where I applied > masks for the entire amp, without too much tuning up the masked > areas and period. > Similarly, I have masked ccd=94 ch=2 for all periods, as this amp > shows flat field error due to poor linearity correction with > low-count levels.   hsc/defects/2013-01-31/defects.dat | 836 ++++++++++++++++++++++++++++++++++-- hsc/defects/2014-04-03/defects.dat | 837 +++++++++++++++++++++++++++++++++++-- hsc/defects/2014-06-01/defects.dat | 836 ++++++++++++++++++++++++++++++++++-- hsc/defects/2014-09-01/defects.dat | 42 -- hsc/defects/2016-04-01/defects.dat | 837 +++++++++++++++++++++++++++++++++++-- hsc/defects/2016-11-22/defects.dat | 837 +++++++++++++++++++++++++++++++++++-- hsc/defects/2016-12-23/defects.dat | 836 ++++++++++++++++++++++++++++++++++-- 7 files changed, 4802 insertions(+), 259 deletions(-)
            Hide
            nlust Nate Lust added a comment -

            Looking at things the mechanics of the changes seem ok. I would request that you change the comment in the dat file (or add another line) that describes what the columns are, similar to what already exists for anyone that may just look at that part in the future. It would avoid any confusion with people who may be thinking in python y,x vs x,y for instance.

            Show
            nlust Nate Lust added a comment - Looking at things the mechanics of the changes seem ok. I would request that you change the comment in the dat file (or add another line) that describes what the columns are, similar to what already exists for anyone that may just look at that part in the future. It would avoid any confusion with people who may be thinking in python y,x vs x,y for instance.
            Hide
            price Paul Price added a comment -

            I've updated the comments, making the identification of the columns more clear. Please let me know if this is not what you had in mind.

            Show
            price Paul Price added a comment - I've updated the comments, making the identification of the columns more clear. Please let me know if this is not what you had in mind.
            Hide
            price Paul Price added a comment -

            Merged to master.

            Show
            price Paul Price added a comment - Merged to master.

              People

              • Assignee:
                price Paul Price
                Reporter:
                price Paul Price
                Reviewers:
                Nate Lust
                Watchers:
                Hisanori Furusawa, Nate Lust, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel