Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5107

Fix effective coordinates for defects in obs_subaru

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      Science Pipelines DM-W16-6
    • Team:
      Data Release Production

      Description

      The defects as defined in obs_subaru (in the hsc/defects/20NN-NN-NN/defects.dat files) are defined in a coordinate system where pixel (0, 0) is the lower left pixel. However, the LSST stack does not use this interpretation, preferring to maintain the coordinate system tied to the electronics. As such, the defect positions are being misinterpreted for the rotated CCDs in HSC (see HSC CCD layout). This needs to be remedied.

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            Here is an example of the problem for visit=2228, ccd=9 (a rotated HSC ccd). (Note that I had to disable overscan correction to get processCcd to run through. This is due to the issue discussed in DM-5094 and will be remedied there).

            The observation date was 2014-04-04 and, for that date, the defect list reads:

            #CCD x0    y0    width height
            # Dead amp
            9    0     0     512   4176
            

            It turns out that, at least in this case, the dead amp pixel values are all "saturated", so the saturation detection (green mask) detects them as such. Running processCcd and pausing to display the masked image before

            and after defect masking (red mask) and interpolation (green mask), (and red + green with 75% transparency == brownish!) reveals

            We see that the defects are being associated with the wrong amp for this rotated ccd.

            Show
            lauren Lauren MacArthur added a comment - Here is an example of the problem for visit=2228, ccd=9 (a rotated HSC ccd). (Note that I had to disable overscan correction to get processCcd to run through. This is due to the issue discussed in DM-5094 and will be remedied there). The observation date was 2014-04-04 and, for that date, the defect list reads: #CCD x0 y0 width height # Dead amp 9 0 0 512 4176 It turns out that, at least in this case, the dead amp pixel values are all "saturated", so the saturation detection (green mask) detects them as such. Running processCcd and pausing to display the masked image before and after defect masking (red mask) and interpolation (green mask), (and red + green with 75% transparency == brownish!) reveals We see that the defects are being associated with the wrong amp for this rotated ccd.
            Hide
            lauren Lauren MacArthur added a comment -

            Paul, would you mind giving this a look? Here is the post fix plot after defect detection and interpolation:

            Jenkins is happy.

            Show
            lauren Lauren MacArthur added a comment - Paul, would you mind giving this a look? Here is the post fix plot after defect detection and interpolation: Jenkins is happy.
            Hide
            price Paul Price added a comment -

            I'm glad you found this, and I'm sure your fix works, but I wonder if it's the best way to go about it. I wonder if we wouldn't be better off fixing the defect lists rather than leaving them in a coordinate system that doesn't match that of the images and fixing them after the fact. It means that if I find a new defect, I need to do some additional manipulation to the coordinates to add them into the list; but those manipulated coordinates are going to be unmanipulated by the code you just put in.

            Show
            price Paul Price added a comment - I'm glad you found this, and I'm sure your fix works, but I wonder if it's the best way to go about it. I wonder if we wouldn't be better off fixing the defect lists rather than leaving them in a coordinate system that doesn't match that of the images and fixing them after the fact. It means that if I find a new defect, I need to do some additional manipulation to the coordinates to add them into the list; but those manipulated coordinates are going to be unmanipulated by the code you just put in.
            Hide
            lauren Lauren MacArthur added a comment -

            I totally agree, but would still suggest we put in this fix for now. Perhaps we can make a ticket that combines the re-ingestion of the HSC calibs and the defect coordinate manipulation and remove all the rotated CCD accommodations there?

            Show
            lauren Lauren MacArthur added a comment - I totally agree, but would still suggest we put in this fix for now. Perhaps we can make a ticket that combines the re-ingestion of the HSC calibs and the defect coordinate manipulation and remove all the rotated CCD accommodations there?
            Hide
            price Paul Price added a comment -

            Fair enough. Would you make that ticket please?

            Show
            price Paul Price added a comment - Fair enough. Would you make that ticket please?
            Hide
            lauren Lauren MacArthur added a comment -

            Ticket is DM-5124.

            Merged to master.

            Show
            lauren Lauren MacArthur added a comment - Ticket is DM-5124 . Merged to master.

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                Paul Price
                Watchers:
                Lauren MacArthur, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel