Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-6147

Set SUSPECT mask in ISR task and make saturation a double

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_isr
    • Story Points:
      5
    • Epic Link:
    • Sprint:
      Alert Production X16 - 5
    • Team:
      Alert Production

      Description

      Implement RFC-190 and mask suspect pixels:

      Add selectLevel to lsst.afw.cameraGeom.AmpInfoCatalog, as a double, and add support for it to lsst.ip.isr.IsrTask, analogously to masking saturation: iif suspectLevel is not nan then set the SUSPECT flag for pixels above the suspect level.

      Also change the type of saturation in the AmpInfoCatalog from int to double, so that the existing test for nan actually works

        Attachments

          Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. DM-5462 Add non-linearity correction to ISR task

          • Done
          is triggered by

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-190 Use nan for saturation level to prevent masking

          • Implemented
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-6246 Vertical overscan off by one again

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-6161 Add unit test for IsrTask

          • Invalid

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-192 Single source for camera data

          • Adopted

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            rowen Russell Owen created issue -
            rowen Russell Owen made changes -
            Field Original Value New Value
            Link This issue is triggered by RFC-190 [ RFC-190 ]
            rowen Russell Owen made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            rowen Russell Owen made changes -
            Link This issue blocks DM-5462 [ DM-5462 ]
            rowen Russell Owen made changes -
            Epic Link DM-3878 [ 20173 ]
            Hide
            Permalink
            rowen Russell Owen added a comment -

            Changes include:

            • Change saturation level in AmpInfoCatalog from int to double, as per RFC-190
            • Add suspectLevel to AmpInfoCatalog
            • Add suspect masking to IsrTask in ip_isr
            • Update amp info in various obs_ packages (including camera generating code, where appropriate). All except obs_subaru set the value to nan (do not mask) for now, since there was no existing value to copy.
            • Update subaru-specific ISR task to get suspect level from the new amp info field, and to treat nan as "do not mask" instead of a large value (the amp info conversion converted such large values to nan, though I did not find any instances)
            Show
            rowen Russell Owen added a comment - Changes include: Change saturation level in AmpInfoCatalog from int to double, as per RFC-190 Add suspectLevel to AmpInfoCatalog Add suspect masking to IsrTask in ip_isr Update amp info in various obs_ packages (including camera generating code, where appropriate). All except obs_subaru set the value to nan (do not mask) for now, since there was no existing value to copy. Update subaru-specific ISR task to get suspect level from the new amp info field, and to treat nan as "do not mask" instead of a large value (the amp info conversion converted such large values to nan , though I did not find any instances)
            rowen Russell Owen made changes -
            Reviewers Nate Lust [ nlust ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            rowen Russell Owen made changes -
            Summary Set SUSPECT mask in ISR task Set SUSPECT mask in ISR task and make saturation a double
            rowen Russell Owen made changes -
            Reviewers Nate Lust [ nlust ] John Parejko [ parejkoj ]
            rowen Russell Owen made changes -
            Watchers Nate Lust, Russell Owen [ Nate Lust, Russell Owen ] Nate Lust, Paul Price, Russell Owen [ Nate Lust, Paul Price, Russell Owen ]
            rowen Russell Owen made changes -
            Reviewers John Parejko [ parejkoj ] Nate Lust [ nlust ]
            rowen Russell Owen made changes -
            Reviewers Nate Lust [ nlust ] John Parejko, Nate Lust [ parejkoj, nlust ]
            Hide
            Permalink
            rowen Russell Owen added a comment - - edited

            Nate Lust please review the obs packages, starting with obs_subaru
            John Parejko please review the changes to the other packages (ip_isr, daf_butlerUtils, and afw)
            Either of you should feel free to comment on any other changes you like, but by splitting this I hoped to give you a break.

            Show
            rowen Russell Owen added a comment - - edited Nate Lust please review the obs packages, starting with obs_subaru John Parejko please review the changes to the other packages (ip_isr, daf_butlerUtils, and afw) Either of you should feel free to comment on any other changes you like, but by splitting this I hoped to give you a break.
            rowen Russell Owen made changes -
            Link This issue relates to DM-6161 [ DM-6161 ]
            Parejkoj John Parejko made changes -
            Reviewers John Parejko, Nate Lust [ parejkoj, nlust ] Nate Lust [ nlust ]
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            I reviewed the non obs_* stuff, and I'm happy with the changes Russell's made post-review.

            Show
            Parejkoj John Parejko added a comment - I reviewed the non obs_* stuff, and I'm happy with the changes Russell's made post-review.
            Hide
            Permalink
            nlust Nate Lust added a comment -

            The obs_* stuff all looks benign and reasonable. I am a bit bothered by the values in fits files being unable to be displayed, that probably indicates some weird formatting written to the files, but that exists before this ticket so there is no reason to look into now.

            Show
            nlust Nate Lust added a comment - The obs_* stuff all looks benign and reasonable. I am a bit bothered by the values in fits files being unable to be displayed, that probably indicates some weird formatting written to the files, but that exists before this ticket so there is no reason to look into now.
            nlust Nate Lust made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            rowen Russell Owen made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            rowen Russell Owen made changes -
            Link This issue relates to DM-6246 [ DM-6246 ]
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            It looks as though a whole bunch of stuff in obs_subaru got blown away with very little explanation in the course of this ticket: we're told that it was "outdated", but not what the replacements are, and it doesn't seem to have been discussed here at all. Can somebody explain what happened?

            Show
            swinbank John Swinbank added a comment - It looks as though a whole bunch of stuff in obs_subaru got blown away with very little explanation in the course of this ticket: we're told that it was "outdated", but not what the replacements are, and it doesn't seem to have been discussed here at all. Can somebody explain what happened?
            rowen Russell Owen made changes -
            Link This issue relates to RFC-192 [ RFC-192 ]
            Hide
            Permalink
            rowen Russell Owen added a comment -

            The deleted code was a broken method of generating camera geometry information from a paf file. The code and paf file are gone because that is deprecated. It can be resurrected from git history should Subaru decide it wants to generate its camera geometry information from a text file, instead of directly modifying the generated results.

            See RFC-192 for more information. Permission was given on the HipChat in the HSC Synchronization room. Given that this was removing broken code, nobody seemed to feel that an RFC or wider review was necessary.

            Note that I took a different route for CFHT because the generating code was less broken (it ran but gave slightly incorrect results): the outdated code is marked as deprecated and prints a warning if used. I figured the folks maintaining obs_cfht could figure out what to do next based on the outcome of RFC-192.

            Show
            rowen Russell Owen added a comment - The deleted code was a broken method of generating camera geometry information from a paf file. The code and paf file are gone because that is deprecated. It can be resurrected from git history should Subaru decide it wants to generate its camera geometry information from a text file, instead of directly modifying the generated results. See RFC-192 for more information. Permission was given on the HipChat in the HSC Synchronization room. Given that this was removing broken code, nobody seemed to feel that an RFC or wider review was necessary. Note that I took a different route for CFHT because the generating code was less broken (it ran but gave slightly incorrect results): the outdated code is marked as deprecated and prints a warning if used. I figured the folks maintaining obs_cfht could figure out what to do next based on the outcome of RFC-192 .

              People

              Assignee:
              rowen Russell Owen
              Reporter:
              rowen Russell Owen
              Reviewers:
              Nate Lust
              Watchers:
              John Parejko, John Swinbank, Nate Lust, Paul Price, Russell Owen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.