Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20137

Some coadd mask planes are wiping out others, instead of being OR'd in

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
      None
    • Team:
      Data Release Production

      Description

      Paraphrased bug report from Andy Goulding (parentheticals are Jim Bosch's musings):

      Some of the mask pixels around bright objects (probably BRIGHT_OBJECT, or maybe DETECTED) are having that plane cleared when they overlap with another mask plane, which mostly appears as narrow rectangles or stripes (INEXACT_PSF or SENSOR EDGE?).   This was particularly obvious in the Deep and UltraDeep fields.

      This sounds like a fairly straightforward case of something assigning to the mask plane when it should be OR'ing into it.  What's confusing (at least to me) is that I thought the candidates for the plane being cleared (BRIGHT_OBJECT, DETECTED) are all set after all of the candidates for the plane that's apparently clearing them (INEXACT_PSF, SENSOR_EDGE).

      Paul Price, Yusra AlSayyad, Lauren MacArthur, do you have any guesses off the top of your heads about what could be going on here?  (i.e. maybe you know something about the order of operations that I've guessed wrong about).

      If we can't spot the effect in processed data we have in hand (e.g. an RC2 run) easily, I'm sure we can get more details from Andy as to exactly which mask planes are interacting poorly.

        Attachments

        1. IMG_20190613_130421.jpg
          712 kB
          Yusra AlSayyad
        2. mask-9615-44-R.png
          331 kB
          Yusra AlSayyad

          Activity

          Hide
          yusra Yusra AlSayyad added a comment -

          Its this line. Maybe the behavior of spans.setMask was misuderstood: https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/assembleCoadd.py#L1099

          Nate Lust?

          Show
          yusra Yusra AlSayyad added a comment - Its this line. Maybe the behavior of spans.setMask was misuderstood: https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/assembleCoadd.py#L1099 Nate Lust ?
          Hide
          yusra Yusra AlSayyad added a comment -

          But I also don't see it in here:

          Show
          yusra Yusra AlSayyad added a comment - But I also don't see it in here:
          Hide
          yusra Yusra AlSayyad added a comment -

          OK Some more info from Jim is that Andy says "Masks look wonky anywhere 17th bit is NOT set." deepCoadd only goes up to 16. So deepCoadd_calexp written by detectCoaddSources is where to look.

          Show
          yusra Yusra AlSayyad added a comment - OK Some more info from Jim is that Andy says "Masks look wonky anywhere 17th bit is NOT set." deepCoadd only goes up to 16. So deepCoadd_calexp written by detectCoaddSources is where to look.
          Hide
          yusra Yusra AlSayyad added a comment -

          OK, this is from a deepCoadd_calexp from the most recent RC2. I still don't see it: https://lsst-web.ncsa.illinois.edu/~yusra/download/9813_33_500x500.png If he's talking about SENSOR_EDGE, it's all still there under the bright object mask.

          {'BAD': 0,
           'BRIGHT_OBJECT': 9,
           'CLIPPED': 13,
           'CR': 3,
           'CROSSTALK': 10,
           'DETECTED': 5,
           'DETECTED_NEGATIVE': 6,
           'EDGE': 4,
           'INEXACT_PSF': 14,
           'INTRP': 2,
           'NOT_DEBLENDED': 11,
           'NO_DATA': 8,
           'REJECTED': 15,
           'SAT': 1,
           'SENSOR_EDGE': 16,
           'SUSPECT': 7,
           'UNMASKEDNAN': 12}
          

          Show
          yusra Yusra AlSayyad added a comment - OK, this is from a deepCoadd_calexp from the most recent RC2. I still don't see it: https://lsst-web.ncsa.illinois.edu/~yusra/download/9813_33_500x500.png If he's talking about SENSOR_EDGE, it's all still there under the bright object mask. {'BAD': 0, 'BRIGHT_OBJECT': 9, 'CLIPPED': 13, 'CR': 3, 'CROSSTALK': 10, 'DETECTED': 5, 'DETECTED_NEGATIVE': 6, 'EDGE': 4, 'INEXACT_PSF': 14, 'INTRP': 2, 'NOT_DEBLENDED': 11, 'NO_DATA': 8, 'REJECTED': 15, 'SAT': 1, 'SENSOR_EDGE': 16, 'SUSPECT': 7, 'UNMASKEDNAN': 12}
          Hide
          yusra Yusra AlSayyad added a comment - - edited

          Andy showed me his screen. He was viewing integer mask plane in ds9. The bright object mask is 2^9 = 512. The SENSOR_EDGE was 2^16 = 65536. Because of the linear scaling, the +512 was drowned out by the the larger integers. Optical illusion. Masks are fine.

          Show
          yusra Yusra AlSayyad added a comment - - edited Andy showed me his screen. He was viewing integer mask plane in ds9. The bright object mask is 2^9 = 512 . The SENSOR_EDGE was 2^16 = 65536 . Because of the linear scaling, the +512 was drowned out by the the larger integers. Optical illusion. Masks are fine.
          Hide
          yusra Yusra AlSayyad added a comment -

          Show
          yusra Yusra AlSayyad added a comment -

            People

            • Assignee:
              Unassigned
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Jim Bosch, Lauren MacArthur, Paul Price, Yusra AlSayyad
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel