Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16864

Investigate relative DcrModel option

    Details

    • Story Points:
      8
    • Epic Link:
    • Sprint:
      AP S19-1, AP S19-2
    • Team:
      Alert Production

      Description

      An option that might improve the DcrModel is to make the model relative to a reference image. Each subfilter model plane would act like a "rubber sheet" that can enhance or suppress the flux of the reference image in a pixel, rather than storing the absolute flux for that subfilter. This may make it easier to apply regularization, and ideally will also prevent noise being amplified.

      If this feature ends up creating more problems than it solves, or otherwise does not perform as well as the original code, it should be closed as Won't Fix.

        Attachments

          Issue Links

            Activity

            Hide
            sullivan Ian Sullivan added a comment -

            I explored using a relative model throughout building the DcrModel, but found it increased the code complexity and computation time. I was able to get the same results by using a smoothly-varying relative model in frequency regularization, which is comparatively quick to calculate. The new regularization significantly decreases chromatic artifacts in the wings of sources, and for the first time allows sensible color images to be made directly from the DcrModel subfilter images (see below).


            To produce the above image, I generated 12 simulated images using StarFast with airmasses ranging from 1.0 to 2.0, each including main sequence stars with Kurucz SEDs and quasars using the Vanden Berk quasar spectrum with redshifts from 0 to 4. I ran dcrAssembleCoadd on the ingested images, and loaded the resulting three DcrCoadd subfilter exposures into DS9, using the longest wavelength subfilter for the red channel, the center subfilter for the green channel, and the shortest wavelength subfilter for the blue channel. Bright objects are clipped and white for this color scale, but for the fainter objects those that are very red are quasars and while those that are blue are mostly cool stars with a steep spectrum in g band.

            Show
            sullivan Ian Sullivan added a comment - I explored using a relative model throughout building the DcrModel, but found it increased the code complexity and computation time. I was able to get the same results by using a smoothly-varying relative model in frequency regularization, which is comparatively quick to calculate. The new regularization significantly decreases chromatic artifacts in the wings of sources, and for the first time allows sensible color images to be made directly from the DcrModel subfilter images (see below). To produce the above image, I generated 12 simulated images using StarFast with airmasses ranging from 1.0 to 2.0, each including main sequence stars with Kurucz SEDs and quasars using the Vanden Berk quasar spectrum with redshifts from 0 to 4. I ran dcrAssembleCoadd on the ingested images, and loaded the resulting three DcrCoadd subfilter exposures into DS9, using the longest wavelength subfilter for the red channel, the center subfilter for the green channel, and the shortest wavelength subfilter for the blue channel. Bright objects are clipped and white for this color scale, but for the fainter objects those that are very red are quasars and while those that are blue are mostly cool stars with a steep spectrum in g band.
            Hide
            sullivan Ian Sullivan added a comment -

            Please note that this ticket includes code in both ip_diffim as well as pipe_tasks, though due to a long-standing bug the pipe_tasks pull request does not show up in Jira.

            Show
            sullivan Ian Sullivan added a comment - Please note that this ticket includes code in both ip_diffim as well as pipe_tasks , though due to a long-standing bug the pipe_tasks pull request does not show up in Jira.
            Hide
            gkovacs Gabor Kovacs added a comment - - edited

            Code looks all right, as far as I understand. A few documentation related minor comments:

            • I'd suggest adding a few lines of docstring to regularizeModelFreq describing that you use a combination of 2 different scale smoothing on the relative model, plus a note on the hard-wired factors of 3. (Where are they from?)
            • May not be related to the present ticket, I could not find any docs from dcrModel.py in pipelines.lsst.io docs builds (e.g. DcrModel). Should classes be imported into the package top level _init_.py ?
            • Not strictly related to the present ticket, I think in the description of getReferenceImage we might want to avoid calling the "mean model" as "template" in any context. It may be also worth noting somewhere that in the present implementation the "reference" is the "mean model" in general, and not something like another standalone image.
            Show
            gkovacs Gabor Kovacs added a comment - - edited Code looks all right, as far as I understand. A few documentation related minor comments: I'd suggest adding a few lines of docstring to regularizeModelFreq describing that you use a combination of 2 different scale smoothing on the relative model, plus a note on the hard-wired factors of 3. (Where are they from?) May not be related to the present ticket, I could not find any docs from dcrModel.py in pipelines.lsst.io docs builds (e.g. DcrModel). Should classes be imported into the package top level _ init _.py ? Not strictly related to the present ticket, I think in the description of getReferenceImage we might want to avoid calling the "mean model" as "template" in any context. It may be also worth noting somewhere that in the present implementation the "reference" is the "mean model" in general, and not something like another standalone image.

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Reviewers:
                Gabor Kovacs
                Watchers:
                Gabor Kovacs, Ian Sullivan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel