Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16595

Add "sky objects" qa plots to pipe_analysis

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Story Points:
      24
    • Epic Link:
    • Team:
      Data Release Production

      Description

      When merging multiband coadd catalogs into a single reference catalog, "sky objects" corresponding to regions of empty sky (i.e. at positions where nothing else has been detected) are added to the catalog and get measured fluxes and shapes for whatever measurement algorithms are setup along with the "real" detected sources. The intention of these objects is for qa purposes (ensuring the final subtracted sky level is indeed zero, in particular). This issue is to add plots to the `pipe_analysis` scripts to make use of these "sky objects", including distributions (histograms) and metrics (mean and standard deviations) of various fluxes (namely aperture and psf fluxes) and their measurement errors.

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment - - edited

            I think this is finally ready for review...do you mind having a look? I've attached a pdf showing the sky source/object histograms for visits/coadds of HSC and DC2 run2.2i data. The focal plane/tract panels are color-coded by the mean metric per unit (ccd/patch).

            I've also added the ability to plot the positions of the sky objects on the coadds overlayed on a grayscale image of the tract. The default is False for these plots as they are large and you have to zoom in to really see anything, but I include one here to demonstrate how bad the sky positioning can be for coadds where there is no data (motivation for another pair-coding ticket to fix this up?!)  These rogue placements also made for a much more difficult time culling them for the histogram plots and stats...

            In case you're unfamiliar with the expectations here, since pipe_analysis still lives in lsst-dm, it is not currently officially subject to the strict code standards of lsst, so you can feel free to give it only a cursory look. However, we do plan on moving it to lsst on github, so any comments would be most welcome and I will do my best to address them all, so also feel free to nit-pick to your desire.

            PR is here.

            Show
            lauren Lauren MacArthur added a comment - - edited I think this is finally ready for review...do you mind having a look? I've attached a pdf showing the sky source/object histograms for visits/coadds of HSC and DC2 run2.2i data. The focal plane/tract panels are color-coded by the mean metric per unit (ccd/patch). I've also added the ability to plot the positions of the sky objects on the coadds overlayed on a grayscale image of the tract. The default is False for these plots as they are large and you have to zoom in to really see anything, but I include one here to demonstrate how bad the sky positioning can be for coadds where there is no data (motivation for another pair-coding ticket to fix this up?!)  These rogue placements also made for a much more difficult time culling them for the histogram plots and stats... In case you're unfamiliar with the expectations here, since pipe_analysis still lives in lsst-dm, it is not currently officially subject to the strict code standards of lsst, so you can feel free to give it only a cursory look. However, we do plan on moving it to lsst on github, so any comments would be most welcome and I will do my best to address them all, so also feel free to nit-pick to your desire. PR is here .
            Hide
            lskelvin Lee Kelvin added a comment -

            A very nice ticket introducing important qa plots into pipe_analysis. I have nothing major to add. A signal-to-noise threshold is hard-coded on line 175 in analysis.py, and it might be nice to add a comment explaining where this has come from. The output plots look great! Looks good to merge to master!

            Show
            lskelvin Lee Kelvin added a comment - A very nice ticket introducing important qa plots into pipe_analysis . I have nothing major to add. A signal-to-noise threshold is hard-coded on line 175 in analysis.py , and it might be nice to add a comment explaining where this has come from. The output plots look great! Looks good to merge to master!
            Hide
            lauren Lauren MacArthur added a comment -

            Thanks Lee! Comment added and merged to master.

            Show
            lauren Lauren MacArthur added a comment - Thanks Lee! Comment added and merged to master.

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Lee Kelvin
              Watchers:
              Lauren MacArthur, Lee Kelvin, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.