Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18532

Investigate cause of increase in Principal Color scatter in tract 9697

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • pipe_analysis
    • None
    • 4
    • DRP S19-4
    • Data Release Production

    Description

      yusra points out that there is an increase in the "wired" wPerp Principal Color scatter for tract 9697 between the w_2018_36 and w_2018_38 HSC RC2 processing runs (and it has persisted ever since). This is the blue "wired" curves: https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_36/pipe_analysis/qaPlots/color/tract-9697/plot-t9697-griPSF-wFit-fit.png vs. https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_38/pipe_analysis/qaPlots/color/tract-9697/plot-t9697-griPSF-wFit-fit.png
      Investigate the cause.

      Attachments

        Activity

          No builds found.
          lauren Lauren MacArthur created issue -
          lauren Lauren MacArthur made changes -
          Field Original Value New Value
          Epic Link DM-16680 [ 235240 ]
          lauren Lauren MacArthur made changes -
          Status To Do [ 10001 ] In Progress [ 3 ]
          lauren Lauren MacArthur added a comment - - edited

          I tracked down this issue. It turns out there were some inconsistencies in doing the transformations to the Principal Colors for the fit and wired coefficients (we do this for both PSF mags and CModel mags and they were getting intermixed). This is fixed here and we now also use the same points in all mean and stddev calculations (i.e. those that were used in the fit and survived the sigma cuts).

          This is the pre-fix plot:
          https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2019_10/pipe_analysis/qaPlots/color/tract-9697/plot-t9697-griPSF-wFit-fit.png
          And here is the same post-fix:
          https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-18532/plots/color/tract-9697/plot-t9697-griPSF-wFit-fit.png

          Updated plot for all tracts can be found (for now!) in: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-18532/plots/color/

          (Oh, and I should add that the issue revealed itself when I changed the flux-to-plot-against column from PSF to CModel. The default set in coaddAnalysis.py is also adopted in colorAnalysis.py...I may not have noticed otherwise!)

          lauren Lauren MacArthur added a comment - - edited I tracked down this issue. It turns out there were some inconsistencies in doing the transformations to the Principal Colors for the fit and wired coefficients (we do this for both PSF mags and CModel mags and they were getting intermixed). This is fixed here and we now also use the same points in all mean and stddev calculations (i.e. those that were used in the fit and survived the sigma cuts). This is the pre-fix plot: https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2019_10/pipe_analysis/qaPlots/color/tract-9697/plot-t9697-griPSF-wFit-fit.png And here is the same post-fix: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-18532/plots/color/tract-9697/plot-t9697-griPSF-wFit-fit.png Updated plot for all tracts can be found (for now!) in: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-18532/plots/color/ (Oh, and I should add that the issue revealed itself when I changed the flux-to-plot-against column from PSF to CModel. The default set in coaddAnalysis.py is also adopted in colorAnalysis.py ...I may not have noticed otherwise!)

          Thanks again for catching this issue.  Can you give this a look?

          lauren Lauren MacArthur added a comment - Thanks again for catching this issue.  Can you give this a look?
          lauren Lauren MacArthur made changes -
          Reviewers Yusra AlSayyad [ yusra ]
          Status In Progress [ 3 ] In Review [ 10004 ]
          yusra Yusra AlSayyad made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]

          Thanks for the review! Merged to master.

          lauren Lauren MacArthur added a comment - Thanks for the review! Merged to master.
          lauren Lauren MacArthur made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]

          People

            lauren Lauren MacArthur
            lauren Lauren MacArthur
            Yusra AlSayyad
            Lauren MacArthur, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Jenkins

                No builds found.