Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31505

Update fgcm to better handle survey edges and reference star outliers

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: fgcmcal
    • Labels:
      None

      Description

      Comparison of Gen2 to Gen3 photometric calibrations on w_30 by Lauren MacArthur turned up some anomalies. The main difference between the Gen2 and the Gen3 runs was caused by a source of randomness that wasn't seedable, fixed on DM-31462.

      However, clicking through the visit-level comparisons turned up a very odd duck, visit 24538 in the i-band:

      Further investigation has shown that (a) the hot spot on this is due to a problem with the PS1 refcat, and the relative change is due to how fgcm handles refstar outliers; and (b) the hard edge at dec<-0.95 is because this image is all alone at the southern edge of the RC2 inputs. At the edge it turns out that there is an inconsistency in the way that fgcm internally computes the photometric calibration and then externally reports the photometric calibration.

      This ticket will update the logic in fgcm to remove this inconsistency, and also more intelligently handle the reference star outliers.

        Attachments

          Issue Links

            Activity

            Hide
            erykoff Eli Rykoff added a comment -

            In the end, this is going to be a small change to fgcm to get rid of what was a dumb default to use refstars to override the internal calibrations in some cases. This leads to hot and cold spots in the reference star photometry bleeding into the photometry (bad) in an inconsistent way (even worse). There are more significant changes in the way the focal plane fitting works that I wanted to include, but realized that this requires DM-21215 which is suddenly urgent and I will work on next.

            I have run the updated (this ticket) and old code on the RC2 w_2021_30 run, and looking at the zeropoints for this visit you can see the difference:
            Old:

            New (this ticket):

            Delta (new - old):

            The new one doesn't have the discontinuity, which is good! (Note that the large scale variations here are due to the illumination correction).

            Show
            erykoff Eli Rykoff added a comment - In the end, this is going to be a small change to fgcm to get rid of what was a dumb default to use refstars to override the internal calibrations in some cases. This leads to hot and cold spots in the reference star photometry bleeding into the photometry (bad) in an inconsistent way (even worse). There are more significant changes in the way the focal plane fitting works that I wanted to include, but realized that this requires DM-21215 which is suddenly urgent and I will work on next. I have run the updated (this ticket) and old code on the RC2 w_2021_30 run, and looking at the zeropoints for this visit you can see the difference: Old: New (this ticket): Delta (new - old): The new one doesn't have the discontinuity, which is good! (Note that the large scale variations here are due to the illumination correction).
            Hide
            erykoff Eli Rykoff added a comment -

            This small change improves things significantly in my tests. Further changes to handle survey edges will have to wait until DM-21215.

            Show
            erykoff Eli Rykoff added a comment - This small change improves things significantly in my tests. Further changes to handle survey edges will have to wait until DM-21215 .
            Hide
            lauren Lauren MacArthur added a comment - - edited

            LGTM.  I presume your “tests” also include similar checks on other visits 

            Let’s remember to keep this change in mind when doing the next run comparisons.

            (and I see you just had a successful Jenkins )

            Show
            lauren Lauren MacArthur added a comment - - edited LGTM.  I presume your “tests” also include similar checks on other visits  Let’s remember to keep this change in mind when doing the next run comparisons. (and I see you just had a successful Jenkins )

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Reviewers:
              Lauren MacArthur
              Watchers:
              Eli Rykoff, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.