Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3338

KPM Measurement: Photometric Repeatability: procCalRep, FY15

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Epic Name:
      KPM: Photometric Repeatability: procCalRep, FY15
    • WBS:
      02C.03
    • Team:
      Alert Production
    • Cycle:
      Summer 2015

      Description

      This will be measured on precursor data (almost certainly from HSC, via DM-2380) using the following procedure:

      • Run visit-level processing (i.e. ProcessCcdTask, probably via the new HSC driver ported on DM-3368). This must include aperture correction (DM-85).
      • Run relative photometric calibration (i.e. meas_mosaic, DM-2674).
      • Select bright stars and match across visits (new scripts, mostly delegating to existing code). Just selecting the stars used for PSF determination would probably work.
      • Plot magnitude deltas (difference from per-object mean magnitudes) vs. magnitude; measure RMS and outliers in magnitude bins.

      Given the limited amount of public HSC data available, this will probably be restricted to at most three bands (gri).

      This measurement will be the same as that for PA1uzy (DLP-316), and PA1gri (DLP-315) in this cycle (DM-3324, DM-3331), since we currently do not have a way of differentiating between pipeline-contributed errors and the complete error budget.

        Attachments

          Issue Links

            Activity

            Hide
            rhl Robert Lupton added a comment -

            I understand that you don't have more time to spend on this, but the next time around we need to use aperture corrected aperture magnitudes.

            This may require getting the aperture correction code running reliably, and than in turn may mean more robust algorithms for measuring the aperture fluxes.

            Show
            rhl Robert Lupton added a comment - I understand that you don't have more time to spend on this, but the next time around we need to use aperture corrected aperture magnitudes. This may require getting the aperture correction code running reliably, and than in turn may mean more robust algorithms for measuring the aperture fluxes.
            Hide
            jbosch Jim Bosch added a comment -

            Following an off-line conversation between Robert Lupton and Zeljko Ivezic, we will use aperture magnitudes here, and to be conservative, we'll use the RMS estimate, so the final FY15 estimate for PA1 is 11.1 mmag, which does meet the FY15 target. With this change, we also meet the minimum spec for PA2/PF1. See the notebook for details:

            https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb

            Show
            jbosch Jim Bosch added a comment - Following an off-line conversation between Robert Lupton and Zeljko Ivezic , we will use aperture magnitudes here, and to be conservative, we'll use the RMS estimate, so the final FY15 estimate for PA1 is 11.1 mmag, which does meet the FY15 target. With this change, we also meet the minimum spec for PA2/PF1. See the notebook for details: https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb
            Hide
            jbosch Jim Bosch added a comment -

            Found a bug in the IPython notebook (fixed in the link in the comment immediately above). Since this has gone through so many modifications, here's a brand new writeup that I hope will indeed be the last one:

            Our FY15 estimate for the PA1 repeatability metric is 10.6 mmag (which meets the target of 13 mmag), measured on 12-pixel aperture fluxes. This cycle, we will use the same estimate for the procCalRep, PA1ugy, and PA1riz metrics, as these represent the same quantity computed on different datasets, and we are limited to one test dataset in FY15. This dataset is a selection of i-band HyperSuprime-Cam engineering data taken in the SDSS Stripe 82 region. This dataset consists of 30s exposures, so it is somewhat similar to projected LSST data in depth, and we've thus far put more effort into tuning the pipeline on HSC data than most other pipelines. Our current calibration approach has many limitations relative to what we ultimately plan to implement for LSST, so there is still plenty of room for improvement here:

            • There's currently no relative calibration being run at all.
            • We have only limited correction for chromatic effects.
            • There's currently no allowance for zeropoint variations smaller than the scale of a CCD (including the effects of pixel area changes). We have not included the outermost CCDs in the HSC focal plane in the processing (where the geometric factor is largest), however, so this this should not play as large a role as it otherwise might.

            We also use a much simpler sample selection than that proposed by the SRD; the SRD suggests using colors and variability to ensure a pure sample of bright, low-variability stars.

            We have also measured the related PA2/PF1 metric; we see PF1=20% of measurements are more than PA2=12.0 mmag from the mean, which means the final minimum spec of PA2=15.0 mmag. However, we see PF1=10% of measurements more than PA2=16.9 mmag, which does not meet the design spec (also PA2=15.0 mmag).

            Annotated code to compute the metrics can be found at https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb.

            Show
            jbosch Jim Bosch added a comment - Found a bug in the IPython notebook (fixed in the link in the comment immediately above). Since this has gone through so many modifications, here's a brand new writeup that I hope will indeed be the last one: Our FY15 estimate for the PA1 repeatability metric is 10.6 mmag (which meets the target of 13 mmag), measured on 12-pixel aperture fluxes. This cycle, we will use the same estimate for the procCalRep, PA1ugy, and PA1riz metrics, as these represent the same quantity computed on different datasets, and we are limited to one test dataset in FY15. This dataset is a selection of i-band HyperSuprime-Cam engineering data taken in the SDSS Stripe 82 region. This dataset consists of 30s exposures, so it is somewhat similar to projected LSST data in depth, and we've thus far put more effort into tuning the pipeline on HSC data than most other pipelines. Our current calibration approach has many limitations relative to what we ultimately plan to implement for LSST, so there is still plenty of room for improvement here: There's currently no relative calibration being run at all. We have only limited correction for chromatic effects. There's currently no allowance for zeropoint variations smaller than the scale of a CCD (including the effects of pixel area changes). We have not included the outermost CCDs in the HSC focal plane in the processing (where the geometric factor is largest), however, so this this should not play as large a role as it otherwise might. We also use a much simpler sample selection than that proposed by the SRD; the SRD suggests using colors and variability to ensure a pure sample of bright, low-variability stars. We have also measured the related PA2/PF1 metric; we see PF1=20% of measurements are more than PA2=12.0 mmag from the mean, which means the final minimum spec of PA2=15.0 mmag. However, we see PF1=10% of measurements more than PA2=16.9 mmag, which does not meet the design spec (also PA2=15.0 mmag). Annotated code to compute the metrics can be found at https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb .
            Hide
            swinbank John Swinbank added a comment -

            Hey Jim Bosch & Simon Krughoff – I am not planning to schedule any more work on the FY15 KPMs unless higher authority demands it (there might need to be a formal report produced at some point, but let's wait for requirements). I stuck all the KPM epics on the DRP side to "in review" by Mario & K-T. Since the photometric repeatability KPMs are nominally AP rather than DRP I didn't want to just do the same here, but I figured I'd let you know my thinking.

            Show
            swinbank John Swinbank added a comment - Hey Jim Bosch & Simon Krughoff – I am not planning to schedule any more work on the FY15 KPMs unless higher authority demands it (there might need to be a formal report produced at some point, but let's wait for requirements). I stuck all the KPM epics on the DRP side to "in review" by Mario & K-T. Since the photometric repeatability KPMs are nominally AP rather than DRP I didn't want to just do the same here, but I figured I'd let you know my thinking.
            Hide
            ktl Kian-Tat Lim added a comment -

            I think https://confluence.lsstcorp.org/pages/viewpage.action?pageId=41785659 is enough of a report. As far as I'm concerned, you can close all of these issues/epics.

            Show
            ktl Kian-Tat Lim added a comment - I think https://confluence.lsstcorp.org/pages/viewpage.action?pageId=41785659 is enough of a report. As far as I'm concerned, you can close all of these issues/epics.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                krughoff Simon Krughoff
                Watchers:
                Jim Bosch, John Swinbank, Kian-Tat Lim, Robert Lupton, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel