Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18635

Investigate jointcal/processCcd photometry models with Run2.1i data

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      3
    • Epic Link:
    • Sprint:
      DRP S19-4
    • Team:
      Data Release Production

      Description

      DESC reported that running jointcal on Run2.1i did not improve the photometric repeatability (esp. wrt the reference catalog) relative to processCcd. This is not surprising because:
      1) The reference catalogs are near perfect and to full depth < 23. They essentially are the input catalogs, and therefore jointcal's benefit of bootstrapping deeper stars is not realized.
      2) There is no vignetting or sub-ccd throughput variability in the imsim images in Run2.1i. Therefore, the photoCalTask model of "1 zeropoint per ccd" is the correct model, and jointcal's ability to have variable throughput across the focal plane and per ccd is not needed.

      However, various test plots (either via James Chiang or the visitAnalysis scripts) show that the calibrated photometry is only consistent with the reference catalogs at the 13-16 mmag level, which is much larger than I would expect for clean simulated data for bright stars (basically, I would expect few mmag consistency).

      This ticket is to describe work to see where in processCcd/jointcal things are going wrong.

        Attachments

          Issue Links

            Activity

            Hide
            erykoff Eli Rykoff added a comment -

            I started with recreating a simple photometric calibration of a single visit, 204706. Only stars from the reference and source catalog were used. Note that selecting stars is 100% essential – some quick tests allowing galaxies into the computation made things go totally off the rails.
            Turns out that the zeropoints as computed from PhotoCalTask were totally consistent with what I was computing, so the plots I will show are from PhotoCalTask.
            The first indication that things are okay is that the zeropoints are consistent at the <2mmag level across the full visit. This implies the uniformity must be at the ~2mmag level (assuming a perfect reference catalog, which we have). See
            Next, we can look at the offsets between the calibrated magnitudes and the reference magnitudes (stars only!) as a function of reference mag: Things look good at the bright end, with some outliers, but the noisy aperture magnitudes really start dominating at refMag>~18. The default cut of 20.5 will be dominated by photometric noise and other contamination, and not the fundamental photometric calibration performance.
            Taking just the refMag<18 sources, the residuals look like: Where sigma_tot is the width of the Gaussian fit to the distribution and sigma_int subtracts off the typical photometric error of the stars in the sample. We're getting good performance at the ~3mmag level.
            Finally, there are no obvious problems across the focal plane:

            In conclusion:

            • The photometric calibration performance is fine, in terms of the consistency with the reference magnitudes at the bright end when the photometric errors are sub-dominant.
            • The photometric model in PhotoCalTask is "correct" for these data, and with a perfect reference catalog there is nothing else for something like jointcal to do.
            • There are some outliers even at the bright end. Robust statistics are needed to avoid being dominated by the large outliers. (I have checked and both IQD and MAD give numbers consistent with my Gaussian fits)
            • There might be additional scatter at fainter magnitudes than expected from the photometric errors, but this might just be due to using a big aperture. I consider these questions as to be outside the scope of this ticket, since they are not something that PhotoCalTask or jointcal can modify.
            • There is a remaining question of why the numbers from Lauren MacArthur's visitAnalysis scripts are giving larger errors than this. Maybe the default is pushing too faint to get at the intrinsic scatter due to errors in photometric calibration?
            Show
            erykoff Eli Rykoff added a comment - I started with recreating a simple photometric calibration of a single visit, 204706. Only stars from the reference and source catalog were used. Note that selecting stars is 100% essential – some quick tests allowing galaxies into the computation made things go totally off the rails. Turns out that the zeropoints as computed from PhotoCalTask were totally consistent with what I was computing, so the plots I will show are from PhotoCalTask . The first indication that things are okay is that the zeropoints are consistent at the <2mmag level across the full visit. This implies the uniformity must be at the ~2mmag level (assuming a perfect reference catalog, which we have). See Next, we can look at the offsets between the calibrated magnitudes and the reference magnitudes (stars only!) as a function of reference mag: Things look good at the bright end, with some outliers, but the noisy aperture magnitudes really start dominating at refMag>~18. The default cut of 20.5 will be dominated by photometric noise and other contamination, and not the fundamental photometric calibration performance. Taking just the refMag<18 sources, the residuals look like: Where sigma_tot is the width of the Gaussian fit to the distribution and sigma_int subtracts off the typical photometric error of the stars in the sample. We're getting good performance at the ~3mmag level. Finally, there are no obvious problems across the focal plane: In conclusion: The photometric calibration performance is fine, in terms of the consistency with the reference magnitudes at the bright end when the photometric errors are sub-dominant. The photometric model in PhotoCalTask is "correct" for these data, and with a perfect reference catalog there is nothing else for something like jointcal to do. There are some outliers even at the bright end. Robust statistics are needed to avoid being dominated by the large outliers. (I have checked and both IQD and MAD give numbers consistent with my Gaussian fits) There might be additional scatter at fainter magnitudes than expected from the photometric errors, but this might just be due to using a big aperture. I consider these questions as to be outside the scope of this ticket, since they are not something that PhotoCalTask or jointcal can modify. There is a remaining question of why the numbers from Lauren MacArthur 's visitAnalysis scripts are giving larger errors than this. Maybe the default is pushing too faint to get at the intrinsic scatter due to errors in photometric calibration?

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Watchers:
              Eli Rykoff, James Chiang, John Parejko, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.