Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5301

Compare LSST and HSC pipelines through through single-frame processing

    Details

    • Story Points:
      10
    • Sprint:
      DRP W16-7, DRP X16-1, DRP X16-2, DRP X16-3
    • Team:
      Data Release Production

      Description

      Following DM-2984, we are confident that the ISR performed by the LSST and HSC stacks is equivalent. Extend that work to cover the whole of single-frame processing (ProcessCcdTask).

      There are two deliverables for this issue:

      • A collections of plots and/or comparison scripts that run on the pipeline outputs that can be used to compare their quality in all the ways we think matter. This should probably be in a new repo. Some of these tests should compare the results of the two pipelines against each other (as that's easier to do), but others should be useful for tracking the quality of the pipeline even after we abandon the HSC fork.
      • A set of new JIRA issues that capture the code transfers we think we will need to get the LSST pipeline up to the same level of quality as the HSC one.

        Attachments

          Issue Links

            Activity

            Hide
            rearmstr Bob Armstrong added a comment -

            Here are comparisons of the pre-aperture corrected fluxes for visit 1322.

            Gaussian Flux

            PSF Flux

            I think this looks pretty good overall. We already saw in DM-4960 that a slightly different selection of stars can cause a 0.01 change in the PSF flux. These are the cause of the differences we see in the PSF flux that we see on the focal plane.

            Show
            rearmstr Bob Armstrong added a comment - Here are comparisons of the pre-aperture corrected fluxes for visit 1322. Gaussian Flux PSF Flux I think this looks pretty good overall. We already saw in DM-4960 that a slightly different selection of stars can cause a 0.01 change in the PSF flux. These are the cause of the differences we see in the PSF flux that we see on the focal plane.
            Hide
            rearmstr Bob Armstrong added a comment -

            If we now look at the same plots after aperture corrections you get:

            Gaussian Flux

            PSF Flux

            The variations seen here could be from using a different set of stars to derive the aperture corrections.

            Show
            rearmstr Bob Armstrong added a comment - If we now look at the same plots after aperture corrections you get: Gaussian Flux PSF Flux The variations seen here could be from using a different set of stars to derive the aperture corrections.
            Hide
            rearmstr Bob Armstrong added a comment -

            Here are a few more examples from a different visit (1254). I also changed a few of the parameter settings that were different.

            This is the pre-aperture corrected magnitudes:

            Now with aperture corrections:

            And the difference as a function of position

            The scatter is less than 1% after the corrections for the psf and gaussian magnitudes. These differences are due to changes in the selection of psf stars for the psf and the aperture corrections.

            Here are more plots that were derived for stars that were selected for the psf
            difference in the 17 pixel aperture flux:

            difference of the e1 components of the ellipticity between the two:

            and the difference in centroids:

            Given these results I am going to close this issue.

            Show
            rearmstr Bob Armstrong added a comment - Here are a few more examples from a different visit (1254). I also changed a few of the parameter settings that were different. This is the pre-aperture corrected magnitudes: Now with aperture corrections: And the difference as a function of position The scatter is less than 1% after the corrections for the psf and gaussian magnitudes. These differences are due to changes in the selection of psf stars for the psf and the aperture corrections. Here are more plots that were derived for stars that were selected for the psf difference in the 17 pixel aperture flux: difference of the e1 components of the ellipticity between the two: and the difference in centroids: Given these results I am going to close this issue.

              People

              • Assignee:
                rearmstr Bob Armstrong
                Reporter:
                swinbank John Swinbank
                Watchers:
                Bob Armstrong, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: