Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5827

Compare LSST and HSC pipelines through through multi-band coadd processing

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Continue the work described in DM-5301 through the standard "multi-band" coadd processing workflow.

      Performing an end-to-end comparison of the stacks will not be possible until meas_mosaic is fully operational on LSST (DM-2674). However, until that point is reached, comparisons are still possible by either:

      • Shepherding data through meas_mosaic and coaddition on HSC, then performing further processing and measurement using the LSST stack;
      • Omitting meas_mosaic from the workflow altogether and performing end-to-end comparisons of the stacks without mosaicking.

      Obviously, neither of these will ultimately be adequate, but they should enable early identification of any major issues.

        Attachments

          Issue Links

            Activity

            Hide
            rearmstr Bob Armstrong added a comment -

            For these results I have skipped running meas_mosaic on LSST and compared the output of 1 filter through multiband processing. I ran meas_mosaic and stacked the image using the HSC software.

            Since I was starting from the same stacked image, I first tried to verify that I detected the same number of objects. However, due to a change in how a background image was computed, the number of objects was different by a few percent. To equalize the number of objects between the code bases, I modified the background estimation code to be the same. I was then able to get the same number of sources between the two pipelines.

            After detection I ran mergeCoaddDetections (which didn't do much because I only had one band) and then measureCoaddSources. Here are the results from a single patch (tract: 8523 patch:4,4).

            Here is the difference between the Psf magnitudes between the two as a function of magnitude and sky position.

            The differences here are quite small, although I can see a trend with position on the sky.

            Here are a couple of plots on the centroid differences and ellipticity.


            In most of these plots you can see some scatter at relatively bright magnitudes. These are mostly due to blended objects. Here is a plot of the magnitude difference for isolated and blended sources. These are likely getting different pixel values when we replace objects with noise which is causing these changes.

            Show
            rearmstr Bob Armstrong added a comment - For these results I have skipped running meas_mosaic on LSST and compared the output of 1 filter through multiband processing. I ran meas_mosaic and stacked the image using the HSC software. Since I was starting from the same stacked image, I first tried to verify that I detected the same number of objects. However, due to a change in how a background image was computed, the number of objects was different by a few percent. To equalize the number of objects between the code bases, I modified the background estimation code to be the same. I was then able to get the same number of sources between the two pipelines. After detection I ran mergeCoaddDetections (which didn't do much because I only had one band) and then measureCoaddSources. Here are the results from a single patch (tract: 8523 patch:4,4). Here is the difference between the Psf magnitudes between the two as a function of magnitude and sky position. The differences here are quite small, although I can see a trend with position on the sky. Here are a couple of plots on the centroid differences and ellipticity. In most of these plots you can see some scatter at relatively bright magnitudes. These are mostly due to blended objects. Here is a plot of the magnitude difference for isolated and blended sources. These are likely getting different pixel values when we replace objects with noise which is causing these changes.

              People

              • Assignee:
                rearmstr Bob Armstrong
                Reporter:
                swinbank John Swinbank
                Watchers:
                Bob Armstrong, John Swinbank, Lauren MacArthur
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel