Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15751

Configure ability to build coadds with either Jointcal or meas_mosaic

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
      None

      Description

      John, I added you as a watcher in case I was duplicating effort.

      Plan is to make lightweight task that can "applyJointcalResults" to calexps (based on https://github.com/lsst/jointcal/blob/master/python/lsst/jointcal/jointcalCoadd.py)  and make choice of meas_mosaic or jointcal retargetable.  Need to add the ability to do apply the photometric calibration in addition to the WCS (like https://github.com/lsst/meas_mosaic/blob/master/python/lsst/meas/mosaic/updateExposure.py#L47) 

       

      Outstanding question: do we want to be able to mix and match? Photo from one and WCS from another?

       

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment - - edited
            Show
            Parejkoj John Parejko added a comment - - edited New Jenkins run, now that I think I've fixed ci_hsc: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29050/pipeline
            Hide
            Parejkoj John Parejko added a comment -

            Yusra AlSayyad: Jenkins passed through ci_hsc, so it's time for review!

            Show
            Parejkoj John Parejko added a comment - Yusra AlSayyad : Jenkins passed through ci_hsc, so it's time for review!
            Hide
            yusra Yusra AlSayyad added a comment -

            Couple notes:
            (1) CoaddDriver.py takes longer (working on quantifying how much to taking out effects of GPFS performance.)
            (2) QA: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2018_44_jointcal vs. https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2018_44

            • Overall, QA plots look better for deep areas and fainter stars.
            • Lauren MacArthur noticed that the brightest stars with calib_psf_used have cmodel fluxes > psf fluxes (=> are classifed as galaxies) in jointcal and not meas mosaic. Spatially this appears in the areas with 1-2 visits and ghosting around very bright stars. I'm not sure how jointcal and meas_mosaic would differ if they use the same input catalogs. By eye the coadds look identical and differences in clipping very minimal in these problematic areas.
            • Recommendation at this point would be to decouple this ticket from the decision of using jointcal for obs_subaru add useMeasMosaic=True to obs_subaru and a ticket to remove it when we're ready. We can take the decision making process to #subaru-hsc.

            Finally, obs_lsstCam (which copied all the configs over from obs_subaru initially)

            Show
            yusra Yusra AlSayyad added a comment - Couple notes: (1) CoaddDriver.py takes longer (working on quantifying how much to taking out effects of GPFS performance.) (2) QA: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2018_44_jointcal vs. https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2018_44 Overall, QA plots look better for deep areas and fainter stars. Lauren MacArthur noticed that the brightest stars with calib_psf_used have cmodel fluxes > psf fluxes (=> are classifed as galaxies) in jointcal and not meas mosaic. Spatially this appears in the areas with 1-2 visits and ghosting around very bright stars. I'm not sure how jointcal and meas_mosaic would differ if they use the same input catalogs. By eye the coadds look identical and differences in clipping very minimal in these problematic areas. Recommendation at this point would be to decouple this ticket from the decision of using jointcal for obs_subaru add useMeasMosaic=True to obs_subaru and a ticket to remove it when we're ready. We can take the decision making process to #subaru-hsc. Finally, obs_lsstCam (which copied all the configs over from obs_subaru initially)
            Hide
            Parejkoj John Parejko added a comment - - edited

            Thanks for the review.

            Changes in what happens around bright vs. faint stars might be due to differences in source selection: I don't know how meas_mosaic does it's source selection, but jointcal uses the AstrometrySourceSelector which selects every "good" "non-blended" source with s/n>10, whether star or galaxy. You can play with configuring that differently, if you think incorrect sources are being included near bright stars.

            New post-review Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29063/pipeline

            Show
            Parejkoj John Parejko added a comment - - edited Thanks for the review. Changes in what happens around bright vs. faint stars might be due to differences in source selection: I don't know how meas_mosaic does it's source selection, but jointcal uses the AstrometrySourceSelector which selects every "good" "non-blended" source with s/n>10 , whether star or galaxy. You can play with configuring that differently, if you think incorrect sources are being included near bright stars. New post-review Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29063/pipeline
            Hide
            Parejkoj John Parejko added a comment -

            Thank you for the review, and for looking at my subsequent updates.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thank you for the review, and for looking at my subsequent updates. Merged and done.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                yusra Yusra AlSayyad
                Reviewers:
                Yusra AlSayyad
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel