Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20161

Run tract-based fgcmcal on RC2 dataset

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      3
    • Epic Link:
    • Sprint:
      DRP S19-6b, DRP F19-1, DRP F19-2
    • Team:
      Data Release Production

      Description

      In conjunction with DM-16704 fgcmcal needs to be run and evaluated on the HSC RC2 dataset, including a comparison to the performance of the full global fit (DM-18015).

        Attachments

        1. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_g.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_g.png
          91 kB
        2. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_i.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_i.png
          92 kB
        3. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_r.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_r.png
          91 kB
        4. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_y.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_y.png
          93 kB
        5. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_z.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_crunched_z.png
          90 kB
        6. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_g.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_g.png
          86 kB
        7. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_i.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_i.png
          90 kB
        8. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_r.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_r.png
          88 kB
        9. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_y.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_y.png
          88 kB
        10. fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_z.png
          fgcmFitCycleHscCookbook-009615_cycle03_sigfgcm_reserved-stars_z.png
          87 kB
        11. fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_g.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_g.png
          95 kB
        12. fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_i.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_i.png
          92 kB
        13. fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_r.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_r.png
          90 kB
        14. fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_y.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_y.png
          91 kB
        15. fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_z.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigfgcm_reserved-stars_crunched_z.png
          92 kB
        16. fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_g.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_g.png
          80 kB
        17. fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_i.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_i.png
          84 kB
        18. fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_r.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_r.png
          86 kB
        19. fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_y.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_y.png
          82 kB
        20. fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_z.png
          fgcmFitCycleHscCookbook-009697_cycle03_sigmaref_z.png
          85 kB
        21. fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_g.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_g.png
          91 kB
        22. fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_i.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_i.png
          91 kB
        23. fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_r.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_r.png
          93 kB
        24. fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_y.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_y.png
          91 kB
        25. fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_z.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigfgcm_reserved-stars_crunched_z.png
          89 kB
        26. fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_g.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_g.png
          86 kB
        27. fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_i.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_i.png
          82 kB
        28. fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_r.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_r.png
          85 kB
        29. fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_y.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_y.png
          80 kB
        30. fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_z.png
          fgcmFitCycleHscCookbook-009813_cycle04_sigmaref_z.png
          83 kB

          Issue Links

            Activity

            Hide
            erykoff Eli Rykoff added a comment -

            I have now successfully run the new tract mode FGCM from DM-16704 on the 3 tracts from RC2 (9615, 9697, 9813). The fgcm_photoCalib files are on lsst-dev01 in a rerun at /datasets/hsc/repo/rerun/private/erykoff/fgcm_tract_w_2019_30/fgcmtract-run4.

            The runtime was approximately 30 minutes for each of the tracts (calibrating all of grizy simultaneously), and approximately 70-75% of that time was spent reading in the calexp sources.

            I have attached some plots here demonstrating the internal metrics. I think I need to run Lauren MacArthur's visit check code as independent validation, and I'd also love to do coadds and have that validated as well.

            The following table summarizes the important numbers. "int" refers to internal star repeatability in mmag. These are for bright (s/n > 100) reserved stars (not used in the fit), and are after subtracting out the typical photometric error. "ref" refers to reference star consistency in mmag. These are for bright-ish (s/n > 50) PS1 reference stars, and are the total value (e.g., does not subtract off photometric errors).

            Band 9615 (int) 9615 (ref) 9697 (int) 9697 (ref) 9813 (int) 9813 (ref)
            g 6.4 22.7 8.3 27.9 7.9 27.7
            r 5.9 21.9 6.4 21.2 9.2 28.2
            i 5.3 18.2 6.4 16.2 6.3 16.7
            z 4.6 20.0 5.3 20.4 5.1 18.5
            y 5.9 24.3 6.0 26.0 6.7 29.2

            The plots also show these broken down by the 25%/50%/25% bluest, middle, and reddest stars.






            Show
            erykoff Eli Rykoff added a comment - I have now successfully run the new tract mode FGCM from DM-16704 on the 3 tracts from RC2 (9615, 9697, 9813). The fgcm_photoCalib files are on lsst-dev01 in a rerun at /datasets/hsc/repo/rerun/private/erykoff/fgcm_tract_w_2019_30/fgcmtract-run4 . The runtime was approximately 30 minutes for each of the tracts (calibrating all of grizy simultaneously), and approximately 70-75% of that time was spent reading in the calexp sources. I have attached some plots here demonstrating the internal metrics. I think I need to run Lauren MacArthur 's visit check code as independent validation, and I'd also love to do coadds and have that validated as well. The following table summarizes the important numbers. "int" refers to internal star repeatability in mmag. These are for bright (s/n > 100) reserved stars (not used in the fit), and are after subtracting out the typical photometric error. "ref" refers to reference star consistency in mmag. These are for bright-ish (s/n > 50) PS1 reference stars, and are the total value (e.g., does not subtract off photometric errors). Band 9615 (int) 9615 (ref) 9697 (int) 9697 (ref) 9813 (int) 9813 (ref) g 6.4 22.7 8.3 27.9 7.9 27.7 r 5.9 21.9 6.4 21.2 9.2 28.2 i 5.3 18.2 6.4 16.2 6.3 16.7 z 4.6 20.0 5.3 20.4 5.1 18.5 y 5.9 24.3 6.0 26.0 6.7 29.2 The plots also show these broken down by the 25%/50%/25% bluest, middle, and reddest stars.
            Hide
            lauren Lauren MacArthur added a comment - - edited

            Do let me know if you would like me to run the visitAnalysis scripts on your outputs (assuming they are on lsst-dev, ah, yes, they are!).  I offer in part as I would be using a not-fully-committed branch for DM-19189 that implements the S/N-based selection for computing metrics that I'm sure you would prefer (to the current mag-based selection on master).  The mutual benefit might work as I do plan to ask you to review that ticket, and any outputs I produce for you here could be the example output for you to view as reviewer 

            Show
            lauren Lauren MacArthur added a comment - - edited Do let me know if you would like me to run the visitAnalysis scripts on your outputs (assuming they are on lsst-dev, ah, yes, they are!) .  I offer in part as I would be using a not-fully-committed branch for DM-19189 that implements the S/N-based selection for computing metrics that I'm sure you would prefer (to the current mag-based selection on master).  The mutual benefit might work as I do plan to ask you to review that ticket, and any outputs I produce for you here could be the example output for you to view as reviewer 
            Hide
            erykoff Eli Rykoff added a comment -

            Lauren MacArthur Yes they are! In order to read them, you need the DM-16704 branches of obs_base and obs_subaru which define the fgcm_tract_photoCalib dataset. But this should be fully compatible with jointcal_photoCalib, with the same dataId requirements.
            I definitely think we want to do this with the S/N based selection, and I'm happy to review that ticket when it's ready. Thanks!

            Show
            erykoff Eli Rykoff added a comment - Lauren MacArthur Yes they are! In order to read them, you need the DM-16704 branches of obs_base and obs_subaru which define the fgcm_tract_photoCalib dataset. But this should be fully compatible with jointcal_photoCalib , with the same dataId requirements. I definitely think we want to do this with the S/N based selection, and I'm happy to review that ticket when it's ready. Thanks!
            Hide
            lauren Lauren MacArthur added a comment -

            Ah, right, I'll need to update the scripts in pipe_analysis to have the option of applying your fgcm photometric calibrations in lieu of the jointcal ones. What should be done about astrometric calibration in this case (can the jointcal ones be used)? I'll go ahead and make a ticket for that.

            Show
            lauren Lauren MacArthur added a comment - Ah, right, I'll need to update the scripts in pipe_analysis to have the option of applying your fgcm photometric calibrations in lieu of the jointcal ones. What should be done about astrometric calibration in this case (can the jointcal ones be used)? I'll go ahead and make a ticket for that.
            Hide
            erykoff Eli Rykoff added a comment -

            I think you should be able to use the jointcal astrometric calibration, or even the original from the calexps. And I see that the ticket is DM-21052, thanks!

            Show
            erykoff Eli Rykoff added a comment - I think you should be able to use the jointcal astrometric calibration, or even the original from the calexps. And I see that the ticket is DM-21052 , thanks!
            Hide
            erykoff Eli Rykoff added a comment -

            The fgcmcal code has successfully run, so I'm closing this ticket. If there are downstream issues we can file more tickets for improvements.

            Show
            erykoff Eli Rykoff added a comment - The fgcmcal code has successfully run, so I'm closing this ticket. If there are downstream issues we can file more tickets for improvements.

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Watchers:
              Eli Rykoff, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: