Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15617

Process HSC PDR1 data with Jointcal as of 2018-08-31.

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Use the current (2018-08-31) version of Jointcal on the latest version of the HSC PDR1 reprocessing.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Running jointcal master on DM-13666 output now.

            Show
            Parejkoj John Parejko added a comment - Running jointcal master on DM-13666 output now.
            Hide
            Parejkoj John Parejko added a comment -

            jointcal run completed over the weekend, all final chi2s were reasonable. I've modified my validate_drp execution scripts now, and have submitted those jobs. Will post here when finished.

            Show
            Parejkoj John Parejko added a comment - jointcal run completed over the weekend, all final chi2s were reasonable. I've modified my validate_drp execution scripts now, and have submitted those jobs. Will post here when finished.
            Hide
            Parejkoj John Parejko added a comment - - edited

            Validate_drp output complete (had to rerun a few of them due to out of memory problems running 5 per node), and summarized in the attached tarball. I modified the plotting script to make three sets of plots: single->jointcal, single->mosaic, mosaic->jointcal, to help with the overall comparison, and I tried to keep the plot limits easily comparable between the two jointcal plots so you can easily flip between them.

            According to validate_drp, only one jointcal astrometric result on these data exceeded an outlier metric (AF1 on HSC-R 9559), and none exceeded the repeatability metrics. For this run, I used the config defaults, which means the visit polynomial is 5th order for astrometry and 7th order for photometry: is it worth running with higher order for either of those to see if we can improve further?

            I'll try to compute the final summary statistics described on the acceptance test page. John Swinbank: should I make a ticket to do that work on?

            I've put this "in review": Jim Bosch, please look at the attached plots and tables. It's up to you whether you want to review the changes I made to the slurm and plotting scripts on the associated PR to jointcal_compare.

            Show
            Parejkoj John Parejko added a comment - - edited Validate_drp output complete (had to rerun a few of them due to out of memory problems running 5 per node), and summarized in the attached tarball. I modified the plotting script to make three sets of plots: single->jointcal, single->mosaic, mosaic->jointcal, to help with the overall comparison, and I tried to keep the plot limits easily comparable between the two jointcal plots so you can easily flip between them. According to validate_drp, only one jointcal astrometric result on these data exceeded an outlier metric (AF1 on HSC-R 9559), and none exceeded the repeatability metrics. For this run, I used the config defaults, which means the visit polynomial is 5th order for astrometry and 7th order for photometry: is it worth running with higher order for either of those to see if we can improve further? I'll try to compute the final summary statistics described on the acceptance test page. John Swinbank : should I make a ticket to do that work on? I've put this "in review": Jim Bosch , please look at the attached plots and tables. It's up to you whether you want to review the changes I made to the slurm and plotting scripts on the associated PR to jointcal_compare .
            Hide
            jbosch Jim Bosch added a comment -

            is it worth running with higher order for either of those to see if we can improve further?

            I think I'd say "not at this time"; I don't think this needs to block jointcal acceptance, and I'd like to minimize those blockers.  Please create a ticket to do this in the future, but unless we have a reason to move it up, let's aim to do it after we try to include the Jacobian as a fixed term in the photometric fit, as I would definitely expect that to affect the desired polynomial order for photometry (and perhaps indirectly the order for astrometry).

            've put this "in review": Jim Bosch, please look at the attached plots and tables. It's up to you whether you want to review the changes I made to the slurm and plotting scripts on the associated PR to jointcal_compare

            Plots look good.  The high outlier fraction in all of the photometric metric measurements makes me worry a bit about the reliability of those metrics, but there are also many potential explanations for that that wouldn't be a source of worry.  Do you (or Michael Wood-Vasey?) have any guesses as to what kinds of objects we're throwing away in those?

            I have not looked at the slurm or plotting scripts, and don't think I need to.

            Show
            jbosch Jim Bosch added a comment - is it worth running with higher order for either of those to see if we can improve further? I think I'd say "not at this time"; I don't think this needs to block jointcal acceptance, and I'd like to minimize those blockers.  Please create a ticket to do this in the future, but unless we have a reason to move it up, let's aim to do it after we try to include the Jacobian as a fixed term in the photometric fit, as I would definitely expect that to affect the desired polynomial order for photometry (and perhaps indirectly the order for astrometry). 've put this "in review":  Jim Bosch , please look at the attached plots and tables. It's up to you whether you want to review the changes I made to the slurm and plotting scripts on the associated PR to  jointcal_compare Plots look good.  The high outlier fraction in all of the photometric metric measurements makes me worry a bit about the reliability of those metrics, but there are also many potential explanations for that that wouldn't be a source of worry.  Do you (or Michael Wood-Vasey ?) have any guesses as to what kinds of objects we're throwing away in those? I have not looked at the slurm or plotting scripts, and don't think I need to.
            Hide
            Parejkoj John Parejko added a comment -

            Looking at the validate_drp output (which lives at /project/parejkoj/DM-11783/DM-15617), it seems apparent that there are a lot more points included in the comparison for jointcal, vs. meas_mosaic. I don't know what this implies, although it might be related to the fact that jointcal saves all of its ccds, whereas I think meas_mosaic just saves the ccds that land on that tract. Also, the structure of some of the plots looks different, but I'm not sure how to interpret that.

            HSC-R 9559 does show a small tail in the AM1/AF1 histogram for jointcal (hence the large AF1 there), but I don't yet know why. The other plots don't appear to be odd.

            FYI, here's an rsync command to pull down all the validate_drp plots into your current directory, organized by calibration and tract:

            rsync -arv --exclude="*.json" -e"ssh" lsst-dev:/project/parejkoj/DM-11783/DM-15617/validate-* .
            

            Show
            Parejkoj John Parejko added a comment - Looking at the validate_drp output (which lives at /project/parejkoj/ DM-11783 / DM-15617 ), it seems apparent that there are a lot more points included in the comparison for jointcal, vs. meas_mosaic. I don't know what this implies, although it might be related to the fact that jointcal saves all of its ccds, whereas I think meas_mosaic just saves the ccds that land on that tract. Also, the structure of some of the plots looks different, but I'm not sure how to interpret that. HSC-R 9559 does show a small tail in the AM1/AF1 histogram for jointcal (hence the large AF1 there), but I don't yet know why. The other plots don't appear to be odd. FYI, here's an rsync command to pull down all the validate_drp plots into your current directory, organized by calibration and tract: rsync -arv --exclude="*.json" -e"ssh" lsst-dev:/project/parejkoj/DM-11783/DM-15617/validate-* .
            Hide
            Parejkoj John Parejko added a comment -

            Pasting the final summary statistics here from DM-15695, for easier reference (since they applied to this ticket's run):

            jointcal tracts that exceed mosaic metric for AF1
            HSC-Z 9558 : (1.55 + 1*2.914 = 4.464) < 4.64

            HSC-I 8524 : (2.84 + 1*2.914 = 5.754) < 6.34

            HSC-I 8521 : (0.495 + 1*2.914 = 3.409) < 4.5

            HSC-R 8521 : (0.329 + 1*2.914 = 3.243) < 4.46

            HSC-I 8523 : (1.14 + 1*2.914 = 4.054) < 5.15

            HSC-Z 8523 : (1.77 + 1*2.914 = 4.684) < 5.85

            HSC-G 9559 : (0.779 + 1*2.914 = 3.693) < 3.83

            HSC-R 9559 : (0.954 + 1*2.914 = 3.868) < 13.4
            HSC-R 9559 : (0.954 + 2*2.914 = 6.782) < 13.4
            HSC-R 9559 : (0.954 + 3*2.914 = 9.696) < 13.4

            jointcal tracts that exceed mosaic metric for AF2
            HSC-R 15831 : (0.106 + 1*2.646 = 2.752) < 3.23

            HSC-I 8523 : (1.29 + 1*2.646 = 3.936) < 4.76

            HSC-R 16009 : (0.19 + 1*2.646 = 2.836) < 3.0

            HSC-Z 9693 : (1.13 + 1*2.646 = 3.776) < 3.83

            Show
            Parejkoj John Parejko added a comment - Pasting the final summary statistics here from DM-15695 , for easier reference (since they applied to this ticket's run): jointcal tracts that exceed mosaic metric for AF1 HSC-Z 9558 : (1.55 + 1*2.914 = 4.464) < 4.64 HSC-I 8524 : (2.84 + 1*2.914 = 5.754) < 6.34 HSC-I 8521 : (0.495 + 1*2.914 = 3.409) < 4.5 HSC-R 8521 : (0.329 + 1*2.914 = 3.243) < 4.46 HSC-I 8523 : (1.14 + 1*2.914 = 4.054) < 5.15 HSC-Z 8523 : (1.77 + 1*2.914 = 4.684) < 5.85 HSC-G 9559 : (0.779 + 1*2.914 = 3.693) < 3.83 HSC-R 9559 : (0.954 + 1*2.914 = 3.868) < 13.4 HSC-R 9559 : (0.954 + 2*2.914 = 6.782) < 13.4 HSC-R 9559 : (0.954 + 3*2.914 = 9.696) < 13.4 jointcal tracts that exceed mosaic metric for AF2 HSC-R 15831 : (0.106 + 1*2.646 = 2.752) < 3.23 HSC-I 8523 : (1.29 + 1*2.646 = 3.936) < 4.76 HSC-R 16009 : (0.19 + 1*2.646 = 2.836) < 3.0 HSC-Z 9693 : (1.13 + 1*2.646 = 3.776) < 3.83

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                swinbank John Swinbank
                Reviewers:
                Jim Bosch
                Watchers:
                Jim Bosch, John Parejko, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel