Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11785

Run jointcal on acceptance data

    Details

    • Story Points:
      8
    • Sprint:
      Alert Production F17 - 9, Alert Production F17 - 10, Alert Production F17 - 11, AP S18-1, AP S18-2, AP S18-3, AP S18-4, AP S18-5, AP S18-6, AP F18-1, AP F18-2
    • Team:
      Alert Production

      Description

      Run jointcal on the acceptance data, and produce catalogs that are ready to be validated via validate_drp.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Note that we are going to have to skip filter NB0921, because it does not have any metrics defined in validate_drp.

            Show
            Parejkoj John Parejko added a comment - Note that we are going to have to skip filter NB0921, because it does not have any metrics defined in validate_drp.
            Hide
            swinbank John Swinbank added a comment -

            Per discussion of 2017-12-04, we agreed that the success criterion for this ticket is to generate the Jointcal run which will actually be used for comparison with meas_mosaic. That will mean (has meant) running Jointcal on the data multiple times, figuring out what breaks, and filing blockers.

            Show
            swinbank John Swinbank added a comment - Per discussion of 2017-12-04, we agreed that the success criterion for this ticket is to generate the Jointcal run which will actually be used for comparison with meas_mosaic. That will mean (has meant) running Jointcal on the data multiple times, figuring out what breaks, and filing blockers.
            Hide
            Parejkoj John Parejko added a comment -

            Latest news here: Pierre Astier's work on outlier rejection have helped photometry fits complete, but I still have NaNs sneaking in, apparently from NaN source catalog centroid_sigma values that aren't getting rejected by the astrometrySourceSelector (e.g., Centroid_flag is still False).

            Show
            Parejkoj John Parejko added a comment - Latest news here: Pierre Astier 's work on outlier rejection have helped photometry fits complete, but I still have NaNs sneaking in, apparently from NaN source catalog centroid_sigma values that aren't getting rejected by the astrometrySourceSelector (e.g., Centroid_flag is still False).
            Hide
            Parejkoj John Parejko added a comment -

            With the work done on DM-14786, I think this ticket may be finally ready to close? I have uploaded summary statistics to DM-11787.

            The code to run the jointcal and validate_drp jobs lives in jointcal_compare. However, validate_drp has moved on enough since the meas_mosaic data was originally processed that I can no longer run validate_drp on the meas_mosaic output that lives in /datasets/hsc/repo/rerun/private/lauren/DM-11786, nor can I work with the old .json files produced from my old validate_drp runs on that data. I do still have the .rst summary files (as uploaded to DM-11787), which will probably be enough for these purposes.

            Show
            Parejkoj John Parejko added a comment - With the work done on DM-14786 , I think this ticket may be finally ready to close? I have uploaded summary statistics to DM-11787 . The code to run the jointcal and validate_drp jobs lives in jointcal_compare . However, validate_drp has moved on enough since the meas_mosaic data was originally processed that I can no longer run validate_drp on the meas_mosaic output that lives in /datasets/hsc/repo/rerun/private/lauren/ DM-11786 , nor can I work with the old .json files produced from my old validate_drp runs on that data. I do still have the .rst summary files (as uploaded to DM-11787 ), which will probably be enough for these purposes.
            Hide
            swinbank John Swinbank added a comment -

            Per discussion of 2018-08-06, we believe that this is done — there's now a complete Jointcal run for comparing with meas_mosaic. Further refinements to that will be handled on separate tickets.

            Show
            swinbank John Swinbank added a comment - Per discussion of 2018-08-06, we believe that this is done — there's now a complete Jointcal run for comparing with meas_mosaic. Further refinements to that will be handled on separate tickets.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Jim Bosch, John Parejko, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel