Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10007 Provide Jointcal requirements document
  3. DM-10729

Near-term jointcal acceptance: make validate_drp use meas_mosaic outputs

    Details

    • Type: Technical task
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: validate_drp
    • Labels:
      None
    • Sprint:
      DRP S17-6, DRP F17-1, DRP F17-2, DRP F17-3, DRP F17-4
    • Team:
      Data Release Production

      Description

      validate_drp already runs astrometric and photometric repeatability of the tests we'd like to use to validate jointcal as a meas_mosaic replacement, but first we need to actually make validate_drp use meas_mosaic.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            After the matches have been made and calibrated, I don't think there's any way to tell from the resulting table which calibrations were used. I believe I've added a flag to the JSON files indicating whether that was done, but the correctness of that would indeed be vulnerable to changes in MatchedMultiVisitDataset.

            Adding a test for that makes sense, but I can't think of a good way to do that without bundling or depending on a significant amount of data. It'd of course be good to eventually get a SQuaSH that uses meas_mosaic or jointcal outputs into CI somehow, and once that's present we may not need and independent test for whether we're using those outputs, since failing to do so should produce a pretty big regression. But of course there's still work to be done before that happens.

            Show
            jbosch Jim Bosch added a comment - After the matches have been made and calibrated, I don't think there's any way to tell from the resulting table which calibrations were used. I believe I've added a flag to the JSON files indicating whether that was done, but the correctness of that would indeed be vulnerable to changes in MatchedMultiVisitDataset . Adding a test for that makes sense, but I can't think of a good way to do that without bundling or depending on a significant amount of data. It'd of course be good to eventually get a SQuaSH that uses meas_mosaic or jointcal outputs into CI somehow, and once that's present we may not need and independent test for whether we're using those outputs, since failing to do so should produce a pretty big regression. But of course there's still work to be done before that happens.
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            Yeah, I didn't have any good lightweight ideas either. Perhaps for now you can just create an issue reminding us that we should add such a test to ci_hsc or eventually lsst_ci once the later runs meas_mosaic.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - Yeah, I didn't have any good lightweight ideas either. Perhaps for now you can just create an issue reminding us that we should add such a test to ci_hsc or eventually lsst_ci once the later runs meas_mosaic .
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            These are good to merge after thinking about the following three issues:

            1. Should the boolean useJointCal be generalized to some calibration_method variable that could handle a variety of different methods that we might experiment with over the years, and even within active DRP?
            2. Can --output be made unnecessary with a Config default?
            3. Create a new issue to someday add a test to ci_hsc or lsst_ci.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - These are good to merge after thinking about the following three issues: 1. Should the boolean useJointCal be generalized to some calibration_method variable that could handle a variety of different methods that we might experiment with over the years, and even within active DRP? 2. Can --output be made unnecessary with a Config default? 3. Create a new issue to someday add a test to ci_hsc or lsst_ci .
            Hide
            jbosch Jim Bosch added a comment -

            1. Should the boolean useJointCal be generalized to some calibration_method variable that could handle a variety of different methods that we might experiment with over the years, and even within active DRP?

            By the time we develop those different methods, they'll be implemented as different SuperTasks with configurable output datasets, I think, and hopefully all of the driver code for validate_drp (probably renamed to something involving "verify") by then) will be too. So yes, eventually we should do this, but I think it'd be premature to do it now.

            2. Can --output be made unnecessary with a Config default?

            I'm afraid not; it's pretty solidly baked into pipe_tasks (which will save software versions even if you don't save any real datasets and disable config and metadata saving, as I've done). That can be turned off on the command-line by using --no-versions, I believe, but I'm not sure if even that will prevent pipe_base from creating the output directory.

            3. Create a new issue to someday add a test to ci_hsc or lsst_ci.

            DM-11571

            Show
            jbosch Jim Bosch added a comment - 1. Should the boolean useJointCal be generalized to some calibration_method variable that could handle a variety of different methods that we might experiment with over the years, and even within active DRP? By the time we develop those different methods, they'll be implemented as different SuperTasks with configurable output datasets, I think, and hopefully all of the driver code for validate_drp (probably renamed to something involving "verify") by then) will be too. So yes, eventually we should do this, but I think it'd be premature to do it now. 2. Can --output be made unnecessary with a Config default? I'm afraid not; it's pretty solidly baked into pipe_tasks (which will save software versions even if you don't save any real datasets and disable config and metadata saving, as I've done). That can be turned off on the command-line by using --no-versions , I believe, but I'm not sure if even that will prevent pipe_base from creating the output directory. 3. Create a new issue to someday add a test to ci_hsc or lsst_ci. DM-11571
            Hide
            jbosch Jim Bosch added a comment -

            Merged to master.

            Show
            jbosch Jim Bosch added a comment - Merged to master.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                Michael Wood-Vasey, Russell Owen
                Watchers:
                Jim Bosch, Michael Wood-Vasey, Russell Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel