Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14287

jointcal now depends on obs_cfht

    XMLWordPrintable

    Details

    • Story Points:
      0.25
    • Sprint:
      AP S18-5
    • Team:
      Alert Production

      Description

      DM-14153 moved some minimal test data out of testdata_jointcal and into jointcal/tests/data. That test data is from cfht Megacam, but I forgot to add a dependency on obs_cfht (jointcal has setupOptional on testdata_jointcal, which itself has setupRequired on the various obs packages). This only matters for installs that don't include the various testdata repos (e.g. the binary builds).

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Hsin-Fang Chiang: this is a trivial change, but it still needs a reviewer...

            Show
            Parejkoj John Parejko added a comment - Hsin-Fang Chiang : this is a trivial change, but it still needs a reviewer...
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Feels a bit weird that jointcal depends on obs_cfht.  obs_decam tests processCcd inside itself so this seems to do the opposite. I guess this is a better direction of dependency, rather than testing jointcal inside obs_cfht?

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Feels a bit weird that jointcal depends on obs_cfht.  obs_decam tests processCcd inside itself so this seems to do the opposite. I guess this is a better direction of dependency, rather than testing jointcal inside obs_cfht?
            Hide
            Parejkoj John Parejko added a comment -

            Jointcal's setupOptional requirement on testdata_jointcal is what most of the tests operate with, but I wanted to move a little bit of data (about 2MB) into jointcal itself, so it could run tests if the testdata package (1.5GB) wasn't available. Jointcal needs fully processed catalogs, so the existing testdata_X or validation_data_X packages aren't enough (like your example re: processCcd and obs_decam), because they contain raw data. Instead of trying to come up with some fake metadata and a fake focal plane model, I used something that already existed.

            Eventually, I'd like to move to using obs_lsstSim for this sort of thing, but there isn't yet appropriate lsstSim data for all of jointcal (though the DC2 simulations might be it, once they are vetted).

            Show
            Parejkoj John Parejko added a comment - Jointcal's setupOptional requirement on testdata_jointcal is what most of the tests operate with, but I wanted to move a little bit of data (about 2MB) into jointcal itself, so it could run tests if the testdata package (1.5GB) wasn't available. Jointcal needs fully processed catalogs, so the existing testdata_X or validation_data_X packages aren't enough (like your example re: processCcd and obs_decam), because they contain raw data. Instead of trying to come up with some fake metadata and a fake focal plane model, I used something that already existed. Eventually, I'd like to move to using obs_lsstSim for this sort of thing, but there isn't yet appropriate lsstSim data for all of jointcal (though the DC2 simulations might be it, once they are vetted).
            Hide
            Parejkoj John Parejko added a comment -

            Thanks for the quick review.

            Show
            Parejkoj John Parejko added a comment - Thanks for the quick review.
            Hide
            swinbank John Swinbank added a comment -

            Since this done, I don't think we need to agonize over it now. However, I do share Hsin-Fang Chiang's doubts about this.

            As far as I know, obs_cfht is only included in lsst_distrib by virtue of being part of the lsst_obs metapackage. That metapackage has an expectation that broken or difficult obs packages can be dropped from it without slowing down the rest of development (see the discussion on RFC-251). Having a core part of the stack depend on a “semi-supported” obs package feels like an unfortunate situation.

            Show
            swinbank John Swinbank added a comment - Since this done, I don't think we need to agonize over it now. However, I do share Hsin-Fang Chiang 's doubts about this. As far as I know, obs_cfht is only included in lsst_distrib by virtue of being part of the lsst_obs metapackage. That metapackage has an expectation that broken or difficult obs packages can be dropped from it without slowing down the rest of development (see the discussion on RFC-251 ). Having a core part of the stack depend on a “semi-supported” obs package feels like an unfortunate situation.
            Hide
            Parejkoj John Parejko added a comment -

            Until we have trivial test lsstSim test data that we can use in jointcal, I don't see a better solution, currently. DC2 may fix this, but I we won't know until later, and then we'll have to extract an appropriate chunk of it to be the test data.

            If that works, I'd be more than happy to remove the obs_cfht dependency.

            Show
            Parejkoj John Parejko added a comment - Until we have trivial test lsstSim test data that we can use in jointcal, I don't see a better solution, currently. DC2 may fix this, but I we won't know until later, and then we'll have to extract an appropriate chunk of it to be the test data. If that works, I'd be more than happy to remove the obs_cfht dependency.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, John Parejko, John Swinbank
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.