Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25014

Coadds converted to Gen 3 without abstract_filter

    Details

    • Story Points:
      3
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      I have a Gen 3 repository that I created from https://github.com/lsst/ap_verify_ci_hits2015/tree/master/templates by running

      convert_gen2_repo_to_gen3.py lsst.obs.decam.DarkEnergyCamera --gen2root ${AP_VERIFY_CI_HITS2015_DIR}/templates/ --gen3root `pwd`/hits2015_gen3/ --config ${AP_VERIFY_CI_HITS2015_DIR}/config/convertRepo_templates.py
      

      using the w_2020_20 stack. Trying to run image differencing using w_2020_20 and these coadds gives

      ValueError: Supplied dataset type (DatasetType(deepCoadd, {abstract_filter, skymap, tract, patch}, ExposureF)) inconsistent with registry definition (DatasetType(deepCoadd, {skymap, tract, patch}, ExposureF))
      

      Since coadds should be parametrized by some kind of filter, this is presumably an error in the conversion.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Thanks, Krzysztof Findeisen - what branch of ap_verify do I need for that? I don't seem to have gotten a config/convertRepo_standalone.py from the first step with _20, and I don't see a tickets/DM-21939 branch of ap_verify. Or do I even need that step in order to do the more relevant one involving the templates?

            Show
            jbosch Jim Bosch added a comment - Thanks, Krzysztof Findeisen - what branch of ap_verify do I need for that? I don't seem to have gotten a config/convertRepo_standalone.py from the first step with _20, and I don't see a tickets/ DM-21939 branch of ap_verify. Or do I even need that step in order to do the more relevant one involving the templates?
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            A copy of standalone is available in /scratch/krzys001/hits2015_gen2/config/convertRepo_standalone.py. I do not intend to commit it.

            Edit: please ignore my previous statement. Of course you need to be able to do the conversion yourself if we're right about the cause...

            Show
            krzys Krzysztof Findeisen added a comment - - edited A copy of standalone is available in /scratch/krzys001/hits2015_gen2/config/convertRepo_standalone.py . I do not intend to commit it. Edit: please ignore my previous statement. Of course you need to be able to do the conversion yourself if we're right about the cause...
            Hide
            jbosch Jim Bosch added a comment -

            Could you chmod +rx that config directory (and file, if needed), please?

            Show
            jbosch Jim Bosch added a comment - Could you chmod +rx that config directory (and file, if needed), please?
            Hide
            krzys Krzysztof Findeisen added a comment -

            Sorry. It should be readable now.

            Show
            krzys Krzysztof Findeisen added a comment - Sorry. It should be readable now.
            Hide
            jbosch Jim Bosch added a comment -

            Krzysztof Findeisen, I think I've got this fixed - I reproduced the original problem, and was able to inspect the repo after conversion with this fix to see that the deepCoadd dataset was defined properly.

            All changes are in obs_base - I was able to find a way to make this the default behavior while still letting HSC override; see code comments.

            The last commit there is the only directly relevant one - the second one is just adding the logging I used to diagnose the problem, and the first is just minor cleanups I made en route to that.

            PR is https://github.com/lsst/obs_base/pull/249

            Jenkins is green: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/31936/pipeline/

            Show
            jbosch Jim Bosch added a comment - Krzysztof Findeisen , I think I've got this fixed - I reproduced the original problem, and was able to inspect the repo after conversion with this fix to see that the deepCoadd dataset was defined properly. All changes are in obs_base - I was able to find a way to make this the default behavior while still letting HSC override; see code comments. The last commit there is the only directly relevant one - the second one is just adding the logging I used to diagnose the problem, and the first is just minor cleanups I made en route to that. PR is https://github.com/lsst/obs_base/pull/249 Jenkins is green: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/31936/pipeline/

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                krzys Krzysztof Findeisen
                Reviewers:
                Krzysztof Findeisen
                Watchers:
                Jim Bosch, Krzysztof Findeisen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel