Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7049

Move patch/tract and config mapping definitions to daf_butlerUtils

    XMLWordPrintable

    Details

    • Story Points:
      4
    • Epic Link:
    • Sprint:
      DRP F16-2, DRP F16-3, DRP F16-4
    • Team:
      Data Release Production

      Description

      Implement RFC-204 by adding new entries for all patch/tract and config mapping definitions to .yaml files in daf_butlerUtils, and removing any such entries that are identical to the common ones from .paf files in obs* packages.

      I think the "common" entry can usually be defined by consensus between any two of obs_cfht, obs_decam, and obs_subaru (and frequently all three). If there are any patch/tract or config datasets for which no two cameras agree, I think we should use obs_subaru's definitions (but I doubt there are any such cases).

      Entries that are not identical to the common ones should not be removed on this issue (that should make this change entirely backwards compatible), but should be documented in new per-camera issues for later standardization.

        Attachments

        1. potential.mod.lists.tar
          40 kB
        2. obskeys.yaml
          0.7 kB
        3. movekeys.yaml
          1 kB
        4. mapper.diffs
          5 kB
        5. hscdiffs.tar
          90 kB
        6. hscdiffs.2.tar
          100 kB
        7. changes.summary
          3 kB

          Issue Links

            Activity

            Hide
            pgee Perry Gee added a comment -

            I notice that deepCoadd: and related exposures have "level:"
            set in conflicting ways. Is this entry needed?

            Show
            pgee Perry Gee added a comment - I notice that deepCoadd: and related exposures have "level:" set in conflicting ways. Is this entry needed?
            Hide
            jbosch Jim Bosch added a comment -

            I notice that deepCoadd: and related exposures have "level:" set in conflicting ways. Is this entry needed?

            The fact that they all work despite having different definitions suggests that it's not needed, but I don't actually understand what it does. Could you ask this on community?

            Show
            jbosch Jim Bosch added a comment - I notice that deepCoadd: and related exposures have "level:" set in conflicting ways. Is this entry needed? The fact that they all work despite having different definitions suggests that it's not needed, but I don't actually understand what it does. Could you ask this on community?
            Hide
            pgee Perry Gee added a comment -

            This came up because I was ending up with deepCoadd and deepCoad_calexp in two places for the non-HSC mappers, and I had to then choose whether to delete these entries from the other mappers (violating our previous assumption that we would not make any material changes to the other mappers on this ticket), or to add an override by those same names in the other mapper.paf files.

            Since K-T is not willing to vouch that changing the level won't affect things (and in fact, he wants to remove this entirely in later code), I vote that we punt on this issue and retain compatibility with the previous paf files for now by moving these two entries from datasets: to exposures: in all the paf files.

            Show
            pgee Perry Gee added a comment - This came up because I was ending up with deepCoadd and deepCoad_calexp in two places for the non-HSC mappers, and I had to then choose whether to delete these entries from the other mappers (violating our previous assumption that we would not make any material changes to the other mappers on this ticket), or to add an override by those same names in the other mapper.paf files. Since K-T is not willing to vouch that changing the level won't affect things (and in fact, he wants to remove this entirely in later code), I vote that we punt on this issue and retain compatibility with the previous paf files for now by moving these two entries from datasets: to exposures: in all the paf files.
            Hide
            pgee Perry Gee added a comment -

            The differerence logs for these changes will now be moved to individual RFC documents for each of the mappers.

            Show
            pgee Perry Gee added a comment - The differerence logs for these changes will now be moved to individual RFC documents for each of the mappers.
            Hide
            Parejkoj John Parejko added a comment -

            This could possibly live in obs_base.

            Show
            Parejkoj John Parejko added a comment - This could possibly live in obs_base.

              People

              Assignee:
              pgee Perry Gee
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Jim Bosch
              Watchers:
              Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Nate Pease [X] (Inactive), Paul Price, Perry Gee
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.