Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7255

Butler Policy class needs to search for policies similar to pex_policy

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: butler
    • Labels:
      None

      Description

      Per KT:

      When loading the Policy, we need to replicate the behavior of pex_policy's GetDefaultPolicy.

      we need to load default policy from certain locations, and then override parameters with more local policies. Order, highest to lowest priority is:

      • User
      • repository
      • obs_*
      • butlerUtils

      For the in-code ones, it should only need something like: os.path.join(utils.getPackageDir(packageName), "policy", policyFilename)

      We may need a default policy somewhere else (e.g. a tests subdirectory). It could be done with an optional dirName="policy" argument. That would just be a helper method to find in-code policies.

      Also need a loadOverride() method that takes a YAML file and merges it on top of an existing policy, with what is read in overriding what was there before.

      We don't think we need inclusion semantics.
      (inclusion semantics is one of the features of pex_policy was that you could put strings starting with "@" in a paf file and have it include a sub-policy from another file.)

        Attachments

          Activity

          Hide
          npease Nate Pease added a comment -

          after this I'd like to do DM-7363

          Show
          npease Nate Pease added a comment - after this I'd like to do DM-7363
          Hide
          npease Nate Pease added a comment - - edited

          I'm hoping Tim will do the review, I included K-T because I imagine he may be interested. There's an example of the "properly" formatted yaml file in obs_test/policy/testMapper.yaml, it was part of a previous checkin.

          Show
          npease Nate Pease added a comment - - edited I'm hoping Tim will do the review, I included K-T because I imagine he may be interested. There's an example of the "properly" formatted yaml file in obs_test/policy/testMapper.yaml, it was part of a previous checkin.
          Hide
          tjenness Tim Jenness added a comment -

          Looks good to me. I think the "test we have a dict by asking for keys()" approach is a bit cute. Using collections.Mapping instance check would be more consistent with the rest of the checks in that function.

          I'd also like the assertRaises tests to be tweaked to use with.

          Show
          tjenness Tim Jenness added a comment - Looks good to me. I think the "test we have a dict by asking for keys()" approach is a bit cute. Using collections.Mapping instance check would be more consistent with the rest of the checks in that function. I'd also like the assertRaises tests to be tweaked to use with .

            People

            • Assignee:
              npease Nate Pease
              Reporter:
              npease Nate Pease
              Reviewers:
              Kian-Tat Lim, Tim Jenness
              Watchers:
              Kian-Tat Lim, Nate Pease, Tim Jenness
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel