Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-811

Create env vars for default repo location(s) at NCSA and the summit

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Adopted
    • Resolution: Unresolved
    • Component/s: DM
    • Labels:
      None

      Description

      Currently, code has no way of knowing whether to use /repo/main/ or /repo/<instrument> to instantiate a butler without doing a bunch of os.path.exists(), not to mention that hard coding paths is bad in itself.

      Proposal: create three env vars for login shells and notebooks for LATISS_REPO_URI, COMCAM_REPO_URI and LSST_REPO_URI in such a way that they can be used for butler instantiation.

      Why three? Because NCSA has /repo/main, but on the summit it's three different repos.

      Actual solution: Modify butler to be able to look up labels in a configuration file. The configuration file is pointed at by a single environment variable DAF_BUTLER_REPOSITORY_INDEX.

        Attachments

          Issue Links

            Activity

            Hide
            ktl Kian-Tat Lim added a comment -

            The environment variable would be set per instance for notebooks via deployment configuration in github.com/lsst-sqre/phalanx. CSCs will not use that mechanism. For Unix logins at NCSA, test stands, and eventually USDF, it may be necessary for users to add this to their personal shell startup file, although at least at NCSA and likely the test stands it should be possible to add this to a common startup file.

            Show
            ktl Kian-Tat Lim added a comment - The environment variable would be set per instance for notebooks via deployment configuration in github.com/lsst-sqre/phalanx. CSCs will not use that mechanism. For Unix logins at NCSA, test stands, and eventually USDF, it may be necessary for users to add this to their personal shell startup file, although at least at NCSA and likely the test stands it should be possible to add this to a common startup file.
            Hide
            tjenness Tim Jenness added a comment -

            I've created DM-32491 for the butler side of this. Merlin Fisher-Levine this RFC will then also need tickets for creating these YAML files and tickets for setting the relevant environment variable and deploying these YAML files on various platforms (although you probably only want to create a ticket for supporting the summit)

            Show
            tjenness Tim Jenness added a comment - I've created DM-32491 for the butler side of this. Merlin Fisher-Levine this RFC will then also need tickets for creating these YAML files and tickets for setting the relevant environment variable and deploying these YAML files on various platforms (although you probably only want to create a ticket for supporting the summit)
            Hide
            tjenness Tim Jenness added a comment -

            I have written a community post to discuss the repository naming conventions: https://community.lsst.org/t/naming-butler-repositories/6112/1

            Show
            tjenness Tim Jenness added a comment - I have written a community post to discuss the repository naming conventions: https://community.lsst.org/t/naming-butler-repositories/6112/1
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Kian-Tat Lim Other than the one remaining ticket to set this for OCPS (DM-34145), I think this is now done everywhere. I think it's fair to say we shipped this as it's now in widespread use. If you agree I'll click the button.

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Kian-Tat Lim Other than the one remaining ticket to set this for OCPS ( DM-34145 ), I think this is now done everywhere. I think it's fair to say we shipped this as it's now in widespread use. If you agree I'll click the button.
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Kian-Tat Lim it looks like DM-34145 still isn't closed, but I don't think that's relevant for closing this RFC as Done. Can I click that now?

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Kian-Tat Lim it looks like DM-34145 still isn't closed, but I don't think that's relevant for closing this RFC as Done. Can I click that now?

              People

              Assignee:
              mfisherlevine Merlin Fisher-Levine
              Reporter:
              mfisherlevine Merlin Fisher-Levine
              Watchers:
              Hsin-Fang Chiang, John Parejko, Kian-Tat Lim, Merlin Fisher-Levine, Michelle Gower, Patrick Ingraham, Robert Lupton, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Planned End:

                  Jenkins

                  No builds found.