Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-811

Create env vars for default repo location(s) at NCSA and the summit

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Adopted
    • Resolution: Unresolved
    • Component/s: DM
    • Labels:
      None

      Description

      Currently, code has no way of knowing whether to use /repo/main/ or /repo/<instrument> to instantiate a butler without doing a bunch of os.path.exists(), not to mention that hard coding paths is bad in itself.

      Proposal: create three env vars for login shells and notebooks for LATISS_REPO_URI, COMCAM_REPO_URI and LSST_REPO_URI in such a way that they can be used for butler instantiation.

      Why three? Because NCSA has /repo/main, but on the summit it's three different repos.

      Actual solution: Modify butler to be able to look up labels in a configuration file. The configuration file is pointed at by a single environment variable DAF_BUTLER_REPOSITORY_INDEX.

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32491 Provide butler registry lookup API

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32880 Add butler registries YAML file to IDF

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33849 Add butler registries YAML file to NCSA, summit, TTS

          • Done
          links to

          Web Link Community discussion on repository labels

          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            18 older comments
            Hide
            Permalink
            ktl Kian-Tat Lim added a comment -

            The environment variable would be set per instance for notebooks via deployment configuration in github.com/lsst-sqre/phalanx. CSCs will not use that mechanism. For Unix logins at NCSA, test stands, and eventually USDF, it may be necessary for users to add this to their personal shell startup file, although at least at NCSA and likely the test stands it should be possible to add this to a common startup file.

            Show
            ktl Kian-Tat Lim added a comment - The environment variable would be set per instance for notebooks via deployment configuration in github.com/lsst-sqre/phalanx. CSCs will not use that mechanism. For Unix logins at NCSA, test stands, and eventually USDF, it may be necessary for users to add this to their personal shell startup file, although at least at NCSA and likely the test stands it should be possible to add this to a common startup file.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I've created DM-32491 for the butler side of this. Merlin Fisher-Levine this RFC will then also need tickets for creating these YAML files and tickets for setting the relevant environment variable and deploying these YAML files on various platforms (although you probably only want to create a ticket for supporting the summit)

            Show
            tjenness Tim Jenness added a comment - I've created DM-32491 for the butler side of this. Merlin Fisher-Levine this RFC will then also need tickets for creating these YAML files and tickets for setting the relevant environment variable and deploying these YAML files on various platforms (although you probably only want to create a ticket for supporting the summit)
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I have written a community post to discuss the repository naming conventions: https://community.lsst.org/t/naming-butler-repositories/6112/1

            Show
            tjenness Tim Jenness added a comment - I have written a community post to discuss the repository naming conventions: https://community.lsst.org/t/naming-butler-repositories/6112/1
            Hide
            Permalink
            mfisherlevine Merlin Fisher-Levine added a comment -

            Kian-Tat Lim Other than the one remaining ticket to set this for OCPS (DM-34145), I think this is now done everywhere. I think it's fair to say we shipped this as it's now in widespread use. If you agree I'll click the button.

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Kian-Tat Lim Other than the one remaining ticket to set this for OCPS ( DM-34145 ), I think this is now done everywhere. I think it's fair to say we shipped this as it's now in widespread use. If you agree I'll click the button.
            Hide
            Permalink
            mfisherlevine Merlin Fisher-Levine added a comment -

            Kian-Tat Lim it looks like DM-34145 still isn't closed, but I don't think that's relevant for closing this RFC as Done. Can I click that now?

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Kian-Tat Lim it looks like DM-34145 still isn't closed, but I don't think that's relevant for closing this RFC as Done. Can I click that now?

              People

              Assignee:
              mfisherlevine Merlin Fisher-Levine
              Reporter:
              mfisherlevine Merlin Fisher-Levine
              Watchers:
              Hsin-Fang Chiang, John Parejko, Kian-Tat Lim, Merlin Fisher-Levine, Michelle Gower, Patrick Ingraham, Robert Lupton, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Planned End:

                  Jenkins

                  No builds found.