Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7564

Update docs to clarify that RHEL6 users always need devtoolset-3

    XMLWordPrintable

    Details

      Description

      Original report:

      Installing lsstsw on RHEL6 (and presumably derivatives like Scientific Linux and CentOS) fails because the built-in git client does not have https support. The "git" transport protocol does work and is a suitable alternative.

      This is a problem within bin/deploy where versiondb.git and lsst_build.git are cloned (fixed on my machine with replacing https: -> git.

      Similarly, in etc/repos.yaml I had to replace all the https: with git: for the downloads to work.

      I am not sure if this is a general solution, however. On Stack Overflow somebody noted that some places block the git: transport protocol. Though I don't know how this can be configured.

      This ticket is the result of some confusion over how RHEL6 users should configure their environment. Documentation should be examined to clarify that it is always needed regardless of newinstall vs lsstsw and regardless of whether people know they are using C++11.

        Attachments

          Issue Links

            Activity

            No builds found.
            erykoff Eli Rykoff created issue -
            Hide
            tjenness Tim Jenness added a comment -

            You have to use devtoolset-3 on your RHEL6 system to build the stack.

            Show
            tjenness Tim Jenness added a comment - You have to use devtoolset-3 on your RHEL6 system to build the stack.
            Show
            tjenness Tim Jenness added a comment - - edited See https://developer.lsst.io/services/lsst-dev.html#select-appropriate-developer-tools for some instructions or https://pipelines.lsst.io/install/newinstall.html?highlight=devtoolset#source-install-redhat-legacy devtoolset-3 comes with git 1.9.3.
            Hide
            jbosch Jim Bosch added a comment -

            Can we mark this as Won't Fix, if the documentation is already in the right place and we just hadn't pointed Eli Rykoff at it? Or are there modifications we need to make to the documentation to make this more clear?

            Show
            jbosch Jim Bosch added a comment - Can we mark this as Won't Fix, if the documentation is already in the right place and we just hadn't pointed Eli Rykoff at it? Or are there modifications we need to make to the documentation to make this more clear?
            Hide
            tjenness Tim Jenness added a comment -

            If devtoolset-3 fixed the problem then I think we just close it. I'm waiting for Eli Rykoff to indicate what the current status is. I think we have a full set of documentation that tells you to use devtoolset-3 on RHEL6.

            Show
            tjenness Tim Jenness added a comment - If devtoolset-3 fixed the problem then I think we just close it. I'm waiting for Eli Rykoff to indicate what the current status is. I think we have a full set of documentation that tells you to use devtoolset-3 on RHEL6.
            Hide
            erykoff Eli Rykoff added a comment -

            It did fix the problem. The documentation I saw about devtoolset-3 (which you helpfully pointed me to) is in the "newinstall.sh" section, but I (and others?) were told that the easiest way to get the stack is via lsstsw which doesn't have this information. I think a cross-link from lsstsw would be helpful (or, I guess, maybe I should've been told to use newinstall.sh instead?)

            Show
            erykoff Eli Rykoff added a comment - It did fix the problem. The documentation I saw about devtoolset-3 (which you helpfully pointed me to) is in the "newinstall.sh" section, but I (and others?) were told that the easiest way to get the stack is via lsstsw which doesn't have this information. I think a cross-link from lsstsw would be helpful (or, I guess, maybe I should've been told to use newinstall.sh instead?)
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Watchers Eli Rykoff, Jim Bosch, Tim Jenness [ Eli Rykoff, Jim Bosch, Tim Jenness ] Eli Rykoff, Jim Bosch, Jonathan Sick, Tim Jenness [ Eli Rykoff, Jim Bosch, Jonathan Sick, Tim Jenness ]
            Hide
            tjenness Tim Jenness added a comment -

            Ok. This has come up a couple of times. I'll adjust the title and assign mark it as a documentation ticket. All RHEL6 users have to install devtoolset-3 regardless of how they are doing the build and it seems there is some confusion over this. Currently it seems that newinstall users are told about it and lsst-dev users are told about it and C++ style guide readers are told about it. There is obviously no easy to find statement about the general issue of development environment.

            Show
            tjenness Tim Jenness added a comment - Ok. This has come up a couple of times. I'll adjust the title and assign mark it as a documentation ticket. All RHEL6 users have to install devtoolset-3 regardless of how they are doing the build and it seems there is some confusion over this. Currently it seems that newinstall users are told about it and lsst-dev users are told about it and C++ style guide readers are told about it. There is obviously no easy to find statement about the general issue of development environment.
            tjenness Tim Jenness made changes -
            Summary RHEL6 and variants don't have https support in git Update docs to clarify that RHEL6 users always need devtoolset-3
            tjenness Tim Jenness made changes -
            Component/s Stack Documentation and UX [ 12880 ]
            tjenness Tim Jenness made changes -
            Description Installing lsstsw on RHEL6 (and presumably derivatives like Scientific Linux and CentOS) fails because the built-in git client does not have https support. The "git" transport protocol does work and is a suitable alternative.

            This is a problem within bin/deploy where versiondb.git and lsst_build.git are cloned (fixed on my machine with replacing https: -> git:).

            Similarly, in etc/repos.yaml I had to replace all the https: with git: for the downloads to work.

            I am not sure if this is a general solution, however. On Stack Overflow somebody noted that some places block the git: transport protocol. Though I don't know how this can be configured.
            Original report:
            {quote}
            Installing lsstsw on RHEL6 (and presumably derivatives like Scientific Linux and CentOS) fails because the built-in git client does not have https support. The "git" transport protocol does work and is a suitable alternative.

            This is a problem within bin/deploy where versiondb.git and lsst_build.git are cloned (fixed on my machine with replacing https: -> git:).

            Similarly, in etc/repos.yaml I had to replace all the https: with git: for the downloads to work.

            I am not sure if this is a general solution, however. On Stack Overflow somebody noted that some places block the git: transport protocol. Though I don't know how this can be configured.
            {quote}

            This ticket is the result of some confusion over how RHEL6 users should configure their environment. Documentation should be examined to clarify that it is always needed regardless of newinstall vs lsstsw and regardless of whether people know they are using C++11.
            tjenness Tim Jenness made changes -
            Team SQuaRE [ 10302 ]
            Hide
            jsick Jonathan Sick added a comment -

            I'll refactor the requirements section and include that content fully on both the lsstsw and newinstall installation pages, rather than just linking from one page to the other.

            Show
            jsick Jonathan Sick added a comment - I'll refactor the requirements section and include that content fully on both the lsstsw and newinstall installation pages, rather than just linking from one page to the other.
            jsick Jonathan Sick made changes -
            Assignee Jonathan Sick [ jsick ]
            jsick Jonathan Sick made changes -
            Epic Link DM-7496 [ 26625 ]
            Hide
            jsick Jonathan Sick added a comment -

            To be clear, the 'cross-link' was there all long (https://pipelines.lsst.io/install/lsstsw.html#prerequisites) but clearly this was a UX failure on my part.

            Show
            jsick Jonathan Sick added a comment - To be clear, the 'cross-link' was there all long ( https://pipelines.lsst.io/install/lsstsw.html#prerequisites ) but clearly this was a UX failure on my part.
            Hide
            tjenness Tim Jenness added a comment - - edited

            It may be that including the words "eups distrib" in that link threw people off and if it just said "prerequisites" that would avoid confusion. Maybe a standalone prereqs pages would be better, linked from both.

            Show
            tjenness Tim Jenness added a comment - - edited It may be that including the words "eups distrib" in that link threw people off and if it just said "prerequisites" that would avoid confusion. Maybe a standalone prereqs pages would be better, linked from both.
            Hide
            jsick Jonathan Sick added a comment -

            A stand-alone pre-reqs page could be useful. One issue is that we don't want conda/docker/CernVM FS users to find the page and think it applies to their installations. There's a lot of UX improvements that can be made here, like using tab sheets to hide pre-req instructions for other platforms.

            Show
            jsick Jonathan Sick added a comment - A stand-alone pre-reqs page could be useful. One issue is that we don't want conda/docker/CernVM FS users to find the page and think it applies to their installations. There's a lot of UX improvements that can be made here, like using tab sheets to hide pre-req instructions for other platforms.
            Hide
            erykoff Eli Rykoff added a comment -

            You're right, it is there. And I saw "eups distrib" and my brain went right past it.

            Show
            erykoff Eli Rykoff added a comment - You're right, it is there. And I saw "eups distrib" and my brain went right past it.
            Hide
            swinbank John Swinbank added a comment -

            Are we sure that devtoolset-3 is really the solution here?

            [swinbank@tiger-sumire ~]$ cat /etc/redhat-release
            Springdale Linux  release 6.8 (Pisa)
            [swinbank@tiger-sumire ~]$ exec scl enable devtoolset-3 bash
            [swinbank@tiger-sumire ~]$ which git
            /usr/bin/git
            [swinbank@tiger-sumire ~]$ git --version
            git version 1.8.3.1
            

            It may be that tiger-sumire is simply broken, but it's not obvious that devtoolset-3 provides an appropriate git.

            Show
            swinbank John Swinbank added a comment - Are we sure that devtoolset-3 is really the solution here? [swinbank@tiger-sumire ~]$ cat /etc/redhat-release Springdale Linux release 6.8 (Pisa) [swinbank@tiger-sumire ~]$ exec scl enable devtoolset-3 bash [swinbank@tiger-sumire ~]$ which git /usr/bin/git [swinbank@tiger-sumire ~]$ git --version git version 1.8.3.1 It may be that tiger-sumire is simply broken, but it's not obvious that devtoolset-3 provides an appropriate git.
            Hide
            tjenness Tim Jenness added a comment -

            Not sure at all. That's why I asked about it at the beginning of the ticket. Is there a RHEL6 channel for getting a git that supports https or do you have to build git from scratch?

            Show
            tjenness Tim Jenness added a comment - Not sure at all. That's why I asked about it at the beginning of the ticket. Is there a RHEL6 channel for getting a git that supports https or do you have to build git from scratch?
            Hide
            erykoff Eli Rykoff added a comment -

            You actually need not just devtoolset-3 but also git19 as shown on https://developer.lsst.io/services/lsst-dev.html#select-appropriate-developer-tools . Now, on the docs this is written as an optional, but it may be a requirement. (Certainly I needed to use git19 as well on my RHEL6 system).

            Show
            erykoff Eli Rykoff added a comment - You actually need not just devtoolset-3 but also git19 as shown on https://developer.lsst.io/services/lsst-dev.html#select-appropriate-developer-tools . Now, on the docs this is written as an optional, but it may be a requirement. (Certainly I needed to use git19 as well on my RHEL6 system).
            Hide
            swinbank John Swinbank added a comment -

            That page claims that "the Developer Toolset includes version 1.9.3 of the Git version control system", which may or may not be true. If it is, then you shouldn't need git19 as well.

            In short, while I definitely think that our documentation should be helpful, it looks as though somebody actually needs to understand what the various packages provide, rather than just shuffling the existing text around.

            Show
            swinbank John Swinbank added a comment - That page claims that "the Developer Toolset includes version 1.9.3 of the Git version control system", which may or may not be true. If it is, then you shouldn't need git19 as well. In short, while I definitely think that our documentation should be helpful, it looks as though somebody actually needs to understand what the various packages provide, rather than just shuffling the existing text around.
            Hide
            erykoff Eli Rykoff added a comment -

            That is a good point. I don't believe that is true that the devtoolset-3 contains any particular version of git.

            Show
            erykoff Eli Rykoff added a comment - That is a good point. I don't believe that is true that the devtoolset-3 contains any particular version of git.
            Hide
            erykoff Eli Rykoff added a comment -

            FYI, I now know why I didn't see the pre-reqs originally. The link I was sent to get started went here: https://developer.lsst.io/build-ci/lsstsw.html (which is also what you get as the top link if you google "lsstsw") but this is different from the "installation" entry-point here: https://pipelines.lsst.io/install/lsstsw.html

            Show
            erykoff Eli Rykoff added a comment - FYI, I now know why I didn't see the pre-reqs originally. The link I was sent to get started went here: https://developer.lsst.io/build-ci/lsstsw.html (which is also what you get as the top link if you google "lsstsw") but this is different from the "installation" entry-point here: https://pipelines.lsst.io/install/lsstsw.html
            Hide
            tjenness Tim Jenness added a comment -

            lsst-dev is a good test bed. If I enable devtoolset-3 I get git version 1.9.3 found in /opt/rh/devtoolset-3/root/usr/bin/git. No need to also enable git19.

            Show
            tjenness Tim Jenness added a comment - lsst-dev is a good test bed. If I enable devtoolset-3 I get git version 1.9.3 found in /opt/rh/devtoolset-3/root/usr/bin/git . No need to also enable git19 .
            Hide
            erykoff Eli Rykoff added a comment -

            The only thing I came up with on the list of what software is in devtoolset-3 is here: https://access.redhat.com/documentation/en-US/Red_Hat_Developer_Toolset/3/pdf/User_Guide/Red_Hat_Developer_Toolset-3-User_Guide-en-US.pdf The only git mentions are Eclipse git plugins, and it's not part of my installation. So perhaps on lsst-dev they added git to the set for simplicity?

            Show
            erykoff Eli Rykoff added a comment - The only thing I came up with on the list of what software is in devtoolset-3 is here: https://access.redhat.com/documentation/en-US/Red_Hat_Developer_Toolset/3/pdf/User_Guide/Red_Hat_Developer_Toolset-3-User_Guide-en-US.pdf The only git mentions are Eclipse git plugins, and it's not part of my installation. So perhaps on lsst-dev they added git to the set for simplicity?
            Hide
            tjenness Tim Jenness added a comment -

            I'd be surprised if they hacked the devtoolset-3 package to include git. The git19 SCL is available on lsst-dev as well.

            Show
            tjenness Tim Jenness added a comment - I'd be surprised if they hacked the devtoolset-3 package to include git . The git19 SCL is available on lsst-dev as well.
            Hide
            erykoff Eli Rykoff added a comment -

            No, but apparently somebody did. I just spent too much time looking at this. It seems that lsst-dev grabs the devtoolset-3 from the (free as in beer) channel https://www.softwarecollections.org/repos/rhscl/devtoolset-3/epel-6-x86_64 which includes devtoolset-3-git. Here at SLAC we get the devtoolset-3 from our local mirror of the official (free as in speech, I suppose) RHEL channel, rhel-x86_64-server-6-rhscl, which does not include devtoolset-3-git. Looking at the official Red Hat source download (free as in speech and beer) site: http://ftp.redhat.com/redhat/linux/enterprise/6Server/en/RHSCL/SRPMS/ it also does not include devtoolset-3-git.

            Therefore, official redhat customers will need to use the git19 package, while those using clones will not.

            I don't know if there are any other missing/extra packages, but boy that is unpleasant.

            Show
            erykoff Eli Rykoff added a comment - No, but apparently somebody did. I just spent too much time looking at this. It seems that lsst-dev grabs the devtoolset-3 from the (free as in beer) channel https://www.softwarecollections.org/repos/rhscl/devtoolset-3/epel-6-x86_64 which includes devtoolset-3-git . Here at SLAC we get the devtoolset-3 from our local mirror of the official (free as in speech, I suppose) RHEL channel, rhel-x86_64-server-6-rhscl, which does not include devtoolset-3-git . Looking at the official Red Hat source download (free as in speech and beer) site: http://ftp.redhat.com/redhat/linux/enterprise/6Server/en/RHSCL/SRPMS/ it also does not include devtoolset-3-git . Therefore, official redhat customers will need to use the git19 package, while those using clones will not. I don't know if there are any other missing/extra packages, but boy that is unpleasant.
            jsick Jonathan Sick made changes -
            Epic Link DM-7496 [ 26625 ] DM-5403 [ 23210 ]
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Jonathan Sick Is this issue still valid as we've since migrated to devtoolset-6?

            Show
            jhoblitt Joshua Hoblitt added a comment - Jonathan Sick Is this issue still valid as we've since migrated to devtoolset-6 ?
            Hide
            erykoff Eli Rykoff added a comment -

            Same issue with devtoolset-6 which I have access to at SLAC.  That is, I need to do scl enable devtoolset-6 git19 tcsh in order to access the new/correct version of git.

            Show
            erykoff Eli Rykoff added a comment - Same issue with devtoolset-6 which I have access to at SLAC.  That is, I need to do scl enable devtoolset-6 git19 tcsh in order to access the new/correct version of git.
            Hide
            tjenness Tim Jenness added a comment -

            we have:

            https://developer.lsst.io/build-ci/platforms.html

            and

            https://developer.lsst.io/coding/cpp_style_guide.html#we-are-writing-c-14

            in the dev guide but the pipelines install docs are still at v14.0 standard (https://pipelines.lsst.io/install/prereqs/centos.html?highlight=devtoolset) so don't mention devtoolset-6 (we need a "beta" version of those pages and I think that's another ticket somewhere.

            Show
            tjenness Tim Jenness added a comment - we have: https://developer.lsst.io/build-ci/platforms.html and https://developer.lsst.io/coding/cpp_style_guide.html#we-are-writing-c-14 in the dev guide but the pipelines install docs are still at v14.0 standard ( https://pipelines.lsst.io/install/prereqs/centos.html?highlight=devtoolset ) so don't mention devtoolset-6 (we need a "beta" version of those pages and I think that's another ticket somewhere.
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31024 ]
            Hide
            tjenness Tim Jenness added a comment -

            Closing this since devtoolset is no longer needed.

            Show
            tjenness Tim Jenness added a comment - Closing this since devtoolset is no longer needed.
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Won't Fix [ 10405 ]

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              erykoff Eli Rykoff
              Watchers:
              Eli Rykoff, Jim Bosch, John Swinbank, Jonathan Sick, Joshua Hoblitt, Tim Jenness
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.