Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4733

lsst-build should support enabling Git LFS in an already-cloned repository

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: lsst_build
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      Science Pipelines DM-W16-6
    • Team:
      Data Release Production

      Description

      A repository which does not use Git LFS is created and described in repos.yaml. It runs through CI, and is cloned onto a Jenkins build slave. Subsequently, the repository configuration in repos.yaml is updated to enable LFS. The build system should notice this change and update the cloned repository on disk appropriately. Currently, it doesn't.

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            Tim Jenness – since Josh is away, would you have time to look at this please? It's just a single commit on tickets/DM-4733 in lsst_build.

            This works for me, but there are a few things to note:

            • It's not clear why this is necessary – is there a downside to simply globally enabling LFS for all of the repositories?
            • I don't understand the gymnastics with GIT_ and SSH_ASKPASS. These were previously being removed from the environment at clone time but not on subsequent fetches, which seems odd. I've attempted to preserve the logic, but since I didn't follow the intent I'm not confident I've properly captured it.
            • The code in this package doesn't seem to follow the coding standards. I've attempted to follow the local conventions rather than the LSST rules.
            Show
            swinbank John Swinbank added a comment - Tim Jenness – since Josh is away, would you have time to look at this please? It's just a single commit on tickets/DM-4733 in lsst_build . This works for me, but there are a few things to note: It's not clear why this is necessary – is there a downside to simply globally enabling LFS for all of the repositories? I don't understand the gymnastics with GIT_ and SSH_ASKPASS . These were previously being removed from the environment at clone time but not on subsequent fetches, which seems odd. I've attempted to preserve the logic, but since I didn't follow the intent I'm not confident I've properly captured it. The code in this package doesn't seem to follow the coding standards. I've attempted to follow the local conventions rather than the LSST rules.
            Hide
            tjenness Tim Jenness added a comment -

            Sorry but I don't know enough about that bit of code to review it. I think we will have to wait for Joshua Hoblitt.

            Show
            tjenness Tim Jenness added a comment - Sorry but I don't know enough about that bit of code to review it. I think we will have to wait for Joshua Hoblitt .
            Hide
            swinbank John Swinbank added a comment -

            Ok, thanks. We should try to avoid single points of failure, though – is there anybody else who could tackle this (maybe Frossie Economou has an idea)?

            Show
            swinbank John Swinbank added a comment - Ok, thanks. We should try to avoid single points of failure, though – is there anybody else who could tackle this (maybe Frossie Economou has an idea)?
            Hide
            jhoblitt Joshua Hoblitt added a comment - - edited

            John Swinbank I believe your branch should work but I want to do some manual testing before merging it. I vaguely recall having seen issues with lfs pull on existing clones not always replacing checked out files from non-lfs checkouts but I suspect it's long since been fixed. An alternative is to delete the repo and re-reclone it.

            Show
            jhoblitt Joshua Hoblitt added a comment - - edited John Swinbank I believe your branch should work but I want to do some manual testing before merging it. I vaguely recall having seen issues with lfs pull on existing clones not always replacing checked out files from non-lfs checkouts but I suspect it's long since been fixed. An alternative is to delete the repo and re-reclone it.
            Hide
            swinbank John Swinbank added a comment -

            In face-to-face discussion, Josh suggests reworking this so that it simply deletes and re-clones rather than attempting to enable LFS in an already checked-out repository. Should be straightforward to do that.

            Show
            swinbank John Swinbank added a comment - In face-to-face discussion, Josh suggests reworking this so that it simply deletes and re-clones rather than attempting to enable LFS in an already checked-out repository. Should be straightforward to do that.
            Hide
            ljones Lynne Jones added a comment -

            This is a blocker for transitioning at least two sims packages (sims_data and sims_sed_library) that are currently starting to cause timeouts in conda installations.
            Is there a resolution coming for this issue?

            Show
            ljones Lynne Jones added a comment - This is a blocker for transitioning at least two sims packages (sims_data and sims_sed_library) that are currently starting to cause timeouts in conda installations. Is there a resolution coming for this issue?
            Hide
            swinbank John Swinbank added a comment -

            I think this work is now obsolete, given the lack of activity over several years. Lynne Jones, I'm sorry for the lack of an earlier response — is it still causing problems for you? If so, give me a shout and we'll reopen this (or file another ticket).

            Show
            swinbank John Swinbank added a comment - I think this work is now obsolete, given the lack of activity over several years. Lynne Jones , I'm sorry for the lack of an earlier response — is it still causing problems for you? If so, give me a shout and we'll reopen this (or file another ticket).

              People

              Assignee:
              swinbank John Swinbank
              Reporter:
              swinbank John Swinbank
              Reviewers:
              Joshua Hoblitt
              Watchers:
              J Matt Peterson [X] (Inactive), John Swinbank, Joshua Hoblitt, Lynne Jones, Scott Daniel, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.