Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33769

Improve timeout, retry and connexion persistence of webDAV client

    XMLWordPrintable

    Details

    • Team:
      External
    • Urgent?:
      No

      Description

      Some scalability tests against dCache (webDAV) showed that better handling of timeouts and retries were necessary as well as improvements in the management of persistent HTTP connexions to avoid overwhelming the servers when thousands of clients execute simultaneously.

      In this issue I propose modifications to improve the situation and some refactoring for making the code simpler when possible including renaming of environment variables previously named LSST_BUTLER_*.

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-34169 Pin responses version in 0.8.0 env

          • Done

            Activity

            Ascending order - Click to sort in descending order
            10 older comments
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Yusra AlSayyad is planning to do a v23.0.2 for step 4 so backporting is going to be needed if you want to avoid local patching in the future. Add the backport requested tag if you decide that backporting is something you want to do.

            Show
            tjenness Tim Jenness added a comment - Yusra AlSayyad is planning to do a v23.0.2 for step 4 so backporting is going to be needed if you want to avoid local patching in the future. Add the backport requested tag if you decide that backporting is something you want to do.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            We have approved this for backporting if you want to do it. We think that v23.0.2rc1 will be made around the end of March.

            Show
            tjenness Tim Jenness added a comment - We have approved this for backporting if you want to do it. We think that v23.0.2rc1 will be made around the end of March.
            Hide
            Permalink
            FabioHernandez Fabio Hernandez added a comment -

            Thank you. I think I am done with the bulk of the back port work. I will push tomorrow.

            Show
            FabioHernandez Fabio Hernandez added a comment - Thank you. I think I am done with the bulk of the back port work. I will push tomorrow.
            Hide
            Permalink
            FabioHernandez Fabio Hernandez added a comment -

            This back port is ready for review.
            Pull request submitted: https://github.com/lsst/daf_butler/pull/666

            Show
            FabioHernandez Fabio Hernandez added a comment - This back port is ready for review. Pull request submitted: https://github.com/lsst/daf_butler/pull/666
            Hide
            Permalink
            FabioHernandez Fabio Hernandez added a comment -

            Pull request merged, thanks a lot for your help.

            I am closing this issue.

            Show
            FabioHernandez Fabio Hernandez added a comment - Pull request merged, thanks a lot for your help. I am closing this issue.

              People

              Assignee:
              FabioHernandez Fabio Hernandez
              Reporter:
              FabioHernandez Fabio Hernandez
              Reviewers:
              Tim Jenness
              Watchers:
              Fabio Hernandez, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.