Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33769

Improve timeout, retry and connexion persistence of webDAV client

    XMLWordPrintable

Details

    • External
    • No

    Description

      Some scalability tests against dCache (webDAV) showed that better handling of timeouts and retries were necessary as well as improvements in the management of persistent HTTP connexions to avoid overwhelming the servers when thousands of clients execute simultaneously.

      In this issue I propose modifications to improve the situation and some refactoring for making the code simpler when possible including renaming of environment variables previously named LSST_BUTLER_*.

      Attachments

        Issue Links

          Activity

            tjenness Tim Jenness added a comment -

            yusra is planning to do a v23.0.2 for step 4 so backporting is going to be needed if you want to avoid local patching in the future. Add the backport requested tag if you decide that backporting is something you want to do.

            tjenness Tim Jenness added a comment - yusra is planning to do a v23.0.2 for step 4 so backporting is going to be needed if you want to avoid local patching in the future. Add the backport requested tag if you decide that backporting is something you want to do.
            tjenness Tim Jenness added a comment -

            We have approved this for backporting if you want to do it. We think that v23.0.2rc1 will be made around the end of March.

            tjenness Tim Jenness added a comment - We have approved this for backporting if you want to do it. We think that v23.0.2rc1 will be made around the end of March.

            Thank you. I think I am done with the bulk of the back port work. I will push tomorrow.

            FabioHernandez Fabio Hernandez added a comment - Thank you. I think I am done with the bulk of the back port work. I will push tomorrow.

            This back port is ready for review.
            Pull request submitted: https://github.com/lsst/daf_butler/pull/666

            FabioHernandez Fabio Hernandez added a comment - This back port is ready for review. Pull request submitted: https://github.com/lsst/daf_butler/pull/666

            Pull request merged, thanks a lot for your help.

            I am closing this issue.

            FabioHernandez Fabio Hernandez added a comment - Pull request merged, thanks a lot for your help. I am closing this issue.

            People

              FabioHernandez Fabio Hernandez
              FabioHernandez Fabio Hernandez
              Tim Jenness
              Fabio Hernandez, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.