Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5910

Add code to the czar to throttle incoming large results.

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      The czar needs code to limit the number of Tasks sending back large results at any given time.

        Attachments

          Issue Links

            Activity

            Hide
            jgates John Gates added a comment -

            The czar and workers appear to handle large results gracefully. The same cannot be said for the proxy. It becomes unresponsive and dies while trying to return results larger than 1GB. This can be seen in the log files where the czar finishes the query and hands it off to the proxy to be returned, and the proxy consumes system resources until it dies.

            Show
            jgates John Gates added a comment - The czar and workers appear to handle large results gracefully. The same cannot be said for the proxy. It becomes unresponsive and dies while trying to return results larger than 1GB. This can be seen in the log files where the czar finishes the query and hands it off to the proxy to be returned, and the proxy consumes system resources until it dies.
            Hide
            abh Andy Hanushevsky added a comment -

            OK, I looked through and found nothing shocking or even not shocking. So, it has passed my review.

            Show
            abh Andy Hanushevsky added a comment - OK, I looked through and found nothing shocking or even not shocking. So, it has passed my review.
            Hide
            salnikov Andy Salnikov added a comment -

            Looks OK, one nitpick comment in PR.

            Show
            salnikov Andy Salnikov added a comment - Looks OK, one nitpick comment in PR.
            Hide
            jgates John Gates added a comment -

            I changed the code so that it uses a single static ThreadPool for all large result handling instead of one ThreadPool per user query.

            Show
            jgates John Gates added a comment - I changed the code so that it uses a single static ThreadPool for all large result handling instead of one ThreadPool per user query.
            Hide
            salnikov Andy Salnikov added a comment -

            John, everything except std::min seems fine. Your segfault comments look disturbing, I'm not sure the reason has anything to do with CzarConfig, but I guess you'll figure it out soon in your new ticket. OK to merge.

            Show
            salnikov Andy Salnikov added a comment - John, everything except std::min seems fine. Your segfault comments look disturbing, I'm not sure the reason has anything to do with CzarConfig, but I guess you'll figure it out soon in your new ticket. OK to merge.
            Hide
            jgates John Gates added a comment -

            This reduced the load on the czar but other resources are being exhausted and it looks like changes to xrootd are needed to solve the issues.

            Show
            jgates John Gates added a comment - This reduced the load on the czar but other resources are being exhausted and it looks like changes to xrootd are needed to solve the issues.

              People

              • Assignee:
                jgates John Gates
                Reporter:
                jgates John Gates
                Reviewers:
                Andy Hanushevsky, Andy Salnikov, Fritz Mueller
                Watchers:
                Andy Hanushevsky, Andy Salnikov, Fritz Mueller, John Gates
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel