Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26486

Qserv Replication worker service crashes after receiving spurious packets

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      The server crashes if a clients sends an incorrect or a random) request. Here is what's reported in the Replication/Ingest/Export worker's log file:

      ...
      2020-08-22T06:14:29.927Z  LWP 528   DEBUG  PROCESSOR  _fetchNextForProcessing  thread: 6  timeout: 1000
      2020-08-22T06:14:29.961Z  LWP 512   DEBUG  PROCESSOR  _fetchNextForProcessing  thread: 1  timeout: 1000
      2020-08-22T06:14:30.084Z  LWP 542   DEBUG  PROCESSOR  _fetchNextForProcessing  thread: 10  timeout: 1000
      2020-08-22T06:14:30.094Z  LWP 508   DEBUG  CONNECTION  _received
      terminate called after throwing an instance of 'std::overflow_error'
        what():  ProtocolBuffer::_extend  requested capacity 1735489335 exceeds the hard limit of Google Protobuf 64000000
      /home/qserv/lsst: line 7:   438 Aborted                 (core dumped) $@
      

      The problem could be triggered by any random request like:

      curl http://qserv-db34:25001/meta/version -X GET
      

      Note that the service has a proprietary (non-HTTP) binary protocol based on Protobuf.

      A proposed solution is to reinforce the code to catch exception std::overflow_error and close a connection with the offending client.

        Attachments

          Issue Links

            Activity

            Show
            gapon Igor Gaponenko added a comment - PR: https://github.com/lsst/qserv/pull/566
            Hide
            npease Nate Pease added a comment -

            looks good.

            Show
            npease Nate Pease added a comment - looks good.

              People

              Assignee:
              gapon Igor Gaponenko
              Reporter:
              gapon Igor Gaponenko
              Reviewers:
              Nate Pease
              Watchers:
              Fritz Mueller, Igor Gaponenko, Nate Pease
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: