Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-1758

Spatial restrictors on child tables are not handled properly

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:

      Description

      Currently, attempting to run a query with a spatial restrictor on a partitioned child table crashes all workers. I think the czar should probably reject such queries, but if not it definitely needs to rewrite them in a different way.

      Here's a sample query on child table DeepForcedSource:

      select * from DeepForcedSource where qserv_areaspec_box(0.5, 1.1, 0.6, 1.2);

      Here's what I see in qserv-czar.log:

      0112 17:43:02.647 [0x7f6d4971d700] INFO  root (app.py:495) - constraint= <lsst.qserv.czar.czarLib.Constraint; proxy of <Swig Object of type 'lsst::qserv::query::Constraint *' at 0x7f6d37c9f0c0> >
      		 
      0112 17:43:02.654 [0x7f6d4971d700] INFO  root (spatial.py:35) - Using 85 stripes and 12 substripes.
      		 
      0112 17:43:02.657 [0x7f6d4971d700] INFO  root (app.py:600) - Looking for indexhints in  [('qserv_areaspec_box', ['0.5', '1.1', '0.6', '1.2'])]
      		 
      0112 17:43:02.659 [0x7f6d4971d700] INFO  root (app.py:610) - lookup got chunks: None
      		 
      0112 17:43:02.661 [0x7f6d4971d700] INFO  root (metadata.py:43) - Using /db1/run/qserv-master/etc/emptyChunks.txt as default empty chunks file.
      		 
      0112 17:43:02.676 [0x7f6d4971d700] INFO  root (metadata.py:46) - Using /db1/run/qserv-master/etc as empty chunks path
      		 
      0112 17:43:02.678 [0x7f6d4971d700] INFO  root (metadata.py:90) - /db1/run/qserv-master/etc/empty_LSST.txt not found while loading empty chunks file.
      		 
      0112 17:43:02.680 [0x7f6d4971d700] ERROR root (metadata.py:68) - Couldn't find /db1/run/qserv-master/etc/empty_LSST.txt, using /db1/run/qserv-master/etc/emptyChunks.txt.
      		 
      0112 17:43:02.685 [0x7f6d35dc4700] DEBUG root (app.py:389) - reporting -1 100 Dispatch Query.
      		 
      0112 17:43:02.685 [0x7f6d35dc4700] DEBUG root (build/qdisp/MessageStore.cc:49) - Msg: -1 100 Dispatch Query.
      		 
      0112 17:43:02.685 [0x7f6d35dc4700] DEBUG root (build/ccontrol/userQueryProxy.cc:118) - EXECUTING UserQuery_submit(2)
      		 
      0112 17:43:02.685 [0x7f6d35dc4700] INFO  root (build/qana/ScanTablePlugin.cc:128) - Squash scan tables: <2 chunks.
      		 
      0112 17:43:02.685 [0x7f6d35dc4700] INFO  root (build/ccontrol/UserQuery.cc:270) - UserQuery::_setupMerger()
      		 
      0112 17:43:02.686 [0x7f6d35dc4700] INFO  root (build/ccontrol/UserQuery.cc:176) - UserQuery beginning submission
      		 
      0112 17:43:02.686 [0x7f6d35dc4700] INFO  root (build/qproc/QuerySession.cc:361) - Non-subchunked
      		 
      0112 17:43:02.686 [0x7f6d35dc4700] DEBUG root (build/qproc/QuerySession.cc:381) - returning  queries: 
      0112 17:43:02.686 [0x7f6d35dc4700] DEBUG root (build/qproc/QuerySession.cc:383) - SELECT * FROM LSST.DeepForcedSource_7310 AS QST_1_ WHERE scisql_s2PtInBox(QST_1_.,QST_1_.,0.5,1.1,0.6,1.2)=1
      		 
      0112 17:43:02.686 [0x7f6d35dc4700] DEBUG root (build/qproc/TaskMsgFactory2.cc:151) - no nextFragment
      		 
      0112 17:43:02.687 [0x7f6d35dc4700] DEBUG root (build/qproc/TaskMsgFactory2.cc:153) - SELECT * FROM LSST.DeepForcedSource_7310 AS QST_1_ WHERE scisql_s2PtInBox(QST_1_.,QST_1_.,0.5,1.1,0.6,1.2)=1
      		 
      0112 17:43:02.687 [0x7f6d35dc4700] DEBUG root (build/qdisp/Executive.cc:396) - Executive (0x1af6750) tracking id=1
      		 
      0112 17:43:02.687 [0x7f6d35dc4700] INFO  root (build/qdisp/Executive.cc:172) - Exec add pth=/chk/LSST/7310
      		 
      0112 17:43:02.687 [0x7f6d35dc4700] DEBUG root (build/qdisp/MessageStore.cc:49) - Msg: 7310 1200 Exec add pth=/chk/LSST/7310
      		 
      Opening xroot://127.0.0.1:1094//chk/LSST/7310
      		 
      ...

      The queries being dispatched are totally bogus. Unsurprisingly, xrootd.log contains:

      terminate called after throwing an instance of 'std::runtime_error'
      		 
        what():  Couldn't get result

      Finally here's the metadata for the table:

      /DBS/LSST/TABLES/DeepForcedSource   READY
      		 
      /DBS/LSST/TABLES/DeepForcedSource/partitioning.json {"dirColName": "deepSourceId", "dirTable": "DeepSource", "subChunks": "0"}
      		 
      /DBS/LSST/TABLES/DeepForcedSource.json  {"compression": "0", "match": "0", "uuid": "7adf9d49-14d4-4d0d-a440-43437fcf3565"}

        Attachments

          Issue Links

          mentioned in

          Page Loading...

            Activity

            Hide
            Permalink
            fritzm Fritz Mueller added a comment -

            This appears to have been addressed in the intervening years, and now works as expected.

            Show
            fritzm Fritz Mueller added a comment - This appears to have been addressed in the intervening years, and now works as expected.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              smonkewitz Serge Monkewitz
              Watchers:
              Fritz Mueller, Serge Monkewitz
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.