Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29945

A lack of graceful degradation in Qserv behavior for the empty or incomplete director index



    • Type: Bug
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:


      The problem

      The current implementation of Qserv would always return the empty result set for queries restricted by object identifiers not found in the director index of a catalog. For example, if Object is the director table of the catalog LSST, and if the object identifier column's name is objectId then the query will return the empty result set for the above stated condition:

      SELECT * FROM LSST.Object WHERE objectId=123456;

      Qserv doesn't differentiate between the following scenarios:

      1. The object does exist in the table, while the director index is empty or incomplete (doesn't have the corresponding entry for the object identifier).
      2. The object doesn't exist in the table, while there is an entry for the non-existing identifier in the director index.
      3. Or, the object doesn't exist in both the table and in the index.

      Scenarios 1 and 2 would be typical for incorrectly ingested catalogs. Scenario 1 is possible for catalogs in which the director index is yet to be constructed by the Qserv Ingest system (index construction may get delay for various operational reasons).

      The proposed solution

      • Reinforce the implementation of the Qserv query Analyzer/Optimizer to turn queries restricted by the object identifiers into the shared scan queries.
      • Consider a possibility of reporting this condition to a user as a warning via a mechanism similar to MySQL's SHOW WARNINGS (see https://dev.mysql.com/doc/refman/8.0/en/show-warnings.html).



          There are no comments yet on this issue.


            Unassigned Unassigned
            gapon Igor Gaponenko
            Fritz Mueller, Igor Gaponenko, John Gates, Nate Pease
            0 Vote for this issue
            4 Start watching this issue