I think the answer is that Qserv today will limit queries to relevant chunks if and only if qserv_ functions are used. (Those qserv_ functions get translated into scisql_ functions on the workers.) No scisql_ functions in the query appear to be recognized for chunk restriction; they are only ever executed on the workers. (There is a special case for scisql_angSep which is used to check for overlap region usage but still not for chunk restriction.)

This matches with the documentation in https://github.com/lsst/qserv/blob/master/UserManual.md#spatial-constraints-should-be-expressed-through-our-qserv_areaspec_-functions.

Fritz Mueller please outsource to someone to confirm my code reading and decide if any clarifications to the documentation are in order.

Does this mean that SUIT should limit the calls to DAX service using only the qserv_ functions? Kian-Tat Lim and Fritz Mueller

If you have an area-restricted query, and you can use qserv_ functions to express your constraint, use those for best performance.

"No scisql_ functions in the query appear to be recognized for chunk restriction; they are only ever executed on the workers"

For SUIT request through DAX, what does this mean, not executed or slow execution?

Xiuqin Wu [X]: qserv_ functions enable Qserv to execute the query on only the chunks that might produce results. Without a qserv_ function (or an objectId condition in the WHERE clause), Qserv has to execute the query on every chunk, which will usually take longer.

So, as Fritz said, if you are using a scisql_ function in the WHERE clause that has a qserv_ equivalent, you should use the qserv_ version for best performance. It will be executed only on the relevant chunks, and it will automatically be converted into the scisql_ function for execution on the workers.

If you are using a scisql_ function that does not have a qserv_ equivalent, you should consider adding a qserv_ restriction function in the WHERE clause. Whether you do so or not, the scisql_ function will still be executed on the workers. But if you don't add a qserv_ function, the query will execute on every chunk and so may be slower.

Xiuqin Wu [X]: We need to edit the [sample queries] https://confluence.lsstcorp.org/display/DM/PDAC+sample+queries+and+test+cases accordingly.

This ticket is shown as "in review". Where will the user instructions / policy be documented persistently?

Presumably (Brian Van Klaveren?) this is an interim solution and eventually this will be done via optimal back-end interpretation of equivalent ADQL constructs?

I do think this would be an interim solution. Long term we should support ADQL, though I there may be some similar restrictions there as well.

I would note that the qserv_areaspec_**(...) are semantically equivalent to scisql_s2PtIn**(psRa, psDecl, ...), where psRa and psDecl are always from the director table (Object) but with the restriction that you may only have one such qserv_areaspec_ directive per query.

And that's probably a good way of thinking about it - the qserv_areaspec_ functions are really directives telling qserv the geometries to restrict the queries to, based on the master partitioning scheme.

+1 to Brian's comment above. We still need to decide where to document before closing the ticket?

Has this ticket become irrelevant now that we are using ADQL?

I believe so (yes, now irrelevant).

TAP notwithstanding, Qserv itself also now automatically infers the most common area restrictions directly from `scisql_s2PtIn...`.

Gregory Dubois-Felsmann: please set status to "Invalid" if you feel the issue is satisfactorily resolved, or clarify required further action here if not?

Kian-Tat Lim: Was the status update to "In Review" on 09/Nov/16 intentional?  I do not see any linked ticket.  Is/was there material requiring review?

As I said at the time, the status was updated to "In Review" because I was asserting that existing documentation already clarified this policy and wanted a check on that.

Thanks, Kian-Tat Lim (sorry I missed your original comment above the fold).  I'll attach this to our current F20 doc epic, and close it when we've updated UserManual.md wrt. auto-inferred spatial constraints.

It seems like this ticket is still relevant in that we are allowing scisql calls directly into the ADQL.

We need to make a decision about this, maybe on a new ticket, since we have to be intentional about extensions to ADQL given:

• We are promising to support those extensions for a long time since people want their queries to work.
• We need to minimize these extensions (do not use scisql for sky area when ADQL works fine)
• We need to whitelist specific extensions.
• We should consider renaming them so they are not locking us into scisql if at some point we change from mariadb.