Details
Description
Master ticket for work/discussion on the ADQL dialect(s) and extensions supported by Rubin services.
Issues to consider:
- Dialect of ADQL supported by Qserv
- How to document it in a user-facing way? What can be captured just in /capabilities and what is deeper (e.g., Qserv limitations on the allowed arguments of CONTAINS() )?
- Are there currently-missing things we could support with limited effort? (AREA? CENTROID? COORD1/2?)
- What's too hard to contemplate? INTERSECTS?
- Generalization of documentation and error messages for ADQL limitations
- Much like Qserv, IRSA's TAP service has specific limitations on the use of the ADQL geometry primitives that cannot be captured by the /capabilities "<feature>" elements.
- Is there something we could do at IVOA level to standardize self-documentation?
- How errors related to the dialect are returned?
- How to return a pointer to user-level documentation on a service's limitations?
- Is TAP's existing /examples notion adequate if used in a consistent way?
- Additional functions provided by Rubin
- E.g., the non-spatial functions in scisql, like flux-to-mags conversions
- Do we intend to support these indefinitely?
- Can we get these advertised to the community? Can we use the existing IVOA registered-ADQL-extension mechanism?
- Are there spatial functions in scisql that don't have a reasonable ADQL equivalent? Should we try to get them into ADQL? Is this a prerequisite to blocking all the scisql spatial functions at the TAP layer?
- Is "scisql" an appropriate prefix or should we change it?
- Compatibility of query dialects between Qserv (Data Release tables and "published" user tables) and Postgres (PPDB tables, Consolidated DB, general user tables, "live ObsTAP")
- Can we make the non-spatial scisql functions available in Postgres?
Original description:
From an Architecture team discussion:
We need to figure out what to do about the legacy of the sciSQL function set in Qserv, and how these will be represented as part of the public interface of the RSP TAP service.
Attachments
Issue Links
- relates to
-
DM-22724 Evaluate the completeness of the ADQL dialect currently supported
- To Do
-
- In Progress
-
- In Progress
-
- In Review
-
- Done
-
- To Do
Activity
Also, just for completeness:
1) Yes, this is not about "Qserv" per se - it's about the dialect of ADQL we support. Issue title changed accordingly.
It becomes a bit about Qserv - and Postgres - if we are trying to keep the dialect as similar as possible for our two back ends.
2) I agree with Fritz Mueller's suggestion that at some point we just have to run down the whole sciSQL list and mark each one as "covered by ADQL construct XYZ", "proposed as a registered RSP extension to ADQL", or "drop from RSP support".
Thanks for the title edit, Gregory Dubois-Felsmann. I like the idea of having the maintenance-passthrough switch available for debug/dev, if it is not too much effort to implement.
This came up again today in terms of the scisql magnitude conversion functions.
This is a perennial issue. We've seen it recur in various guises, and it's become more important recently because we're trying to get our TAP /capabilities responses to conform better to our true supported dialects, and because Firefly is beginning to condition its TAP UI on what is advertised, in order to deflect users from submitting queries that are just going to fail right away. (The most notable recent example is the support of UPLOAD, relevant to multi-object searches.)
There are many related tickets, including DM-8237, DM-14535, DM-22724, DM-35110, and the recent DM-37934 and DM-39074.
I'm going to take over this ticket as the master ticket for this subject, for now. (I'd use an epic, but I'm afraid of awakening some automated EVM beast if I do the wrong thing.)
For the production TAP service in the 2022+ (DP0.2+) era, I think that's correct:
For maintenance/debugging/development purposes, it seems it would be useful to have an option to the TAP service that would disable this error-checking - for instance, for when we are debugging a new UDF. This could be a deployment-level option (i.e., "TAP on the development RSP instances allows pass-throughs").