Details
-
Type:
Story
-
Status: To Do
-
Resolution: Unresolved
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Team:Architecture
-
Urgent?:No
Description
The TAP interface uses IVOA ADQL as the query language. Spatial queries are converted internally into scisql spatial calls and the scisql spatial queries are not part of the public interface. There are other scisql APIs that are wanted but we have to be careful and intentional about the way we make them public.
At issue:
- Once we make a scisql function part of the public API, how long are we required to support that interface? Will people expect their queries from DR1 to work on DR2?
- If we change the Qserv backend from MariaDB to something else are we required to port the scisql interface or translate it?
- Should we have an explicit Rubin API that is like scisql but explicitly decoupled in the ADQL interface?
- Should we have a whitelist of extensions rather than making everything visible?
I feel that we should be intentional about what we do here since there can be long-term unforeseen ramifications if we let default behaviors through.
A whitelist is much better than allowing all scisql and UDFs to work – the spatial APIs should not be callable at all. An explicit rubin extension API is better than calling it scisql everywhere.
We also should work with the IVOA to extend ADQL to support the features we need. We have never tried to do that before so we can't blame IVOA for this. Does ADQL have an explicit extension mechanism?
I don't think there's any way that we would (or even could) let all valid MariaDB SQL through the TAP layer. The very fact that Fritz Mueller couldn't use a photometry scisql function is an indication that we have to explicitly allow such. ("Allow list" is becoming a preferred term, by the way.)
All of these fall under the User Defined Functions in section 2.5 of the ADQL 2.0 standard. They can be advertised in the TAP service capabilities via TAPRegExt. If IVOA wants to add them to ADQL (even as optional, but with a particular definition if available), that would be great; translating them at that point would be trivial.
scisql was an arbitrary name picked for this package. I'm not sure picking another arbitrary name to map to that one adds that much. In particular, the ability to do medians or convert from magnitudes to nanojanskys is hardly a Rubin-specific thing, so I'm not sure it would be proper to have a rubin or lsst prefix.
Please also consider the opportunity cost of not providing these functions. Users will have to copy and paste complex expressions into their SELECT and WHERE clauses. They may get them wrong, particularly complicated numerical constants.