Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-652

Explicitly add ObsTAP support to the LSP requirements, LDM-554

    XMLWordPrintable

Details

    Description

      In the "VO first" redesign of the LSST data service APIs, we adopted a plan to prioritize the use of ObsTAP over SIAv2 for image metadata service, although with the intent to provide both in the end.

      An "ObsTAP service", briefly, is simply the provision of a table (or view) in a TAP service, containing image metadata in ObsCore format (http://www.ivoa.net/documents/ObsCore/20170509/index.html), and with the explicitly prescribed table name ivoa.ObsCore.

      That is, once you have a database, and a TAP service, you can have an ObsTAP service simply by serving the right data from that database.

      It was, in particular, intended that we use ObsTAP internally, for several reasons, including:

      • Once you have an ObsTAP service, there is a conceptually trivial way to implement SIAv2 as a layer on top of it, and there is existing code to do so (e.g., https://github.com/opencadc/dal/tree/master/cadc-sia).
      • ObsTAP is more flexible and is permitted to return data about a wider range of data products than SIAv2.
      • ObsTAP permits the use of ADQL queries to efficiently gather statistics about the available data (e.g., via SELECT DISTINCT queries) to allow an introspective generation of information to guide a user in exploring the space of data products.

      Because of an oversight, this design choice was not reflected in the LSP Requirements, LDM-554. I would like to remedy that.

      I would like to modify requirement DMS-API-REQ-0022 from

      Specification: The API Aspect shall provide for retrieval of image and visit metadata via TAP ADQL queries.

      to

      Specification: The API Aspect shall provide for retrieval of image and visit metadata via TAP ADQL queries. In addition to the provision of tables exposing the CAOM2 data model (see DMS-API-REQ-0021), the API Aspect shall provide an ObsTAP service exposing an IVOA ObsCore-compliant, simplified data model for the image and visit metadata.

      and add to DMS-API-REQ-0016 (SIA Service for Image Availability) the following text:

      Discussion: The SIA service should present the same ObsCore view of the image and visit metadata that is provided by the ObsTAP service (see DMS-API-REQ-0022), subject to the restriction on dataproduct_type from Section 2.1.14 of the SIAv2 standard.

      Attachments

        Issue Links

          Activity

            This would codify what we were already planning to do, so I support this.

            ctslater Colin Slater added a comment - This would codify what we were already planning to do, so I support this.

            Planned end was yesterday. Given the lack of disagreement above, I will proceed to implement the above-proposed requirement text in MagicDraw and produce a draft of LDM-554 for the CCB to consider.

            gpdf Gregory Dubois-Felsmann added a comment - Planned end was yesterday. Given the lack of disagreement above, I will proceed to implement the above-proposed requirement text in MagicDraw and produce a draft of LDM-554 for the CCB to consider.

            ok

            womullan Wil O'Mullane added a comment - ok
            lguy Leanne Guy added a comment -

            I support this RFC

            lguy Leanne Guy added a comment - I support this RFC
            tjenness Tim Jenness added a comment -

            gpdf do you have a timeline for a draft document? Can we change the planned end?

            tjenness Tim Jenness added a comment - gpdf do you have a timeline for a draft document? Can we change the planned end?
            tjenness Tim Jenness added a comment -

            The CCB discussed this today and given that we have three LDM-554 changes being considered we are happy to process these tickets in parallel without requiring the MagicDraw updates which seem to be blocking progress.

            tjenness Tim Jenness added a comment - The CCB discussed this today and given that we have three LDM-554 changes being considered we are happy to process these tickets in parallel without requiring the MagicDraw updates which seem to be blocking progress.

            Added in implementation: Discussion text for DMS-API-REQ-0022:

            Existing community experience, such as at CADC, is that it is possible to provide an ObsTAP service as a view on a join of CAOM2 tables.

             

            gpdf Gregory Dubois-Felsmann added a comment - Added in implementation: Discussion text for DMS-API-REQ-0022: Existing community experience, such as at CADC, is that it is possible to provide an ObsTAP service as a view on a join of CAOM2 tables.  
            gpdf Gregory Dubois-Felsmann added a comment - - edited

            Implementation got hung up on a MagicDraw problem back in 2020, and then overlooked.

            Picking this up again.

            After experience with verification activities over the past couple of years, I am proposing for the final implementation that we split DMS-API-REQ-0022 into two parts: ObsTAP (keeping the requirement ID) and a new requirement for TAP/ADQL service of the future CAOM2 tables.

            I.e., now:

            Access to Image and Visit Metadata: ObsTAP
            DMS-API-REQ-0022

            Specification:
            The API Aspect shall provide for retrieval of image and visit metadata via TAP ADQL queries,
            including an ObsTAP service exposing an IVOA ObsCore-compliant, simplified data model for the image and visit metadata, meeting at least version 1.1 of the ObsCore standard.

            Discussion:
            Existing community experience, such as at CADC, is that it is possible to provide an ObsTAP service as a view on a join of CAOM2 tables.
            It is also acceptable to provide a materialized ``ivoa.ObsCore'' table for the ObsTAP service.

            Access to Image and Visit Metadata: CAOM2
            DMS-API-REQ-xx06-RFC-652

            Specification:
            The API Aspect shall provide for retrieval of CAOM2-based image and visit metadata (see DMS-API-REQ-0021) via TAP ADQL queries.

            gpdf Gregory Dubois-Felsmann added a comment - - edited Implementation got hung up on a MagicDraw problem back in 2020, and then overlooked. Picking this up again. After experience with verification activities over the past couple of years, I am proposing for the final implementation that we split DMS-API-REQ-0022 into two parts: ObsTAP (keeping the requirement ID) and a new requirement for TAP/ADQL service of the future CAOM2 tables. I.e., now: Access to Image and Visit Metadata: ObsTAP DMS-API-REQ-0022 Specification: The API Aspect shall provide for retrieval of image and visit metadata via TAP ADQL queries, including an ObsTAP service exposing an IVOA ObsCore-compliant, simplified data model for the image and visit metadata, meeting at least version 1.1 of the ObsCore standard. Discussion: Existing community experience, such as at CADC, is that it is possible to provide an ObsTAP service as a view on a join of CAOM2 tables. It is also acceptable to provide a materialized ``ivoa.ObsCore'' table for the ObsTAP service. Access to Image and Visit Metadata: CAOM2 DMS-API-REQ-xx06- RFC-652 Specification: The API Aspect shall provide for retrieval of CAOM2-based image and visit metadata (see DMS-API-REQ-0021) via TAP ADQL queries.

            People

              gpdf Gregory Dubois-Felsmann
              gpdf Gregory Dubois-Felsmann
              Christine Banek, Colin Slater, Fritz Mueller, Frossie Economou, Gregory Dubois-Felsmann, Kenny Lo, Leanne Guy, Tim Jenness, Wil O'Mullane
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Planned End:

                Jenkins

                  No builds found.