Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14330

Review ObsCore and SIA V2 and implement basic metadata retrieval

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ImgServ, metaserv
    • Labels:

      Description

      The goal should be a proof of concept implementation of an SIA V2 metadata interface.

      Some goals of this ticket:

      • Develop understanding of ObsCore metadata and how it relates to current metadata

      • Develop ObsCore tables of W13 SDSS Image Metadata

      • Be able to return a URL to imgserv which can return an image for SDSS data

      • Develop an ObsCore metadata loading mechanism, ideally based on SQLAlchemy so we aren't tied to a specific database

      • May reuse data from WISE SIA V2 interface/ObsCore tables at IRSA

       - Out of scope is copying WISE images to PDAC, we can rely on the accessUrls at IRSA

       

      VOTable XML output may be generated from the VOTable XML template in dbserv:
      https://github.com/lsst/dax_dbserv/blob/master/python/lsst/dax/dbserv/templates/votable.xml.j2

       

      For examples of SIA queries which work at IPAC:

      (from https://irsa.ipac.caltech.edu/ibe/sia.html#examples )

      1. Search for ALLWISE images within 10 arcseconds of ra,dec = (19.005875, -5.084472). Return the result as in comma-separated value format.
        wget -O example1.csv "https://irsa.ipac.caltech.edu/SIA?COLLECTION=wise_allwise&POS=circle+-164.7+-5.8+0.002777&RESPONSEFORMAT=CSV"
      2. Return only the WISE channel 1 data products.
        wget -O example2.csv "https://irsa.ipac.caltech.edu/SIA?COLLECTION=wise_allwise&POS=circle+-164.7+-5.8+0.002777&RESPONSEFORMAT=CSV&BAND=3.4e-6"
      3. Return only data products that are FITS images (as opposed to gzipped files or text files).
        wget -O example3.csv "https://irsa.ipac.caltech.edu/SIA?COLLECTION=wise_allwise&POS=circle+-164.7+-5.8+0.002777&RESPONSEFORMAT=CSV&BAND=3.4e-6&FORMAT=image/fits"

        Attachments

          Issue Links

            Activity

            Hide
            bvan Brian Van Klaveren added a comment -

            Note on Implementation for positional searches:

            POS is the keyword that will be most interesting for spatial searches:

            http://www.ivoa.net/documents/SIA/20151223/REC-SIA-2.0-20151223.html#toc12

            In ObsCore, you store the s_region column as a string, which isn't quite indexable as-is, and there's no such things as corners like we have in ScienceCcdExposure. However, we are allowed to have extra columns, so we can have the corner columns and do the spatial searches that way using the scisql UDFs.

            When importing data that's not ours, especially that which derives also from ObsTAP/SIA, that may mean we need to extract the corners of an image from the s_region column just to make sure things work smoothly with scisql.

            Show
            bvan Brian Van Klaveren added a comment - Note on Implementation for positional searches: POS is the keyword that will be most interesting for spatial searches: http://www.ivoa.net/documents/SIA/20151223/REC-SIA-2.0-20151223.html#toc12 In ObsCore, you store the s_region column as a string, which isn't quite indexable as-is, and there's no such things as corners like we have in ScienceCcdExposure. However, we are allowed to have extra columns, so we can have the corner columns and do the spatial searches that way using the scisql UDFs. When importing data that's not ours, especially that which derives also from ObsTAP/SIA, that may mean we need to extract the corners of an image from the s_region column just to make sure things work smoothly with scisql.
            Hide
            fritzm Fritz Mueller added a comment -

            Kenny has transitioned off the DAX team, and further work on ObsCore is separately ticketed and resourced on the SQuaRE team.

            Show
            fritzm Fritz Mueller added a comment - Kenny has transitioned off the DAX team, and further work on ObsCore is separately ticketed and resourced on the SQuaRE team.

              People

              Assignee:
              kennylo Kenny Lo
              Reporter:
              bvan Brian Van Klaveren
              Watchers:
              Brian Van Klaveren, Fritz Mueller, Gregory Dubois-Felsmann, Tatiana Goldina, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.