Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17872

TAP returns incorrect column type when SELECT contains math

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: dax
    • Labels:
      None

      Description

      When I submit the query 

      SELECT ra, decl, w1mag, w1mag - w2mag FROM wise_00.allwise_p3as_psd
      WHERE qserv_areaspec_circle(283.831250,-30.545278,0.20)=1

      through pyvo, the values in the result column "w1mag - w2mag" are all strings instead of floats. Manipulating the results with pandas seems to kind of work anyways, but it becomes excruciatingly slow.

      I can check what the XML looks like with:

      query_string = ("SELECT ra, decl, w1mag, w1mag - w2mag FROM wise_00.allwise_p3as_psd "
      {{ "WHERE qserv_areaspec_circle(283.831250,-30.545278,0.20)=1 ")}}
      query = pyvo.dal.TAPQuery(query=query_string, baseurl='http://lsst-lsp-stable.ncsa.illinois.edu/api/tap')
      res = query.execute_stream()
      res_str = res.read()
      res_str[:1500]

      and I see: 

      <FIELD name="w1mag - w2mag" datatype="char" arraysize="*" />

       

       

        Attachments

          Issue Links

            Activity

            Hide
            cbanek Christine Banek added a comment -

            Made a PR to CADC upstream, and it has the full type detection of the system.  This is deployed in a few places on a private build, but we're waiting for the upstream PR to get fixed.

            Show
            cbanek Christine Banek added a comment - Made a PR to CADC upstream, and it has the full type detection of the system.  This is deployed in a few places on a private build, but we're waiting for the upstream PR to get fixed.
            Hide
            cbanek Christine Banek added a comment -
            Show
            cbanek Christine Banek added a comment - Upstream PR:  https://github.com/opencadc/tap/pull/116
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Repeating, today, the user's query from https://github.com/rubin-dp0/Support/issues/12:

            SELECT mag_g,mag_r, mag_g-mag_r as color
            FROM dp01_dc2_catalogs.object
            WHERE CONTAINS(POINT('ICRS', ra, dec),CIRCLE('ICRS', 55, -35, 1))=1
            AND (mag_g <21 AND mag_r <21)
            

            which was reported as solved, and confirmed by the user, on 2021-07-31, now fails again:

            ("color" is reported as "char".)

            Also of interest: the PR to CADC was never acted on.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Repeating, today, the user's query from https://github.com/rubin-dp0/Support/issues/12: SELECT mag_g,mag_r, mag_g-mag_r as color FROM dp01_dc2_catalogs.object WHERE CONTAINS (POINT( 'ICRS' , ra, dec ),CIRCLE( 'ICRS' , 55, -35, 1))=1 AND (mag_g <21 AND mag_r <21) which was reported as solved, and confirmed by the user, on 2021-07-31, now fails again: ("color" is reported as "char".) Also of interest: the PR to CADC was never acted on.
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Just tested again. color is still being reported as "char".

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Just tested again. color is still being reported as "char".
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Reopened ticket

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Reopened ticket

              People

              Assignee:
              cbanek Christine Banek
              Reporter:
              ctslater Colin Slater
              Reviewers:
              Frossie Economou
              Watchers:
              Christine Banek, Colin Slater, Fritz Mueller, Frossie Economou, Gregory Dubois-Felsmann
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.