Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-1571

Setup Qserv for SUI tests

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Data Archive, Qserv
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      DB_W15_01, DB_W15_02
    • Team:
      Data Access and Database

      Description

      Setup Qserv on lsst-db2 with and load some reasonable data set (perhaps PT 1.2). One potential caveat: we need to setup access for some accounts that are ideally other than our internal qsmaster.

        Attachments

          Issue Links

            Activity

            Hide
            tatianag Tatiana Goldina added a comment -

            I can access Qserv at lsst-db2.slac.stanford.edu:4040 from both shadev1 and shadev2 in IPAC and run the queries.

            It also can be accessed from any machine with ssh access to shadev1,2 using private port forwarding with this procedure. (Thank you, Trey Roby, for figuring it out.)

            From your localhost:
            ssh -L 7001:lsst-db2.slac.stanford.edu:4040 shadev1

            then 127.0.0.1:7001 on your localhost will get you to lsst-db2.slac.stanford.edu:4040

            Which means we can access Qserv for SUI from local development environment.

            Show
            tatianag Tatiana Goldina added a comment - I can access Qserv at lsst-db2.slac.stanford.edu:4040 from both shadev1 and shadev2 in IPAC and run the queries. It also can be accessed from any machine with ssh access to shadev1,2 using private port forwarding with this procedure. (Thank you, Trey Roby, for figuring it out.) From your localhost: ssh -L 7001:lsst-db2.slac.stanford.edu:4040 shadev1 then 127.0.0.1:7001 on your localhost will get you to lsst-db2.slac.stanford.edu:4040 Which means we can access Qserv for SUI from local development environment.
            Hide
            tatianag Tatiana Goldina added a comment -

            There is something wrong with Science_Ccd_Exposure and DeepCoadd in Qserv for SUI tests. Both have only one record. Also I can not retrieve data from Science_Ccd_Exposure, see below.

            sql> select count from DeepCoadd
            [2015-02-06 13:41:26] 1 row(s) retrieved starting from 0 in 753/756 ms

            (IPAC version returns 48)

            sql> select count from Science_Ccd_Exposure
            [2015-02-06 13:34:46] 1 row(s) retrieved starting from 0 in 829/836 ms

            (IPAC version returns 26528)

            sql> select * from Science_Ccd_Exposure
            [2015-02-06 13:39:37] 1 row(s) retrieved starting from 0 in 927/970 ms
            [2015-02-06 13:39:37] [S1009] Cannot convert value '0000-00-00 00:00:00' from column 32 to TIMESTAMP.
            [2015-02-06 13:39:37] [S1009] Value '[B@548997d1' can not be represented as java.sql.Timestamp

            Show
            tatianag Tatiana Goldina added a comment - There is something wrong with Science_Ccd_Exposure and DeepCoadd in Qserv for SUI tests. Both have only one record. Also I can not retrieve data from Science_Ccd_Exposure, see below. sql> select count from DeepCoadd [2015-02-06 13:41:26] 1 row(s) retrieved starting from 0 in 753/756 ms (IPAC version returns 48) sql> select count from Science_Ccd_Exposure [2015-02-06 13:34:46] 1 row(s) retrieved starting from 0 in 829/836 ms (IPAC version returns 26528) sql> select * from Science_Ccd_Exposure [2015-02-06 13:39:37] 1 row(s) retrieved starting from 0 in 927/970 ms [2015-02-06 13:39:37] [S1009] Cannot convert value '0000-00-00 00:00:00' from column 32 to TIMESTAMP. [2015-02-06 13:39:37] [S1009] Value '[B@548997d1' can not be represented as java.sql.Timestamp
            Hide
            jammes Fabrice Jammes added a comment - - edited

            Hi Tatiana Goldina,

            The 1 row problem has been identified and will be solved in DM-1974.
            The java.sql.Timestamp error could be related to JDBC driver error, as it doesn't appears with built-in mysql client. I've opened DM-1982 for this one.

            Show
            jammes Fabrice Jammes added a comment - - edited Hi Tatiana Goldina , The 1 row problem has been identified and will be solved in DM-1974 . The java.sql.Timestamp error could be related to JDBC driver error, as it doesn't appears with built-in mysql client. I've opened DM-1982 for this one.
            Hide
            jammes Fabrice Jammes added a comment -

            Hi Tatiana Goldina,

            DeepCoadd, Science_Ccd_Exposure tables should be correctly loaded now. Could you please let me know if it works? This ticket is now done, so I propose we open new tickets or use DM-1974, DM-1982 for talking about future problems.

            Thanks,

            Fabrice

            Show
            jammes Fabrice Jammes added a comment - Hi Tatiana Goldina , DeepCoadd, Science_Ccd_Exposure tables should be correctly loaded now. Could you please let me know if it works? This ticket is now done, so I propose we open new tickets or use DM-1974 , DM-1982 for talking about future problems. Thanks, Fabrice
            Hide
            tatianag Tatiana Goldina added a comment -

            Thank you! All the counts now match those we have in IPAC. The timestamp issue is also gone, since all the timestamps are valid. But just in case, I will adopt a solution suggested by Jacek: convert all-zero timestamps to nulls by setting zeroDateTimeBehavior=convertToNull parameter in MySQL JDBC driver.

            Show
            tatianag Tatiana Goldina added a comment - Thank you! All the counts now match those we have in IPAC. The timestamp issue is also gone, since all the timestamps are valid. But just in case, I will adopt a solution suggested by Jacek: convert all-zero timestamps to nulls by setting zeroDateTimeBehavior=convertToNull parameter in MySQL JDBC driver.

              People

              • Assignee:
                jammes Fabrice Jammes
                Reporter:
                fritzm Fritz Mueller
                Reviewers:
                Tatiana Goldina
                Watchers:
                Fabrice Jammes, Jacek Becla, Tatiana Goldina, Trey Roby, Xiuqin Wu [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel