Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12910

Summary of WISE catalogs loaded into PDAC in 2017

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Scope

      This tickets represents a summary of WISE catalogs loaded into PDAC Qserv in the year of 2017.

      Properties of the dataset are documented at:

      Catalog descriptions

      AllWISE

      WISE All-Sky, 3-band Cryo, and Post-Cryo databases, the single exposure image meta data

      catalog (link to IPAC) PDAC database table (link to PDAC schema)
      WISE All-Sky Single Exposure (L1b) Image Inventory Table wise_00 allsky_4band_p1bm_frm
      WISE 3-Band Cryo Single Exposure (L1b) Image Inventory Table wise_00 allsky_3band_p1bm_frm
      WISE Post-Cryo Single Exposure (L1b) Image Inventory Table wise_00 allsky_2band_p1bm_frm

      WISE All-Sky, 3-band Cryo, and Post-Cryo databases

      NeoWISE

      catalog (link to IPAC) PDAC database table (link to PDAC schema)
      NEOWISE-R Year 1 Single Exposure (L1b) Source Table neowiser_yr1_00 neowiser_yr1_p1bs_psd

      Changes made to the original schema

      Data types

      The original schema files of the input datasets had to be adjusted to map Oracle-specific types into standard SQL or MySQL types. The table below show the mapping:

      Oracle MySQL
      SERIAL8 BIGINT
      SMALLFLOAT FLOAT
      INT8 BIGINT
      VARCHAR2(N BYTE) VARCHAR(N)
      NUMBER(M,D) DECIMAL(M,D)
      NUMBER(10,0) INT
      NUMBER(10) INT
      NUMBER(19) BIGINT

      Indexes

      The original schema files had no indexes. A minimal set (depending on a table) of indexes were added into the output schema.

      Replaced dec with decl as column names

      Due to a bug in Qserv implementation (see DM-9736) reserved words of the SQL-92 (or MySQL) can't be used to identifiers in tables schema. One particular problem of the original WISE dataset is caused by using dec (for declination) as the column names of both tables. The problem was discovered later on when testing the loaded dataset (see the log of downstream comments for details). A workaround deployed then was to rename the columns of both tables as decl.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gapon Igor Gaponenko
                Reporter:
                gapon Igor Gaponenko
                Watchers:
                Gregory Dubois-Felsmann, Igor Gaponenko
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel