Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8239

PDAC: objectId 3219370448785419 present in test database, but missing in PDAC Qserv

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:

      Description

      Yi Mei says:

      I found another problem: the database query

      select * from RunDeepForcedSource where objectId=3219370448785419;

      gave me 808 records but

      curl -o outForcedSource -d 'query=select+*+from+RunDeepForcedSource+where+objectId=3219370448785419;' http://lsst-qserv-dax01.ncsa.illinois.edu:5000/db/v0/tap/sync

      gave me nothing.

      (Note that the direct mysql query above was run against the test database gapon_sdss_stripe92_patch366_0)

        Attachments

          Issue Links

            Activity

            Hide
            fritzm Fritz Mueller added a comment -

            A similar query for a known-loaded objectId in the qserv cluster executes correctly:

            select * from RunDeepForcedSource where objectId=3448068867358968;

            Show
            fritzm Fritz Mueller added a comment - A similar query for a known-loaded objectId in the qserv cluster executes correctly: select * from RunDeepForcedSource where objectId=3448068867358968;
            Hide
            gapon Igor Gaponenko added a comment -

            The problem has been investigated. And here is the conclusion:

            The object in question belongs to the overlap area processed during S13 DC by IN2P3 in patch=366,0. The IN2P3's version of the patch was not included into the merged Stripe82 catalog. The NCSA's version was used instead. User hit a very narrow hole in the merged dataset right on the edge.

            Note that this is a well known problem with the current version of the Stripe82 catalog loaded into PDAC/Qserv database sdss_stripe82_00. The problem is well understood, and it will be solved in a refined version of the catalog.

            I picked that patch by a pure accident when preparing that tiny monolithic catalog for SUIT in an anticipation of the PDAC deployment. I wouldn't draw any conclusions from this as this monolithic catalog is not meant to be used to Q&A PDAC! For that (later) purpose we're planning to have the full merged catalog on some dedicated hardware....when/if it will be available.

            Show
            gapon Igor Gaponenko added a comment - The problem has been investigated. And here is the conclusion: The object in question belongs to the overlap area processed during S13 DC by IN2P3 in patch=366,0 . The IN2P3's version of the patch was not included into the merged Stripe82 catalog. The NCSA's version was used instead. User hit a very narrow hole in the merged dataset right on the edge. Note that this is a well known problem with the current version of the Stripe82 catalog loaded into PDAC/Qserv database sdss_stripe82_00 . The problem is well understood, and it will be solved in a refined version of the catalog. I picked that patch by a pure accident when preparing that tiny monolithic catalog for SUIT in an anticipation of the PDAC deployment. I wouldn't draw any conclusions from this as this monolithic catalog is not meant to be used to Q&A PDAC! For that (later) purpose we're planning to have the full merged catalog on some dedicated hardware....when/if it will be available.
            Hide
            gapon Igor Gaponenko added a comment - - edited

            This is how the patch info could be found from the original (before the merge) version of the catalog brought from IN2P3:

            USE lsst_prod_DC_2013_2;
            SELECT id,coadd_id FROM RunDeepSource WHERE id=3219370448785419;
            

            +------------------+----------+
            | id               | coadd_id |
            +------------------+----------+
            | 3219370448785419 | 23986179 |
            +------------------+----------+
            

            SELECT deepCoaddId,tract,patch FROM DeepCoadd WHERE deepCoaddId=23986179;
            

            +-------------+-------+-------+
            | deepCoaddId | tract | patch |
            +-------------+-------+-------+
            |    23986179 |     0 | 366,0 |
            +-------------+-------+-------+
            

            Show
            gapon Igor Gaponenko added a comment - - edited This is how the patch info could be found from the original (before the merge) version of the catalog brought from IN2P3: USE lsst_prod_DC_2013_2; SELECT id,coadd_id FROM RunDeepSource WHERE id=3219370448785419; +------------------+----------+ | id | coadd_id | +------------------+----------+ | 3219370448785419 | 23986179 | +------------------+----------+ SELECT deepCoaddId,tract,patch FROM DeepCoadd WHERE deepCoaddId=23986179; +-------------+-------+-------+ | deepCoaddId | tract | patch | +-------------+-------+-------+ | 23986179 | 0 | 366,0 | +-------------+-------+-------+

              People

              • Assignee:
                gapon Igor Gaponenko
                Reporter:
                ymei Yi Mei [X] (Inactive)
                Watchers:
                Fritz Mueller, Igor Gaponenko, Yi Mei [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel