Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29318

Refined testing plan and tools for KPM50

    XMLWordPrintable

    Details

    • Type: Story
    • Status: In Progress
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      Goals

      The default testing plan for KPM50 as it's outlined in https://github.com/lsst/qserv/blob/master/admin/python/lsst/qserv/testing/config/common.yaml needs to be refined to be compatible with actual table schemas of the catalog.

      Other actions

      Migrate tools and configurations into the dedicated Git package https://github.com/lsst-dm/qserv-perf-tests

        Attachments

          Issue Links

            Activity

            Hide
            gapon Igor Gaponenko added a comment - - edited

            Objects identifiers for the objectId-based tests

            One of the tests to be run requires having a collection of the object identifiers (the primary keys of the "director" table Object) to be available. The test would randomly select identifiers from the collection. It's required to ensure that objects referred to by the identifiers were uniformly distributed across the worker nodes. In this case, the load will be equally distributed within the cluster. The identifiers were extracted from the Object table using the following query:

            SELECT chunkId, MIN(objectId) FROM kpm50.Object GROUP BY chunkId;
            

            The query reported 146332 unique identifiers each mapping to a unique chunk (there are 146332 such chunks in the catalog).
            The collection was put into the following file of the Git package https://github.com/lsst-dm/qserv-perf-tests:

            datasets/kpm50/one_object_per_chunk.txt
            

            Show
            gapon Igor Gaponenko added a comment - - edited Objects identifiers for the objectId-based tests One of the tests to be run requires having a collection of the object identifiers (the primary keys of the "director" table Object ) to be available. The test would randomly select identifiers from the collection. It's required to ensure that objects referred to by the identifiers were uniformly distributed across the worker nodes. In this case, the load will be equally distributed within the cluster. The identifiers were extracted from the Object table using the following query: SELECT chunkId, MIN (objectId) FROM kpm50.Object GROUP BY chunkId; The query reported 146332 unique identifiers each mapping to a unique chunk (there are 146332 such chunks in the catalog). The collection was put into the following file of the Git package https://github.com/lsst-dm/qserv-perf-tests: datasets/kpm50/one_object_per_chunk.txt

              People

              Assignee:
              gapon Igor Gaponenko
              Reporter:
              gapon Igor Gaponenko
              Watchers:
              Andy Salnikov, Colin Slater, Fritz Mueller, Igor Gaponenko, Nate Pease
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated: