Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-658

Database name used by integration tests should use their own dedicated database, not "LSST"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      Currently automated tests use the database "LSST", and they will go ahead and delete that database without any warning. This is far from ideal, we should have a more unique database name. How about something like qservAutoTest_<uniqueId>.

        Attachments

          Issue Links

            Activity

            Hide
            jammes Fabrice Jammes added a comment -

            Be careful of collisions between users concurrently launching the same test case on the same Qserv instance.

            Show
            jammes Fabrice Jammes added a comment - Be careful of collisions between users concurrently launching the same test case on the same Qserv instance.
            Hide
            vaikunth Vaikunth Thukral added a comment -

            Hi Jacek,

            This is ready to review, updates were made to both qserv and qserv_testdata. Not sure who else to set as reviewer.

            Branch:
            u/vaikunth/DM-658

            Show
            vaikunth Vaikunth Thukral added a comment - Hi Jacek, This is ready to review, updates were made to both qserv and qserv_testdata. Not sure who else to set as reviewer. Branch: u/vaikunth/ DM-658
            Hide
            jbecla Jacek Becla added a comment -

            Vaikunth: looks good. I didn't try if it works, I only looked at the code. Small comments are on github. It might be useful to document somewhere how to change the db name for integration tests. Adding a comment to this story would be a start, although I am sure there are other places in our documentation where we could add that. Thanks!

            Show
            jbecla Jacek Becla added a comment - Vaikunth: looks good. I didn't try if it works, I only looked at the code. Small comments are on github. It might be useful to document somewhere how to change the db name for integration tests. Adding a comment to this story would be a start, although I am sure there are other places in our documentation where we could add that. Thanks!
            Hide
            vaikunth Vaikunth Thukral added a comment -

            As you suggested, I'll summarize here how we changed the db name for the integration tests in a simple form to begin with:

            1) The DB name was hard-coded to "LSST" and created during the configuration procedure. The same name followed through into qservMeta and qservw_worker during their creation, as well as into the individual test cases.
            2) The creation of these databses is removed from the configuration procedure in qserv.
            3) In qserv_testdata, qservdataloader.py and benchmark.py have been updated to use an automated name, depending on the test case currently running.
            4) The same name, of the form "qservTest_caseXX", is used to create the relevant database names during the tests (and not at config step), as well as table names in qservMeta and qservw_worker.
            5) There are also some updates regarding the loading and deletion of these test cases so they do not interfere with each other and can be run individually in any sequence with "qserv-check-intergration.py --case=XX" or all at once with "qserv-test-integration.py"

            I think this covers most of it, Fabrice Jammes may have some comments or corrections to what I've listed.

            Show
            vaikunth Vaikunth Thukral added a comment - As you suggested, I'll summarize here how we changed the db name for the integration tests in a simple form to begin with: 1) The DB name was hard-coded to "LSST" and created during the configuration procedure. The same name followed through into qservMeta and qservw_worker during their creation, as well as into the individual test cases. 2) The creation of these databses is removed from the configuration procedure in qserv. 3) In qserv_testdata, qservdataloader.py and benchmark.py have been updated to use an automated name, depending on the test case currently running. 4) The same name, of the form "qservTest_caseXX", is used to create the relevant database names during the tests (and not at config step), as well as table names in qservMeta and qservw_worker. 5) There are also some updates regarding the loading and deletion of these test cases so they do not interfere with each other and can be run individually in any sequence with "qserv-check-intergration.py --case=XX" or all at once with "qserv-test-integration.py" I think this covers most of it, Fabrice Jammes may have some comments or corrections to what I've listed.
            Hide
            jammes Fabrice Jammes added a comment -

            Thanks Vaikunth Thukral for this good summary.

            Show
            jammes Fabrice Jammes added a comment - Thanks Vaikunth Thukral for this good summary.
            Hide
            vaikunth Vaikunth Thukral added a comment -

            This is finished, just wanted to mention that one of the comments was regarding the naming of emptyChunkPath and emptyChunkListFile in qser-czar configuration. I've added comments in the file about why we needed 2 separate variables, but a proper cleanup would likely require a new ticket.

            Show
            vaikunth Vaikunth Thukral added a comment - This is finished, just wanted to mention that one of the comments was regarding the naming of emptyChunkPath and emptyChunkListFile in qser-czar configuration. I've added comments in the file about why we needed 2 separate variables, but a proper cleanup would likely require a new ticket.

              People

              Assignee:
              vaikunth Vaikunth Thukral
              Reporter:
              fritzm Fritz Mueller
              Reviewers:
              Jacek Becla
              Watchers:
              Fabrice Jammes, Jacek Becla, Vaikunth Thukral
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.