Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7346

Modify obs_sdss to support pytest

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      DRP F16-3, DRP F16-4
    • Team:
      Data Release Production

      Description

      This ticket is for the work of migrating the obs_sdss tests such that they run with the py.test test runner.

        Attachments

          Issue Links

            Activity

            Hide
            fred3m Fred Moolekamp added a comment -

            Currently this test depends on a connection to "lsst-db.ncsa.illinois.edu" to download SDSS images for the tests. In the long run it might be better to include those images in the test directory so that the tests are run properly every time.

            Show
            fred3m Fred Moolekamp added a comment - Currently this test depends on a connection to "lsst-db.ncsa.illinois.edu" to download SDSS images for the tests. In the long run it might be better to include those images in the test directory so that the tests are run properly every time.
            Hide
            tjenness Tim Jenness added a comment -

            It's interesting that the database is hard-coded into the obs_sdss module with Simon Krughoff's name in it. How much data volume are we talking about?

            In daf_persistence this is how tests are skipped: https://github.com/lsst/daf_persistence/blob/master/tests/DbStorage_1.py#L40

            Show
            tjenness Tim Jenness added a comment - It's interesting that the database is hard-coded into the obs_sdss module with Simon Krughoff 's name in it. How much data volume are we talking about? In daf_persistence this is how tests are skipped: https://github.com/lsst/daf_persistence/blob/master/tests/DbStorage_1.py#L40
            Hide
            fred3m Fred Moolekamp added a comment -

            I'm not sure how much data is there, I don't have access to that repository. In this case I think it was better to skip the tests by function as opposed to by class, since some classes had both skipped and not skipped methods. Unless we want to restructure the tests so that all of the functions that require the external dataset are in one class while the tests that can be run regardless are in another, but if we copy the data into the repo this is not necessary.

            Show
            fred3m Fred Moolekamp added a comment - I'm not sure how much data is there, I don't have access to that repository. In this case I think it was better to skip the tests by function as opposed to by class, since some classes had both skipped and not skipped methods. Unless we want to restructure the tests so that all of the functions that require the external dataset are in one class while the tests that can be run regardless are in another, but if we copy the data into the repo this is not necessary.
            Hide
            tjenness Tim Jenness added a comment -

            Skipping each test independently is fine.

            Show
            tjenness Tim Jenness added a comment - Skipping each test independently is fine.
            Hide
            nlust Nate Lust added a comment -

            Add a bit of documentation above the dbauth section explaining what its there for and it looks good to merge for me.

            Show
            nlust Nate Lust added a comment - Add a bit of documentation above the dbauth section explaining what its there for and it looks good to merge for me.
            Hide
            tjenness Tim Jenness added a comment - - edited

            I've just run py.test on the testSelectSdssImages.py file (for DM-7509) and it looks like the standard error output does not get captured because it turns up before the tests are run.

            I think the fix is to move the logic to setUpClass or alternatively change the skip message itself to include the information that is currently printed. At the moment the information just gets lost and you get the simplistic skip message. It does, of course, turn up if you just run the test by hand from the command line with python but that won't help Jenkins.

            Show
            tjenness Tim Jenness added a comment - - edited I've just run py.test on the testSelectSdssImages.py file (for DM-7509 ) and it looks like the standard error output does not get captured because it turns up before the tests are run. I think the fix is to move the logic to setUpClass or alternatively change the skip message itself to include the information that is currently printed. At the moment the information just gets lost and you get the simplistic skip message. It does, of course, turn up if you just run the test by hand from the command line with python but that won't help Jenkins.
            Hide
            fred3m Fred Moolekamp added a comment -

            Should I open a new ticket to fix this?

            Show
            fred3m Fred Moolekamp added a comment - Should I open a new ticket to fix this?
            Hide
            tjenness Tim Jenness added a comment -

            See DM-7523

            Show
            tjenness Tim Jenness added a comment - See DM-7523

              People

              • Assignee:
                fred3m Fred Moolekamp
                Reporter:
                fred3m Fred Moolekamp
                Reviewers:
                Nate Lust
                Watchers:
                Fred Moolekamp, Nate Lust, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel