Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-32692

Split tests in obs_lsst down

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: obs_lsst
    • Labels:
      None
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      I don't know which test is the super long one, but there is one very tall pole in the tests for obs_lsst (run scons -j 24 and watch all threads except one finish quite quickly, and then one test sit there finishing for ages).

      Finding which test this is and splitting it down will allow this package, and hence the whole stack, to build quite a bit faster (>1 min I bet).

        Attachments

          Activity

          Hide
          tjenness Tim Jenness added a comment - - edited

          pytest can tell you if you use the --durations=10 option.

          79.16s call     tests/test_ingest.py::LatissIngestTestCase::testCopy
          76.84s call     tests/test_convert2to3.py::ImSimGen2To3TestCase::test_convert
          74.71s call     tests/test_convert2to3.py::TS3Gen2To3TestCase::test_convert
          59.46s call     tests/test_ingest.py::LatissIngestTestCase::testHardLink
          59.44s call     tests/test_ingest.py::LatissIngestTestCase::testInPlace
          54.18s call     tests/test_convert2to3.py::PhoSimGen2To3TestCase::test_convert
          47.23s call     tests/test_convert2to3.py::TS8Gen2To3TestCase::test_convert
          46.78s call     tests/test_ingest.py::LatissIngestTestCase::testSymLink
          45.82s call     tests/test_ingest.py::LatissIngestTestCase::testLink
          34.80s call     tests/test_convert2to3.py::LatissGen2To3TestCase::test_convert
          

          which tells you that the ingest test cases and the 2to3 test cases are the problem but more investigation will be needed than a single test that takes a while.

          Show
          tjenness Tim Jenness added a comment - - edited pytest can tell you if you use the --durations=10 option. 79.16s call tests/test_ingest.py::LatissIngestTestCase::testCopy 76.84s call tests/test_convert2to3.py::ImSimGen2To3TestCase::test_convert 74.71s call tests/test_convert2to3.py::TS3Gen2To3TestCase::test_convert 59.46s call tests/test_ingest.py::LatissIngestTestCase::testHardLink 59.44s call tests/test_ingest.py::LatissIngestTestCase::testInPlace 54.18s call tests/test_convert2to3.py::PhoSimGen2To3TestCase::test_convert 47.23s call tests/test_convert2to3.py::TS8Gen2To3TestCase::test_convert 46.78s call tests/test_ingest.py::LatissIngestTestCase::testSymLink 45.82s call tests/test_ingest.py::LatissIngestTestCase::testLink 34.80s call tests/test_convert2to3.py::LatissGen2To3TestCase::test_convert which tells you that the ingest test cases and the 2to3 test cases are the problem but more investigation will be needed than a single test that takes a while.
          Hide
          tjenness Tim Jenness added a comment -

          70% of the time in th LATISS ingest tests is spent reading the data back in and doing the amplifier testing and that was added in DM-29370 (in the checkRepo method).

          Show
          tjenness Tim Jenness added a comment - 70% of the time in th LATISS ingest tests is spent reading the data back in and doing the amplifier testing and that was added in DM-29370 (in the checkRepo method).

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            mfisherlevine Merlin Fisher-Levine
            Watchers:
            Christopher Waters, Merlin Fisher-Levine, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:

                Jenkins Builds

                No builds found.