Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15851

sconsUtils does not reliably remove the .failed file

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: sconsUtils
    • Labels:
      None
    • Sprint:
      Arch 2018-10-01, Arch 2018-10-08, Arch 2018-10-15
    • Team:
      Architecture

      Description

      Following the merge of DM-11693, we are discovering that sconsUtils is no longer deleting the .xml.failed file when it is re-run. The "running tests twice as part of install" problem has been fixed but normal developer workflow of typing scons, getting a failure, fixing the failure, typing scons again, is now broken. This is because the second time you type scons the .failed file is not removed, so pytest now passes but sconsUtils sees the .failed file and stops everything and reports that there was a problem (when there wasn't). Being forced to manually remove the .failed file is annoying.

        Attachments

          Issue Links

            Activity

            No builds found.
            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Link This issue is triggered by DM-11693 [ DM-11693 ]
            tjenness Tim Jenness made changes -
            Risk Score 0
            Hide
            bvan Brian Van Klaveren added a comment -

            I think if somebody is building all targets for SCons then we need to always obliterate the previous pytest information.

            If somebody is only performing a scons install, then we DO NOT obliterate previous outputs.

            Show
            bvan Brian Van Klaveren added a comment - I think if somebody is building all targets for SCons then we need to always obliterate the previous pytest information. If somebody is only performing a scons install, then we DO NOT obliterate previous outputs.
            Hide
            tjenness Tim Jenness added a comment -

            Developers type scons repeatedly and so we have to make that work.

            Show
            tjenness Tim Jenness added a comment - Developers type scons repeatedly and so we have to make that work.
            Hide
            erykoff Eli Rykoff added a comment -

            I'm not sure what the difference is between "building all targets" and "only performing a scons install", especially for a python-only package.

            Show
            erykoff Eli Rykoff added a comment - I'm not sure what the difference is between "building all targets" and "only performing a scons install", especially for a python-only package.
            Hide
            tjenness Tim Jenness added a comment -

            See the linked ticket DM-11693scons always runs the tests, and scons install always runs the tests, so when you use eupspkg to install you run the tests twice unless the second time scons can work out that it doesn't need to.

            Show
            tjenness Tim Jenness added a comment - See the linked ticket DM-11693 – scons always runs the tests, and scons install always runs the tests, so when you use eupspkg to install you run the tests twice unless the second time scons can work out that it doesn't need to.
            Hide
            bvan Brian Van Klaveren added a comment - - edited

            Let me rephrase that - If a developer explicitly asks to execute a scons install - which is actually rare and something that is mostly executed in CI or in eupspkg install (I think), then we do not clean up previous .failed files. Otherwise, we always clean up .failed files.

             

            A fix would add:

            @rm -f ${{TARGET $TARGET.failed;}}

            Back into the SCons Command that executes pytest, before running pytest, if there's no install is not in SCons.Script.COMMAND_LINE_TARGETS.

            Show
            bvan Brian Van Klaveren added a comment - - edited Let me rephrase that - If a developer explicitly asks to execute a scons install - which is actually rare and something that is mostly executed in CI or in eupspkg install (I think), then we do not clean up previous .failed files. Otherwise, we always clean up .failed files.   A fix would add: @rm -f ${{TARGET $ TARGET .failed;}} Back into the SCons Command that executes pytest, before running pytest, if there's no install  is not in SCons.Script.COMMAND_LINE_TARGETS.
            Hide
            erykoff Eli Rykoff added a comment -

            Okay, I see.  That makes sense to me.

            Show
            erykoff Eli Rykoff added a comment - Okay, I see.  That makes sense to me.
            bvan Brian Van Klaveren made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            bvan Brian Van Klaveren added a comment -

            I have a fix in, this should obliterate files when you are not performing an install, and preserve the files if you are doing an install

            Show
            bvan Brian Van Klaveren added a comment - I have a fix in, this should obliterate files when you are not performing an install, and preserve the files if you are doing an install
            Hide
            bvan Brian Van Klaveren added a comment -

            Jenkins job: 28745, passed linux so far, still waiting on Mac.

            I tried testing this locally with a test case that fails and it seems to be working as intended.

            Show
            bvan Brian Van Klaveren added a comment - Jenkins job: 28745, passed linux so far, still waiting on Mac. I tried testing this locally with a test case that fails and it seems to be working as intended.
            bvan Brian Van Klaveren made changes -
            Reviewers Tim Jenness [ tjenness ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            ktl Kian-Tat Lim made changes -
            Sprint Arch 2018-10-01 [ 788 ] Arch 2018-10-01, Arch 2018-10-08 [ 788, 789 ]
            ktl Kian-Tat Lim made changes -
            Sprint Arch 2018-10-01, Arch 2018-10-08 [ 788, 789 ] Arch 2018-10-01, Arch 2018-10-08, Arch 2018-10-15 [ 788, 789, 798 ]
            Hide
            bvan Brian Van Klaveren added a comment -

            Tim Jenness I think this should be fixed adequately. I ran a CI 10/12/2018 (job #28844) and it seems to be behaving as normal when running scons directly.

            Show
            bvan Brian Van Klaveren added a comment - Tim Jenness I think this should be fixed adequately. I ran a CI 10/12/2018 (job #28844) and it seems to be behaving as normal when running scons directly.
            Hide
            tjenness Tim Jenness added a comment -

            I can confirm that this fixes the problem for me.

            Show
            tjenness Tim Jenness added a comment - I can confirm that this fixes the problem for me.
            tjenness Tim Jenness made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            bvan Brian Van Klaveren added a comment -

            Merged.

            Show
            bvan Brian Van Klaveren added a comment - Merged.
            bvan Brian Van Klaveren made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              bvan Brian Van Klaveren
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Tim Jenness
              Watchers:
              Brian Van Klaveren, Eli Rykoff, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.