Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-32220

HTCondor plugin copying execution butler back from job

    XMLWordPrintable

Details

    Description

      Discovered that the plugin is copying back the sqlite3 file at the end of the job. Change the code so that it doesn't.

      Attachments

        Issue Links

          Activity

            I didn't notice this with the (no bps) htcondor only tests I did for the previous transfer ticket because the executable didn't modify the sqlite file. So in the previous tests, it didn't bring back the sqlite file. After adding a "touch" to the script, the test did bring back the sqlite file. I double checked that Brock's jobs aren't bringing back the butler.yaml file which makes sense because that file isn't being modified by pipetask.

            mgower Michelle Gower added a comment - I didn't notice this with the (no bps) htcondor only tests I did for the previous transfer ticket because the executable didn't modify the sqlite file. So in the previous tests, it didn't bring back the sqlite file. After adding a "touch" to the script, the test did bring back the sqlite file. I double checked that Brock's jobs aren't bringing back the butler.yaml file which makes sense because that file isn't being modified by pipetask.

            From the HTCondor manual:

            To restrict the output files or permit entire directory contents to be transferred, specify the exact list with transfer_output_files . Delimit the list of file names, directory names, or paths with commas. When this list is defined, and any of the files or directories do not exist as the job exits, HTCondor considers this an error, and places the job on hold. Setting transfer_output_files to the empty string (“”) means no files are to be transferred.
            

            Tested the empty string with my updated (no bps) htcondor only test and that brings back the stdout/stderr but not the sqlite file.

            mgower Michelle Gower added a comment - From the HTCondor manual: To restrict the output files or permit entire directory contents to be transferred, specify the exact list with transfer_output_files . Delimit the list of file names, directory names, or paths with commas. When this list is defined, and any of the files or directories do not exist as the job exits, HTCondor considers this an error, and places the job on hold. Setting transfer_output_files to the empty string (“”) means no files are to be transferred. Tested the empty string with my updated (no bps) htcondor only test and that brings back the stdout/stderr but not the sqlite file.

            Changes approved, can be merged to the main branch.

            mkowalik Mikolaj Kowalik added a comment - Changes approved, can be merged to the main branch.

            Your backport request has been approved. See my longer comment on DM-32217 for more info on backporting

            yusra Yusra AlSayyad added a comment - Your backport request has been approved. See my longer comment on DM-32217 for more info on backporting

            People

              mgower Michelle Gower
              mgower Michelle Gower
              Mikolaj Kowalik
              Brock Brendal [X] (Inactive), Michelle Gower, Mikolaj Kowalik, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.