Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10624

Duplicate log files from running pipe_drivers tasks

    XMLWordPrintable

    Details

      Description

      Currently, running pipe_drivers tasks using SLURM gives one log file saved directly by lsst.log as job_$JOBNAME.log and another log file saved by the batch system catching the stdout/stderr. This seems unnecessarily confusing.

      IIRC I first added it to ease the transition from using pex_logging only, to mixing pex_logging+log, and to using only log in the pipelines. Now pex_logging is no longer used, maybe we only need to keep one of the two?

      I like Paul Price's idea at DM-8359:

      Maybe we should just remove the log file altogether, and rely on the batch system catching the stdout? Then we need to make sure that the host and PID are being prepended to the log messages.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            I believe we can close this ticket as won't fix because pipe_drivers+slurm is being replaced by bps.

            Show
            tjenness Tim Jenness added a comment - I believe we can close this ticket as won't fix because pipe_drivers+slurm is being replaced by bps.
            Hide
            jbosch Jim Bosch added a comment -

            Closing because it'll go away with Gen2, as Tim said.

            Show
            jbosch Jim Bosch added a comment - Closing because it'll go away with Gen2, as Tim said.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, Jim Bosch, Tim Jenness
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.