Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3911

HSC backport: avoid I/O race conditions config write out

    Details

    • Templates:
    • Story Points:
      1
    • Sprint:
      Science Pipelines DM-W16-1, Science Pipelines DM-W16-2
    • Team:
      Data Release Production

      Description

      This is a port of HSC-1106

      When running tasks that write out config settings files (processCcd.py, for example), if multiple processes start simultaneously, an I/O race condition can occur in writing these files. This is solved here by writing to temp files and then renaming them to the correct destination filename in a single operation. Also, to avoid similar race conditions in the backup file creation (e.g. config.py~1, config.py~2, ...), a --no-backup-config option (to be used with --clobber-config) is added here to prevent the backup copies being made. The outcome for this option is that the config that are still recorded are for the most recent run.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  lauren Lauren MacArthur
                  Reporter:
                  lauren Lauren MacArthur
                  Reviewers:
                  Nate Pease
                  Watchers:
                  John Swinbank, Kian-Tat Lim, Lauren MacArthur, Nate Pease, Paul Price
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel