Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_persistence, pex_config, pipe_base
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:Science Pipelines DM-W16-1, Science Pipelines DM-W16-2
-
Team:Data Release Production
Description
This is a port of HSC-1106
When running tasks that write out config settings files (processCcd.py, for example), if multiple processes start simultaneously, an I/O race condition can occur in writing these files. This is solved here by writing to temp files and then renaming them to the correct destination filename in a single operation. Also, to avoid similar race conditions in the backup file creation (e.g. config.py~1, config.py~2, ...), a --no-backup-config option (to be used with --clobber-config) is added here to prevent the backup copies being made. The outcome for this option is that the config that are still recorded are for the most recent run.
Attachments
Issue Links
- blocks
-
DM-3937 HSC backport: temporary file handling in butler
- Invalid
- is triggering
-
DM-4215 Butler: SafeFile and SafeFileName can overwrite good with bad in some cases
- Won't Fix
- relates to
-
DM-3372 Port, replace, or defer HSC-side provenance of EUPS products
- Done
-
DM-4173 Butler: add support for write-once-compare-same outputs
- Won't Fix
- mentioned in
-
Page Loading...
Aside from needing to move the new code in butler to a separate file it all looks good to me. Tim pointed out a couple things that can be cleaned up too, I guess.