Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_persistence, pex_config, pipe_base
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:Science Pipelines DM-W16-1, Science Pipelines DM-W16-2
-
Team:Data Release Production
Description
This is a port of HSC-1106
When running tasks that write out config settings files (processCcd.py, for example), if multiple processes start simultaneously, an I/O race condition can occur in writing these files. This is solved here by writing to temp files and then renaming them to the correct destination filename in a single operation. Also, to avoid similar race conditions in the backup file creation (e.g. config.py~1, config.py~2, ...), a --no-backup-config option (to be used with --clobber-config) is added here to prevent the backup copies being made. The outcome for this option is that the config that are still recorded are for the most recent run.
Attachments
Issue Links
- blocks
-
- Invalid
- is triggering
-
- Won't Fix
- relates to
-
- Done
-
- Won't Fix
- mentioned in
-
Page Loading...
Activity
Great, thanks Nate Pease [X]. I have made all the changes and pushed tickets/DM-3911 branches. I am running a Jenkins now if you want to have a quick look. Otherwise, I'll assume I'm ok to merge once the Jenkins build is successful.
Ok, I checked the most recent changes in daf_persistence, they look great. I'll let Tim Jenness look at the other packages if he wants to; it's ok by me for you to checkin after Jenkins passes.
Merged to master.
...& added to release notes. Thanks!
Aside from needing to move the new code in butler to a separate file it all looks good to me. Tim pointed out a couple things that can be cleaned up too, I guess.