Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10627

Do not allow Executor for silent removal of existing dataset repositories

    Details

      Description

      Executor can work in two modes. It can either use existing input repository or create one from scratch in a specified location. Currently, when in latter mode, by default, it deletes any directory in that location before starting building a new one. The mode is selected automatically based on presence certain fields in job description. It may lead to accidental removal of an existing dataset repository if a malformed job description file is used. The goal is to make Executor much more careful when dealing with existing dataset repositories and remove anything only if explicitly asked.

        Attachments

          Activity

          Hide
          mkowalik Mikolaj Kowalik added a comment -

          Hsin-Fang Chiang Could you please take a look at one more modification of the Executor?

          Now it isn't trying to remove an input dataset repository behind the scenes in any case. Though, just in case, if you want to run it, use on a repository which accidental lost will be acceptable for you.

          Show
          mkowalik Mikolaj Kowalik added a comment - Hsin-Fang Chiang Could you please take a look at one more modification of the Executor ? Now it isn't trying to remove an input dataset repository behind the scenes in any case. Though, just in case, if you want to run it, use on a repository which accidental lost will be acceptable for you.
          Hide
          hchiang2 Hsin-Fang Chiang added a comment -

          The code changes look okay to me with a quick look. I did not try anything manually, though. I do not really understand the codes and the design either, but may not be able to look further before tomorrow. So if you want a more careful review, you may want to ask somebody else.

          Show
          hchiang2 Hsin-Fang Chiang added a comment - The code changes look okay to me with a quick look. I did not try anything manually, though. I do not really understand the codes and the design either, but may not be able to look further before tomorrow. So if you want a more careful review, you may want to ask somebody else.
          Hide
          mkowalik Mikolaj Kowalik added a comment -

          That's fine. The code is still rather experimental so all what I needed was a sanity check. Thanks!

          Show
          mkowalik Mikolaj Kowalik added a comment - That's fine. The code is still rather experimental so all what I needed was a sanity check. Thanks!

            People

            • Assignee:
              mkowalik Mikolaj Kowalik
              Reporter:
              mkowalik Mikolaj Kowalik
              Reviewers:
              Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, Mikolaj Kowalik
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: