Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_butler
-
Story Points:2
-
Team:Architecture
-
Urgent?:No
Description
During development of a pipelinetask it may happen that the developer will erroneously define an incorrect datasetType (with incorrect dimensions, or storageClass, etc.). At this point it is impossible to recover without blowing away the registry or sqlite hacking because any fix to the entry will raise a ValueError: Supplied dataset type inconsistent with registry definition error.
We should not allow the ability to delete datasets if a dataset type is associated with them that is being deleted. This would imply that the dataset type has been used. Instead a user should first delete the relevant collection/dataset before deleting the dataset type.
Attachments
Issue Links
Activity
I was wondering about the user interface. How brave are we allowing anyone to force delete anything (by mistake using --force on a calexp dataset type for example). As opposed to having pipetask have an option for overriding the dataset type definition (by pruning and re-registering).
The database constraints as defined will guarantee that we get an exception unless there are no datasets of this dataset type at all in the repo, and I was planning to embrace that and leave it to users to clean up datasets first.
Ok, so you *don't* want the proposed force=True option then.
I think that's where I stand, though I realize now I had only skimmed the original ticket description and hadn't noticed that part of it until now.
Basically, it's really easy to write a super safe version of this without --force because the database constraint does all of the work, and it's a pain to write a version with --force that's any kind of safe because dataset deletion across Registry and Datastore is so fraught. So I think we need a very good reason to do --force.
Hopefully a quick review. I've self-reviewed it on GitHub with all the things I'm uncertain about. Probably don't need two reviewers so whoever gets there first is fine.
Reviewed; mostly just answers to your questions.
I reviewed it too, looks good.
This should probably be accompanied by a command-line tool (as a butler subcommand).