Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ci_hsc_gen3, daf_butler, ip_isr, obs_base, obs_decam, obs_lsst, obs_subaru, pipe_base, pipe_tasks
-
Labels:
-
Story Points:0
-
Team:Architecture
-
Urgent?:No
Description
At present, RawIngestTask is responsible for adding three different types of dimension records:
- exposure
- visit
- visit_detector_region
We should move the latter two to a separate script that is run after RawIngestTask and (at least at present) only after all associated raws for a visit have been ingested. That will open the door to (later) being able to define different visit definitions for the same exposure (DM-15536).
This will not on its own allow individual raws to be ingested incrementally - RawIngestTask still needs all raws for an exposure to be present in a single invocation so we can add the exposure dimension record once without needing to see if it already exists in the Registry. That could be fixed by using Database.sync in a new high-level counterpart to Registry.insertDimensionData that does perform such a check, but we'll first need to figure out how to handle transactions in ingest - Database.sync is not and probably cannot be part of a larger transaction.
But even that is an easier problem to solve than the corresponding one for visits, because (in addition to the synchronization problem) the visit dimension record needs to contain a region that is currently computed from the regions of all of the detectors that went into it (but could be computed from cameraGeom in the future).
All work was done on
DM-15536; I should have closed this a while ago.