Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15536

Add level of indirection in defining Visits from Exposures

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      In discussions with Robert Lupton, we decided that it would be best for multiple mutually-inconsistent associations of exposures (snaps) into visits should be permitted in the same Gen3 data repository.  We cannot assume that we will be able to do that association in advance and not want to revisit it in later reprocessing.

      On the database side, I think we'd need to add a field to the visit table indicating which of several named systems the visit belongs to, while ensuring that visit IDs are unique across all systems.  We'd select exactly one system via configuration to be used by any butler client, though we'd need to think about when we'd have to be able to refer to other systems instead of relying on that selection (e.g. when interacting with datasets produced on some other system).  We'll also need to move the exposure.visit field into a separate join table.

      The way information about visits will be transmitted from the LSST hardware is via a "GROUP" header card; to make use of that, we should:

      • propagate that through ObservationInfo into the exposure table during raw ingest;
      • remove visit definition (and all of the stuff dealing with visit regions) from raw ingest;
      • add a new task to be run after ingest that creates visits (with regions, where appropriate) from groups of exposures with the same GROUP value.  That would involve generating visit IDs and names from GROUP via some algorithm.

      For instruments that want visits to be 1-1 with exposures, we'll just define GROUP accordingly.  If we ever want to define a set of visits that isn't strictly derived from GROUP, we can write an alternate version of the task that populates the same tables via some other means.

        Attachments

          Issue Links

            Activity

            No work has yet been logged on this issue.

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Tim Jenness
              Watchers:
              Gregory Dubois-Felsmann, Jim Bosch, Michelle Gower, Nate Lust, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.