Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13944

add id to VisitInfo

    XMLWordPrintable

    Details

      Description

      VisitInfo needs to contain a visit-level identifier: visitId. This might be easiest to set after the gen3 butler is out (which will better codify the definition of visit).

      Note: per the discussion below, the end result was an id field on VisitInfo, which could hold either the full focal plane exposure or visit identifier (depending on whether we have a snap, or combined visit).

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            What you've written here will propagate the exposure_id through the system from the raws. Deciding how gen3 is going to change the ID to the gen3 visit_id is the responsibility of the pipeline task that actually converts and exposure to a visit.

            It would be very great if cameras that don't have an exposure/visit distinction (e.g. all of them, right now) had the same id for the raw and butler exposure and visit fields. I don't know how that all happens. We'll need a ticket to describe how that all should play out.

            Show
            Parejkoj John Parejko added a comment - What you've written here will propagate the exposure_id through the system from the raws. Deciding how gen3 is going to change the ID to the gen3 visit_id is the responsibility of the pipeline task that actually converts and exposure to a visit. It would be very great if cameras that don't have an exposure/visit distinction (e.g. all of them, right now) had the same id for the raw and butler exposure and visit fields. I don't know how that all happens. We'll need a ticket to describe how that all should play out.
            Hide
            tjenness Tim Jenness added a comment -

            The exposure_id in gen3 Butler is identical to the ObservationInfo.exposure_id (that's where the butler works it out). If someone writes their own raw-ingest task then we can't guarantee anything but for our code it's absolutely consistent. The raw formatter is the thing that guarantees it in the file but only if it also uses ObservationInfo (all ours do since they fill in VisitInfo using the MakeRawVisitInfoViaObsInfo function). Yes, a butler that uses one definition of exposure_id but a raw formatter that returns something else is by definition broken.

            visit ID is an entirely butler construct. Whatever is in the visit part of the DataId is the visit ID that must be put in the file that is written out.

            Show
            tjenness Tim Jenness added a comment - The exposure_id in gen3 Butler is identical to the ObservationInfo.exposure_id (that's where the butler works it out). If someone writes their own raw-ingest task then we can't guarantee anything but for our code it's absolutely consistent. The raw formatter is the thing that guarantees it in the file but only if it also uses ObservationInfo (all ours do since they fill in VisitInfo using the MakeRawVisitInfoViaObsInfo function). Yes, a butler that uses one definition of exposure_id but a raw formatter that returns something else is by definition broken. visit ID is an entirely butler construct. Whatever is in the visit part of the DataId is the visit ID that must be put in the file that is written out.
            Hide
            Parejkoj John Parejko added a comment - - edited

            My point is that we should ensure that exposure_id == visit_id, and that that value matches the butler visit field for all instruments that don't have an actual difference between "visit" and "exposure". I think that's a middleware question.

            Show
            Parejkoj John Parejko added a comment - - edited My point is that we should ensure that exposure_id == visit_id, and that that value matches the butler visit field for all instruments that don't have an actual difference between "visit" and "exposure". I think that's a middleware question.
            Hide
            tjenness Tim Jenness added a comment -

            My point is that the only place that visits come from is butler. Nothing else can work it out itself since it doesn't know the visit system that is being used. I do "ensure" that visit_id==exposure_id in butler for HSC and DECam because you will see that ObservationInfo.visit_id == ObservationInfo.exposure_id for those. Nothing should be assuming this to be true though.

            Show
            tjenness Tim Jenness added a comment - My point is that the only place that visits come from is butler. Nothing else can work it out itself since it doesn't know the visit system that is being used. I do "ensure" that visit_id==exposure_id in butler for HSC and DECam because you will see that ObservationInfo.visit_id == ObservationInfo.exposure_id for those. Nothing should be assuming this to be true though.
            Hide
            Parejkoj John Parejko added a comment -

            Thank you for the review and discussion, Tim Jenness and Jim Bosch. Now we at least have a place to put focal plane identifier numbers.

            Show
            Parejkoj John Parejko added a comment - Thank you for the review and discussion, Tim Jenness and Jim Bosch . Now we at least have a place to put focal plane identifier numbers.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Tim Jenness
              Watchers:
              Jim Bosch, John Parejko, Russell Owen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.