Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30948

Change gen3 schema for visit to add seq_num

    XMLWordPrintable

    Details

    • Story Points:
      4
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      Robert Lupton has requested that the first sequence number in a visit be added to the visit dimension record definition in the gen3 registry schema. We also wish to store the group_id to allow grouping of visits.

      In the general case this will allow a user to specify a dataId for the raw that can also be used to obtain the calexp so long as they are specifying the first raw and not the second raw.

      day_obs is already in the visit definition.

      There are some caveats:

      • If a registry has multiple visit_system definitions the day_obs+seq_num may not be able to uniquely identify a calexp.
      • Visit definitions use the exposure dimension record (they do not look at files) so determining the first sequence in the visit depends on all the visit members being ingested.

      To make this robust I think ObservationInfo is going to have to be modified to have the first_seq and end_seq concepts from CAP-763 – this would require that we also change the exposure dimension record to include first_seq and end_seq since without the former we can't define the visit and without the latter the tooling can't warn about possible missing exposures in the visit definition.

        Attachments

          Issue Links

            Activity

            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Link This issue relates to CAP-763 [ CAP-763 ]
            tjenness Tim Jenness made changes -
            Labels gen3-middleware gen3-middleware gen3-registry-incompatibility
            tjenness Tim Jenness made changes -
            Link This issue relates to DM-33942 [ DM-33942 ]
            tjenness Tim Jenness made changes -
            Description [~rhl] has requested that the first sequence number in a visit be added to the visit dimension record definition in the gen3 registry schema.

            In the general case this will allow a user to specify a dataId for the raw that can also be used to obtain the calexp so long as they are specifying the first raw and not the second raw.

            day_obs is already in the visit definition.

            There are some caveats:

            * If a registry has multiple visit_system definitions the day_obs+seq_num may not be able to uniquely identify a calexp.
            * Visit definitions use the exposure dimension record (they do not look at files) so determining the first sequence in the visit depends on all the visit members being ingested.

            To make this robust I think ObservationInfo is going to have to be modified to have the first_seq and end_seq concepts from CAP-763 -- this would require that we also change the exposure dimension record to include first_seq and end_seq since without the former we can't define the visit and without the latter the tooling can't warn about possible missing exposures in the visit definition.
            [~rhl] has requested that the first sequence number in a visit be added to the visit dimension record definition in the gen3 registry schema. We also wish to store the group_id to allow grouping of visits.

            In the general case this will allow a user to specify a dataId for the raw that can also be used to obtain the calexp so long as they are specifying the first raw and not the second raw.

            day_obs is already in the visit definition.

            There are some caveats:

            * If a registry has multiple visit_system definitions the day_obs+seq_num may not be able to uniquely identify a calexp.
            * Visit definitions use the exposure dimension record (they do not look at files) so determining the first sequence in the visit depends on all the visit members being ingested.

            To make this robust I think ObservationInfo is going to have to be modified to have the first_seq and end_seq concepts from CAP-763 -- this would require that we also change the exposure dimension record to include first_seq and end_seq since without the former we can't define the visit and without the latter the tooling can't warn about possible missing exposures in the visit definition.
            tjenness Tim Jenness made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] Done [ 10002 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Jim Bosch, Kian-Tat Lim, Merlin Fisher-Levine, Robert Lupton, Tim Jenness
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.