Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-836

Schema changes for butler

    XMLWordPrintable

Details

    • RFC
    • Status: Implemented
    • Resolution: Done
    • DM
    • None

    Description

      A number of schema changes are required to support the new way that the commissioning team wish to define visits. Given that any schema changes are disruptive we would like to make a number of changes all at once.

      Motivated by Commissioning

      • Visits are now defined from the first sequence number in the visit and the expected end sequence number. Visits can now only have one or two snaps.
      • The exposure record therefore needs seq_start and seq_end fields and the visit record needs to have seq_num added since the visit is now defined in terms of the first sequence number in the visit. We also wish to store the group_id in the visit record to allow groups of visits to be found.
      • Some commissioning tasks require the azimuth to be recorded for the exposure. Zenith angle is already recorded. The average azimuth will also be added to the visit record.
      • Visit systems are now explicitly either always one exposure per visit or else one or two exposures per visit based on the seq_start/seq_end exposure values. There is no longer the concept of arbitrary visit systems as originally envisaged. Given that in both systems the visit_id will now be identical for the first exposure in the visit, we wish to register a default visit system per-instrument that can be used to deal with an ambiguous dataId.
      • We are now taking data at the summit which look like on-sky observations even though the instrument is not on the telescope. In order to make it possible to reliably filter out these observations from real on-sky observations we wish to add an "is_simulated" boolean field to the exposure record. This will be true if any system involved in the observation is simulated.

      Other dimension changes

      • Add support for healpix dimensions.

      Internal registry changes

      • Remove ON DELETE CASCADE from collection tag rows. Currently a dataset can be deleted from a RUN even if it is part of a TAGGED collection. This is considered dangerous since you may not be aware that it was in a TAGGED collection.

      Notes

      • Adding amplifier dimensions (DM-29584) is not being considered for this RFC. Is what we currently have usable by the commissioning team?
      • Opaque dataIds are something we need but feel that the implementation is sufficiently under-specified (DM-33751) that we should defer the work to a later schema change.

      Attachments

        Issue Links

          Activity

            erykoff Eli Rykoff added a comment -

            Enthusiastic +1 to the very short list of "other dimension changes". Healpix dimensions, yes please!

            erykoff Eli Rykoff added a comment - Enthusiastic +1 to the very short list of "other dimension changes". Healpix dimensions, yes please!
            tjenness Tim Jenness added a comment -

            After discussion we would also like to record the group_id in the visit record since this would allow us to determine groups of visits (such as focus sequences).

            tjenness Tim Jenness added a comment - After discussion we would also like to record the group_id in the visit record since this would allow us to determine groups of visits (such as focus sequences).
            tjenness Tim Jenness added a comment -

            Since visit_system will now only be able to have two states (a visit for every exposure or a visit based on the seq_start/seq_end metadata), we are considering whether treating it as a boolean can result in some improvements.

            tjenness Tim Jenness added a comment - Since visit_system will now only be able to have two states (a visit for every exposure or a visit based on the seq_start/seq_end metadata), we are considering whether treating it as a boolean can result in some improvements.

            This looks good to me, thank you, except that I have to check on DM-29584 with the camera team

            rhl Robert Lupton added a comment - This looks good to me, thank you, except that I have to check on DM-29584 with the camera team

            jchiang confirms that DM-29584 is not a problem (and can probably be closed)

            rhl Robert Lupton added a comment - jchiang confirms that DM-29584 is not a problem (and can probably be closed)

            People

              tjenness Tim Jenness
              tjenness Tim Jenness
              Andy Salnikov, Colin Slater, Eli Rykoff, Eric Bellm, Gregory Dubois-Felsmann, Jim Bosch, John Parejko, Kian-Tat Lim, Leanne Guy, Merlin Fisher-Levine, Michelle Butler [X] (Inactive), Peter Ferguson, Robert Lupton, Steve Pietrowicz, Tim Jenness, Wil O'Mullane, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                Jenkins

                  No builds found.