Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31079

Butler define-visits ignores collection

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      It seems that the define-visits command line is not respecting the collections parameter and so every single call to define-visits runs on every single "exposure" for that instrument.

      This was noted on a community post and looking at the code the queryDataIds is not constrained by the collection at all. The collection is passed to the DefineVisitsTask but is only used to retrieve the relevant camera (which means it's ignored and the default camera is returned). If registry is not used to work out the WCS the collection is used again to get the WCS for the specific raw.

      This implies that there are really two collections of interest.

      • The collection constraining the queryDataIds and associated with the butler.get of raw.wcs (if needed).
      • The calibration collection used to locate the camera geometry.

      As currently implemented DefineVisitsTask is effectively ignoring the collections parameter because loadCamera falls back to the generic one and we use registry to calculate the WCS.

      The quick fix here is to add collections to the queryDataIds method in the script layer.

      Longer term we will need to consider adding a calibration collection parameter.

        Attachments

          Issue Links

            Activity

            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Remote Link This issue links to "Community Post (Web Link)" [ 29267 ]
            tjenness Tim Jenness made changes -
            Description It seems that the define-visits command line is not respecting the collections parameter and so every single call to define-visits runs on every single "exposure" for that instrument.

            This was noted on [a community post|https://community.lsst.org/t/define-visits-for-a-single-collection/5236/2] and looking at the code the {{queryDataIds}} is not constrained by the collection at all. The collection is passed to the DefineVisitsTask but is only used to retrieve the relevant camera (which means it's ignored and the default camera is returned). If registry is not used to work out the WCS the collection is used again to get the WCS for the specific raw.

            This implies that there are really two collections of interest.

            * The collection constraining the queryDataIds and associated with the butler.get of raw.wcs (if needed)..
            * The calibration collection used to locate the camera geometry.

            As currently implemented DefineVisitsTask is effectively ignoring the collections parameter because loadCamera falls back to the generic one and we use registry to calculate the WCS.

            The quick fix here is to add collections to the queryDataIds method in the script layer.

            Longer term we will need to consider adding a calibration collection parameter.
            It seems that the define-visits command line is not respecting the collections parameter and so every single call to define-visits runs on every single "exposure" for that instrument.

            This was noted on [a community post|https://community.lsst.org/t/define-visits-for-a-single-collection/5236/2] and looking at the code the {{queryDataIds}} is not constrained by the collection at all. The collection is passed to the DefineVisitsTask but is only used to retrieve the relevant camera (which means it's ignored and the default camera is returned). If registry is not used to work out the WCS the collection is used again to get the WCS for the specific raw.

            This implies that there are really two collections of interest.

            * The collection constraining the queryDataIds and associated with the butler.get of raw.wcs (if needed).
            * The calibration collection used to locate the camera geometry.

            As currently implemented DefineVisitsTask is effectively ignoring the collections parameter because loadCamera falls back to the generic one and we use registry to calculate the WCS.

            The quick fix here is to add collections to the queryDataIds method in the script layer.

            Longer term we will need to consider adding a calibration collection parameter.
            tjenness Tim Jenness made changes -
            Assignee Tim Jenness [ tjenness ]
            tjenness Tim Jenness made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            tjenness Tim Jenness made changes -
            Reviewers Jim Bosch [ jbosch ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            tjenness Tim Jenness made changes -
            Reviewers Jim Bosch [ jbosch ] Nate Pease [ npease ]
            npease Nate Pease [X] (Inactive) made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Nate Pease [X] (Inactive)
              Watchers:
              Jim Bosch, Nate Pease [X] (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.