Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28922

pipe.tasks.Functor doesn't work with deferred dataref if not multi-level

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      2
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      DM-27008 updated pipe.tasks.functors to work with DeferredDatasetRefs. However, this appears to be only tested for multilevel-column parquet tables. The following should work to compute the dec in degrees, but there's a bug to fix:

      from lsst.daf.butler import Butler
       
      butler = Butler('/project/hsc/gen3repo/rc2w02_ssw03')
      collection = 'u/sr525/colorAnalysisTest2'
       
      ref = list(butler.registry.queryDatasets('qaTractTable_forced', collections=collection))[0]
      deferredRef = butler.getDeferred(ref, collections=collection)
       
      from lsst.pipe.tasks.functors import RAColumn
       
      ra_func = RAColumn()
      ra_func(deferredRef)
      

      Should be straightforward to fix this and add an appropriate test.

        Attachments

          Issue Links

            Activity

            Show
            tmorton Tim Morton [X] (Inactive) added a comment - Jenkins running at https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33624/pipeline
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Looks good. Happy to see this going through as I'm finishing up DM-18000 since the data AP feeds through the functors won't be multi-level.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Looks good. Happy to see this going through as I'm finishing up DM-18000 since the data AP feeds through the functors won't be multi-level.

              People

              Assignee:
              tmorton Tim Morton [X] (Inactive)
              Reporter:
              tmorton Tim Morton [X] (Inactive)
              Reviewers:
              Chris Morrison
              Watchers:
              Chris Morrison, Chris Morrison [X] (Inactive), Tim Morton [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.