Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28922

pipe.tasks.Functor doesn't work with deferred dataref if not multi-level

    XMLWordPrintable

Details

    • Bug
    • Status: Done
    • Resolution: Done
    • None
    • None
    • None
    • 2
    • Data Release Production
    • No

    Description

      DM-27008 updated pipe.tasks.functors to work with DeferredDatasetRefs. However, this appears to be only tested for multilevel-column parquet tables. The following should work to compute the dec in degrees, but there's a bug to fix:

      from lsst.daf.butler import Butler
       
      butler = Butler('/project/hsc/gen3repo/rc2w02_ssw03')
      collection = 'u/sr525/colorAnalysisTest2'
       
      ref = list(butler.registry.queryDatasets('qaTractTable_forced', collections=collection))[0]
      deferredRef = butler.getDeferred(ref, collections=collection)
       
      from lsst.pipe.tasks.functors import RAColumn
       
      ra_func = RAColumn()
      ra_func(deferredRef)
      

      Should be straightforward to fix this and add an appropriate test.

      Attachments

        Issue Links

          Activity

            tmorton Tim Morton [X] (Inactive) added a comment - Jenkins running at https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33624/pipeline

            Looks good. Happy to see this going through as I'm finishing up DM-18000 since the data AP feeds through the functors won't be multi-level.

            cmorrison Chris Morrison [X] (Inactive) added a comment - Looks good. Happy to see this going through as I'm finishing up DM-18000 since the data AP feeds through the functors won't be multi-level.

            People

              tmorton Tim Morton [X] (Inactive)
              tmorton Tim Morton [X] (Inactive)
              Chris Morrison
              Chris Morrison, Chris Morrison [X] (Inactive), Tim Morton [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.