Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30536

Emergent Work and Planned Gen3 Middleware Work for DRP in F21B

    XMLWordPrintable

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      f21b-drp-emergent
    • Story Points:
      63
    • WBS:
      02C.04.01
    • Team:
      Data Release Production
    • Cycle:
      Fall 2021

      Description

      In addition to miscellaneous minor improvements to low-level primitives and middleware, this epic includes a few major projects:

      • reorganizing pipeline and configuration file content for DRP as per RFC-775;
      • improving performance, flexibility, and diagnostics when building QuantumGraphs;
      • prototyping a new architecture for scalable PipelineTask execution;
      • designing improved user interfaces for local execution of PipelineTasks.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
         
        		DM-29884

        Fix exception in Gen3+applyColorTerms logic branch of jointcal

        		 1 Arun Kannawadi Done
         
        		DM-30624

        Document gen2 to gen3 refcat ingestion

        		 2 Lee Kelvin Done
         
        		DM-32330

        Register DECam narrow-band filters in skymap

        		 1 Lee Kelvin Done
         
        		DM-32294

        Clarify dataset ID vs. UUID in butler docs

        		 2 Jim Bosch Done
         
        		DM-31903

        Add update kwargs/option to instrument registration

        		 1 Jim Bosch Done
         
        		DM-31254

        afw.geom.Polygon.createImage yields different data points for Intel and ARM

        		 2 Eli Rykoff Done
         
        		DM-28111

        Include RUN name in QG dot diagrams for resolved DatasetRefs

        		 1 Jim Bosch Done
         
        		DM-32487

        fix compilation for osx-arm64 for jointcal

        		 2 Eli Rykoff Done
         
        		DM-32463

        ndarray online docs are empty

        		 0.5 Jim Bosch Done
         
        		DM-32459

        Improve QG gen diagnostics and update FAQ accordingly

        		 2 Jim Bosch Done
         
        		DM-32454

        Dataset constraints in queryDimensionRecords can't involve extra dimensions

        		 2 Jim Bosch Done
         
        		DM-32583

        Remove incorrect python linking and unnecessary macos flat_namespace

        		 4 Eli Rykoff Done
         
        		DM-31966

        Minimal viable pipetask python API

        		 3 Jim Bosch Done
         
        		DM-32030

        Fixup Jointcal after DM-31825

        		 2 Yusra AlSayyad Done
         
        		DM-31583

        Add interfaces to dataset summary table data and use it to improve query generation

        		 4 Jim Bosch Done
         
        		DM-31548

        patch_skypix_overlap table needs an additional index

        		 3 Jim Bosch Done
         
        		DM-32733

        Config.loadFromStream doesn't actually accept a file-like object

        		 1 Jim Bosch Done
         
        		DM-32698

        Fix argument type in GraphBuilder skipExistingIn fallback logic

        		 0.5 Jim Bosch Done
         
        		DM-31710

        Optimize crosstalkSources lookup in IsrTask

        		 1 Jim Bosch Done
         
        		DM-31700

        Butler parquet formatter fails to load table with no indexes

        		 1 Dan Taranu Done
         
        		DM-31685

        Update dev guide tiger shared stack information

        		 1 Lee Kelvin Done
         
        		DM-31783

        Register DECam N419 and N540 filters

        		 2 Lee Kelvin Done
         
        		DM-32663

        Avoid isinstance check in pex.config.Field.__get__

        		 0.5 Nate Lust Done
         
        		DM-31769

        Add user control over dataset constraints in QG generation queries

        		 16 Nate Lust Done
         
        		DM-32065

        Skip gen2 test_read_CuratedCalibs in pipe_tasks

        		 1 Lee Kelvin Done
         
        		DM-31035

        Reading an exposure may fail with certain metadata keys set

        		 2 Eli Rykoff Done
         
        		DM-30708

        Investigate CalibCombineByFilterTask failures

        		 Unassigned Won't Fix
         
        		DM-32588

        Clean up and test afw.image deepcopy and pickle implementations

        		 Unassigned Invalid
         
        		DM-32699

        Backwards compatibility for moving PSF quality stats from makeWarp to calibrate

        		 Unassigned Invalid
         
        		DM-16042

        Make CompareWarp the Default Coaddition Algorithm

        		 Yusra AlSayyad Invalid
         
        		DM-21794

        Add origin component to all autoincrement data IDs in Registry

        		 Jim Bosch Invalid
         
        		DM-12651

        Implement merge from data repository exports

        		 0 Unassigned Invalid

          Activity

          There are no comments yet on this issue.

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Watchers:
            Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.