Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26803

AP pipeline infrastructure & Gen 3 updates in F20B

    Details

    • Type: Epic
    • Status: In Progress
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      f20b-ap-infrastructure
    • Story Points:
      115.63
    • WBS:
      02C.03.05
    • Team:
      Alert Production
    • Cycle:
      Fall 2020

      Description

      Infrastructure work to support Alert Production. This includes Gen 3 middleware to run ap_verify at scale, conversion of documentation to numpydoc, and reworking afw::filter.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
        DM-25161

        Develop proposal for PSF representation in astropy CCDData

        12 Unassigned To Do
        DM-25160

        Record use of CCDData alert stamps in DPDD and DMTN-093

        4 Eric Bellm To Do
        DM-24326

        Add a PSF object to the Astropy CCData representation

        8 Unassigned To Do
        DM-23032

        CreateSDM functor for bitpacking mutiple flag columns

        4 Chris Morrison To Do
        DM-22743

        Create runQuantum unittest for DiaPipelineTask

        2 Chris Morrison To Do
        DM-23197

        Update functors in obs_hsc object.yaml

        4 Chris Morrison To Do
        DM-21382

        Separate GenericMap's visitor API from its implementation

        6 Krzysztof Findeisen To Do
        DM-21267

        Enforce consistency between the expected filter columns in ap_association and dax_ppdb.

        8 Chris Morrison To Do
        DM-21216

        GenericMap converts C-strings to bool

        2 Krzysztof Findeisen To Do
        DM-20970

        timeMethod gives platform-dependent results

        3 Unassigned To Do
        DM-26028

        Generalize ap_verify to use other executors

        Krzysztof Findeisen To Do
        DM-16861

        Convert afw.image to numpydoc

        6 Unassigned To Do
        DM-16078

        convert coadd_utils to numpydoc

        John Swinbank To Do
        DM-17070

        Clean up lsst.afw.multiband

        2 Unassigned To Do
        DM-16773

        ap_verify_hits2015 hard to manage with lsstsw

        Unassigned To Do
        DM-16205

        Document ap_pipe using new Task doc framework

        4 Unassigned To Do
        DM-17458

        Complete docstrings in ip_diffim

        2 Unassigned To Do
        DM-17989

        Compute RA/dec covariances on DIAObjects

        5 Chris Morrison To Do
        DM-14136

        convert ip_isr documentation to numpydoc

        6 Unassigned To Do
        DM-27797

        Write up how-to guide for PostGres on lsst-devl

        1 Meredith Rawls To Do
        DM-23862

        Split data ID handling off from butlerTests.makeTestRepo

        4 Krzysztof Findeisen To Do
        DM-26231

        Remove defaultFilter in LoadReferenceObjectsConfig

        2 John Parejko To Do
        DM-26224

        Deprecate defaultFilter in LoadReferenceObjectsConfig

        1 John Parejko To Do
        DM-27173

        Add doc field to obs.base.FilterDefinition

        2 John Parejko To Do
        DM-23366

        Rework functor tests to test specific input/output.

        4 Chris Morrison To Do
        DM-27175

        Add InstrumentLabel to new exposures

        3 Krzysztof Findeisen To Do
        DM-27412

        Rework daf.butler.tests.makeTestCollection

        2 Krzysztof Findeisen To Do
        DM-23156

        Add ways to test a PipelineTask's init inputs/outputs

        4 Krzysztof Findeisen To Do
        DM-27178

        Standardize aliases on Gen 2 Exposure get

        4 Krzysztof Findeisen To Do
        DM-26140

        Centralize Gen 3 pipeline configuration info for ap_verify datasets

        6 Krzysztof Findeisen To Do
        DM-21268

        GenericMap can't distinguish int/float sizes in Python

        6 Krzysztof Findeisen To Do
        DM-23330

        Doxygen warnings in pipe_base

        2 Krzysztof Findeisen To Do
        DM-26108

        Add lightweight test for connections

        4 Krzysztof Findeisen To Do
        DM-17071

        Remove afw.math.mathLib

        4 Krzysztof Findeisen To Do
        DM-24402

        Handle "curated" datasets more generically in Gen 2 ap_verify

        4 Krzysztof Findeisen To Do
        DM-18714

        Complete ap_pipe abort causes problems at metrics stage

        4 Krzysztof Findeisen To Do
        DM-16824

        Convert meas_algorithms to numpydoc format

        Sam Quist In Progress
        DM-18000

        Implement SDM system inside ap_association

        8 Chris Morrison In Progress
        DM-27169

        Use FilterLabel in Exposure/ExposureInfo

        8 Krzysztof Findeisen In Review
         
        DM-22814

        Remove afw APIs deprecated in DM-17566

        6 John Parejko Done
         
        DM-23344

        Remove Synpipe from lsst_distrib

        1 John Swinbank Done
         
        DM-22663

        Reimplement make_apdb.py for Gen 3

        6 Krzysztof Findeisen Done
         
        DM-21333

        Implement afw.image.Filter replacement(s)

        4 Krzysztof Findeisen Done
         
        DM-20994

        Write how to make gen3 obs package guide

        4 John Parejko Done
         
        DM-27450

        Review past EPO Zooniverse API

        2 John Parejko Done
         
        DM-27099

        Re-bridge ap_verify and ctrl_mpexec

        4 Krzysztof Findeisen Done
         
        DM-26662

        ap_verify import slow in Gen 3

        6 Krzysztof Findeisen Done
         
        DM-26615

        Remove afw.filter dependence from DCR model

        2 Ian Sullivan Done
         
        DM-26614

        Create RFC for new filter design

        2 Krzysztof Findeisen Done
         
        DM-26375

        Allow databases other than SQLite when running ap_verify

        6 Krzysztof Findeisen Done
         
        DM-26259

        Create design sketch for new filter system

        6 Krzysztof Findeisen Done
         
        DM-16857

        Convert afw.detection to numpydoc

        1 Kenneth Herner Done
         
        DM-24263

        Rework API for providing data IDs to test butlers

        4 Krzysztof Findeisen Invalid

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              sullivan Ian Sullivan
              Reporter:
              swinbank John Swinbank
              Watchers:
              John Swinbank
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Summary Panel