Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23740

Emergent work for DRP in S20B

    Details

    • Type: Epic
    • Status: In Progress
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Epic Name:
      s20b-drp-emergent
    • Story Points:
      144
    • WBS:
      02C.04.01
    • Team:
      Data Release Production
    • Cycle:
      Spring 2020

      Description

      This Epic captures all work maintaining the Science Pipelines including the SDM-ification work by Yusra and work on ISR and calibration products by Chris.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
        DM-21279

        Unify yaml and FITS camera generation

        Christopher Waters To Do
        DM-21275

        Add agnostic dataBBox accessor

        Christopher Waters To Do
        DM-21277

        Add useful detector AssemblyState

        Christopher Waters To Do
        DM-22351

        Add config to turn off warp-making in CoaddDriver

        1 Yusra AlSayyad To Do
        DM-21274

        Add to/fromDict methods for afw cameraGeom objects

        Christopher Waters To Do
        DM-23200

        Add matchedVisitMetric configs to obs_subaru

        1 Unassigned To Do
        DM-21956

        Fetch functors from registry instead of single module

        4 Yusra AlSayyad To Do
        DM-21976

        Decide on behavior of ParquetTable if one requested column does not exist

        Unassigned To Do
        DM-21954

        Check on StarGalaxyLabeller

        1 Tim Morton To Do
        DM-20272

        Suppress irrelevant warning when reading metadata objects

        Unassigned To Do
        DM-20776

        Update afw docs to show all SpanSet initializations

        1 Unassigned To Do
        DM-20380

        getCcdImage() in cameraGeom utils doesn't return a tuple anymore

        1 Yusra AlSayyad To Do
        DM-24349

        Write a Source.yaml for obs_lsst/imSim

        Yusra AlSayyad To Do
        DM-24162

        Look into setting NUMEXPR_MAX_THREADS

        2 Unassigned To Do
        DM-24581

        Fix inconsistent BatchCmdLineTask walltimes

        Unassigned To Do
        DM-20258

        Warping an ExposureU gives all zeroes back

        Unassigned To Do
        DM-13483

        Add sphgom.Pixelization for HEALPix

        10 Unassigned To Do
        DM-12179

        Identify changes in HSC processCcd matching between w_2017_39 and w_2017_38

        Unassigned To Do
        DM-12058

        Investigate PSF problems found in MEDS generation

        Bob Armstrong To Do
        DM-12057

        Debug MEDS file creation and ngmix comparisons

        2 Bob Armstrong To Do
        DM-11446

        Refactor Coadd Tasks: Remodel varietal coadd datatypes as data IDs

        6 Yusra AlSayyad To Do
        DM-11331

        Benchmark new statistics code for coaddition

        2 Unassigned To Do
        DM-11275

        Don't import mpi when using a different batch system

        Unassigned To Do
        DM-11276

        getPlaneBitMask and the like should return MaskPixel in Python, not int

        1 Unassigned To Do
        DM-11098

        Audit existing tasks and update them to comply with RFC-352

        6 Yusra AlSayyad To Do
        DM-11003

        Segfault when attempting to construct an image

        1 Unassigned To Do
        DM-10844

        Remove old afw::math::Statistics

        8 Unassigned To Do
        DM-10792

        Write test for AssembleCoaddTask.setBrightObjectMasks

        Unassigned To Do
        DM-10313

        Export INT_MIN to python and use it to initialise hdu as appropriate

        1 Unassigned To Do
        DM-10293

        Bad exposure scales doesn't tell which exposures are bad

        1 Unassigned To Do
        DM-13541

        More "RuntimeWarning: invalid value encountered in"

        Unassigned To Do
        DM-12948

        Convolve masks in modelPsfMatch with nearest neighbor

        2 Yusra AlSayyad To Do
        DM-12410

        Follow up on PSFEx masking problems

        1 Unassigned To Do
        DM-12274

        Add WCS to Coadd nImage

        1 Yusra AlSayyad To Do
        DM-11522

        Set NO_DATA bit in coadd mask iff 0 epochs contribute to a pixel in assembleCoadd

        2 Yusra AlSayyad To Do
        DM-11428

        Rename phuInfo and infoList in obs_packages and base class

        1 Unassigned To Do
        DM-11419

        afwTable rows define offsets as int32, causing problems with very large (>2Gb) rows

        Unassigned To Do
        DM-12032

        Debug MEDS file creation and ngmix comparisons

        4 Jim Bosch To Do
        DM-11964

        afw tables silently truncates long variable-length string fields when saved as FITS

        Unassigned To Do
        DM-11924

        Create tests for obs_ctio0m9

        4 Unassigned To Do
        DM-10380

        Allow numpy bool scalars as input when setting Flag table fields

        2 Unassigned To Do
        DM-11515

        tests.getTempFilePath should remove any existing file

        1 Unassigned To Do
        DM-11125

        lsstDebug behaviour does not match documentation

        1 Unassigned To Do
        DM-10843

        Implement new afw::math::Statistics

        8 Unassigned To Do
        DM-10147

        Document (and/or fix) how to test deep equality of ndarrays

        1 Jim Bosch To Do
        DM-8692

        ConvolvedFluxPlugin errors are underestimated

        6 Unassigned To Do
        DM-8580

        Diagnose upstream pybind11 problem with some vector overloads

        2 Unassigned To Do
        DM-8495

        Update DMTN-015 following Zackay discussion

        2 Jim Bosch To Do
        DM-8146

        Remove deprecated FFTW API usage from meas_base

        Unassigned To Do
        DM-6446

        Remove boost dependencies where possible

        1 Unassigned To Do
        DM-8438

        Add PSF-Matched Co-adds to ci_hsc

        3 Yusra AlSayyad To Do
        DM-8244

        Spurious warnings in base_CircularApertureFlux forced mode

        1 Unassigned To Do
        DM-7715

        Make algorithm metadata writing FITS-compliant

        8 Jim Bosch To Do
        DM-6722

        ingestCalibs doesn't respect calib repo path

        Unassigned To Do
        DM-5845

        ci_hsc fails with "too many open files"

        2 John Swinbank To Do
        DM-6278

        Investigate proper precision for afw::image::Image pixel transforms

        1 Unassigned To Do
        DM-6099

        Improve afw.table Astropy view support

        4 Unassigned To Do
        DM-6056

        Remove PTR and CONST_PTR macros

        Unassigned To Do
        DM-5347

        Add tests for recent improvements to CModel

        3 Jim Bosch To Do
        DM-9610

        Refactor Coadd Tasks: Add Pipe Task that makes all Coadds for DRP

        6 Yusra AlSayyad To Do
        DM-9159

        Catch-and-ignore should not apply to any exception

        4 Unassigned To Do
        DM-10012

        PropertySet.combine() should take const& PropertySet not std::shared_ptr<const PropertySet>

        Unassigned To Do
        DM-9550

        Check if memory is being released at the end of assembleSubregion in assembleCoadd.py

        Unassigned To Do
        DM-9116

        Failure to persist footprints with SOURCE_IO_NO_HEAVY_FOOTPRINTS

        2 Unassigned To Do
        DM-9814

        Investigate pybind11 base class data member exposure from derived

        1 Unassigned To Do
        DM-9637

        Port SafeClipAssembleCoadd to new Coadd Task

        8 Yusra AlSayyad To Do
        DM-6600

        Clean up naming of multiband tasks and scripts

        1 Unassigned To Do
        DM-15100

        Add sphgeom::ConvexPolygon::intersect

        6 Unassigned To Do
        DM-14619

        Enable SourceDetection to ignore pixels with mask bit set

        Unassigned To Do
        DM-15305

        Improve debug functionality to the brighter fatter kernel generation task

        2 Unassigned To Do
        DM-15030

        Refactor bright object mask setting into a subtask

        2 Unassigned To Do
        DM-15363

        Mask operations don't work with bit masks returned by Mask's methods

        Unassigned To Do
        DM-14927

        Remove the term “ghoulie” from LDM-151

        1 John Swinbank To Do
        DM-14041

        skymap.baseSkyMap.findTract assumes {{tractinfo}} objects are sortable

        0.5 Unassigned To Do
        DM-13744

        Run clang-tidy checks and cleanup on afw

        Unassigned To Do
        DM-13730

        Remove requirement for reference ccd from fgcmBuildStars

        1 Yusra AlSayyad To Do
        DM-17471

        Save intermediate data product with pre-interpolation pixel values

        6 Lauren MacArthur To Do
        DM-17438

        LoadReferenceObjectsTask does not forward epoch

        Unassigned To Do
        DM-16802

        ISR order-of-operations not uniquely defined

        Christopher Waters To Do
        DM-16178

        The NOT_DEBLENDED mask plane is not getting set

        Unassigned To Do
        DM-17731

        the dimensions of PSF model skyrocket

        Yusra AlSayyad To Do
        DM-17019

        Setting spatial order to 0 in psfex results in a Segmentation fault

        Unassigned To Do
        DM-16313

        DPDD Object Table: Compute KronRad50, KronRad90

        Unassigned To Do
        DM-16042

        Make CompareWarp the Default Coaddition Algorithm

        4 Yusra AlSayyad To Do
        DM-15810

        Upgrade obs_subaru to numpydoc status

        Sophie Reed To Do
        DM-17351

        Investigate why DirectMatchTask checks its config object

        Unassigned To Do
        DM-15648

        Specify which BF kernel to use with configuration file

        4 Unassigned To Do
        DM-15408

        measureCrosstalk.py is not thread safe

        Unassigned To Do
        DM-15384

        SourceCatalog repr throws error if catalog is empty

        2 Unassigned To Do
        DM-15631

        Add per-amplifier brighter-fatter correction support

        Unassigned To Do
        DM-15761

        Add useful data to testdata_subaru

        Christopher Waters To Do
        DM-6822

        Add meas_extensions_ngmix to lsst_distrib

        2 John Swinbank To Do
        DM-3156

        Support computing statistics on a scalar afw table field

        Unassigned To Do
        DM-4668

        audit obs_subaru defaults and move them to lower-level code

        4 Lauren MacArthur To Do
        DM-4663

        Documentation for forcedPhotCcd.py and forcedPhotCoadd.py

        Unassigned To Do
        DM-4660

        Documentation for makeCoaddTempExp

        Unassigned To Do
        DM-4661

        Documentation for assembleCoadd

        Unassigned To Do
        DM-3417

        Fix overestimation of aperture correction error

        8 Jim Bosch To Do
        DM-4743

        Make deblender more robust against weird PSF dimensions

        2 Bob Armstrong To Do
        DM-4740

        Audit and document obs_subaru scripts

        5 Unassigned To Do
        DM-644

        update overview docs to clarify roles of meas_multifit and shapelet packages

        2 Jim Bosch To Do
        DM-20381

        Make initial ap_verify_hsc_pdr1

        Yusra AlSayyad In Progress
        DM-24638

        Convert TransformSourceTableTask and friends to Gen3

        6 Yusra AlSayyad In Progress
        DM-23483

        Write ForcedPhotDiffimTask

        2 Clare Saunders In Progress
        DM-24441

        Explore noise covariance after coaddition

        4 Morgan Schmitz In Progress
        DM-11572

        Implement overloads for new afw::math::Statistics

        4 Unassigned In Progress
        DM-10200

        Change default for invert in Box2I constructors, and then remove invert entirely.

        4 Sophie Reed In Progress
        DM-5795

        Add Python properties for getters and setters in afw::geom and shapelet

        1 Jim Bosch In Progress
        DM-13983

        Use -pedantic to enforce C++ standards compliance

        0.5 Jim Bosch In Progress
        DM-14992

        Make sure proxy iterators are not declared random-access

        2 Jim Bosch In Progress
         
        DM-22138

        validateDrp arguments (including brightSnr) not passed/parsed correctly

        2 Dan Taranu Done
         
        DM-22256

        Remove SkipUnless PyArrow when PyArrow works with macos stack

        1 Yusra AlSayyad Done
         
        DM-23380

        Remove lsst.fgcmcal.FgcmFitCycleConfig.pixelScale

        0.25 Eli Rykoff Done
         
        DM-24908

        Change keys for imsim obs_lsst back to visit from expId

        2 Eli Rykoff Done
         
        DM-24864

        Update processCcdWithFakes to add the sky_source column.

        2 Sophie Reed Done
         
        DM-24857

        Add processCcdWithFakesDriver to bin.src

        1 Sophie Reed Done
         
        DM-24707

        Publish Initial dmtn-148.lsst.io

        12 Christopher Waters Done
         
        DM-24517

        Reading of raws from existing imsim & phosim repos broken by DM-23584

        4 Lauren MacArthur Done
         
        DM-24495

        Convert config overrides to use file in several obs packages

        2 Jim Bosch Done
         
        DM-24404

        Squash astropy ecsv read warnings

        1 Jim Bosch Done
         
        DM-24346

        Don't add fakes outside of the valid polygon area for the ccd

        2 Sophie Reed Done
         
        DM-23971

        Undo TE1 "regression"

        3 Dan Taranu Done
         
        DM-25112

        Update ci_lsst for bitrot

        3 Lauren MacArthur Done
         
        DM-24379

        WriteSourceTable: add option to get local calib columns from calexp (or external calib)

        2 Yusra AlSayyad Done
         
        DM-24334

        Create quickLookExp exposure type to support commissioning

        4 Merlin Fisher-Levine Done
         
        DM-24285

        fitsExposureFormatter fails to read "Exposure" entries correctly

        1 Christopher Waters Done
         
        DM-24062

        Write Source Table transform tasks and Source.yaml for obs_subaru

        7 Yusra AlSayyad Done
         
        DM-25149

        Fix w18 ISR regression

        1 Christopher Waters Done
         
        DM-24466

        Can't run processCcd on multiple CCDs

        2 Andrés Alejandro Plazas Malagón Done
         
        DM-24424

        Fix bug in buildTractImage

        2 Sophie Reed Done
         
        DM-24553

        Make a driver script for processCcdWithFakes

        4 Sophie Reed Done
         
        DM-6981

        Add column setters for Flag types in catalogs

        1 Jim Bosch Done
         
        DM-25116

        Investigate regression in matched visit metrics in w_2020_19 RC2

        1 Yusra AlSayyad Done
         
        DM-25187

        daf_butler registry EllipsisType breaks pipelines.lsst.io doc build

        1 Jim Bosch Done
         
        DM-24585

        Update policies in obs_lsst to work with visualizeVisit.py

        6 Lauren MacArthur Done
        DM-23252

        Profile and mitigate the runtime of jointcal

        8 Nate Lust In Review
        DM-21697

        SpanSet.fromShape() fails for circles with radii > 46340 pixels

        Nate Lust In Review
        DM-24669

        Update RC2 dashboards in squash now that we're ingesting 3 tracts.

        Angelo Fausti In Review
        DM-23983

        Cannot apply crosstalk in Gen 3 DECam processing

        Christopher Waters In Review
        DM-9647

        Unit test in meas deblender succeeds but throws traceback

        Sophie Reed In Review
        DM-22187

        Convert DPDD change requests to tickets and annotate Object.yaml

        1 Yusra AlSayyad Reviewed
        DM-15548

        Add instruction to push to branch in developer.lsst.io/work/flow.html#merging

        1 Yusra AlSayyad Reviewed
         
        DM-15817

        convert obs_sdss to numpydoc status

        Unassigned Won't Fix

          Activity

            People

            • Assignee:
              yusra Yusra AlSayyad
              Reporter:
              yusra Yusra AlSayyad
              Watchers:
              Yusra AlSayyad
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Summary Panel