Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16576

Continue Butler Gen3 coding

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      S19 Arch Butler Gen3
    • Story Points:
      60
    • WBS:
      02C.02.02.03
    • Team:
      Architecture
    • Cycle:
      Spring 2019

      Description

      Meet Middleware milestones. This includes:

      • Supporting daf_butler enhances, particularly in terms of datastore functionality.
      • Supporting obs_lsst development.
      • Updating astro_metadata_translator.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
         
        DM-19877

        Replace PropertySet.get with getScalar or getArray

        1 Tim Jenness Done
         
        DM-19873

        Implement PropertySet.getitem and return get()

        2 Tim Jenness Done
         
        DM-19506

        Adjust defect FITS files to be compatible with DS9

        1 Tim Jenness Done
         
        DM-19485

        Fix length of instrument name in gen 3 butler schema

        0.5 Tim Jenness Done
         
        DM-18864

        Update afw to support undefined values in FITS headers

        1 Tim Jenness Done
         
        DM-19916

        Investigate URI inconsistencies in daf_butler LocationFactory

        2 Tim Jenness Done
         
        DM-19752

        monowl string parse in gen2 butler ingest of ts8 data

        0.5 Tim Jenness Done
         
        DM-19583

        Investigate butler gen 3 configuration system

        3 Tim Jenness Done
         
        DM-19409

        Add getCenter to Box2I

        1 Tim Jenness Done
         
        DM-19383

        Fix BOT translator when DARKTIME is present

        0.5 Tim Jenness Done
         
        DM-19373

        Add defect list support to butler gen 3

        4 Tim Jenness Done
         
        DM-18708

        Investigate ingest problems with BOT data

        1 Tim Jenness Done
         
        DM-19627

        Add text file serialization to meas_algorithms Defects class

        2 Tim Jenness Done
         
        DM-19618

        Allow gen3.py to be called with an external butler configuration

        2 Tim Jenness Done
         
        DM-19682

        Fix DAYOBS calculation and allow for gen2 header correction for ingest

        1 Tim Jenness Done
         
        DM-19671

        setConfigRoot sometimes needs to not update the root

        1 Tim Jenness Done
         
        DM-19331

        Fix detector name for HSC

        2 Tim Jenness Done
         
        DM-19188

        Add system for reading header corrections from external files

        4 Tim Jenness Done
         
        DM-18748

        Missing ExpTime in auxTel teststand images cause ingestImages.py to abort

        3 Tim Jenness Done
         
        DM-17903

        imsim ci_lsst tests fail with bad key ccd

        1 Tim Jenness Done
         
        DM-17866

        Support BOT data gen 2 ingest once data exists

        3 Tim Jenness Done
         
        DM-18577

        Fix PosixDataStore naming

        4 Tim Jenness Done
         
        DM-18576

        Issue warnings if translator methods are shadowed

        1 Tim Jenness Done
         
        DM-18438

        Specify string lengths in PosixDatastoreRecords table

        2 Tim Jenness Done
         
        DM-18181

        Provide tool to validate datastore template configurations

        8 Tim Jenness Done
         
        DM-17978

        Visit id not being cast as int

        0.5 Tim Jenness Done
         
        DM-17837

        Fix incorrect detector serial in TS8 RTM-004

        0.5 Tim Jenness Done
         
        DM-17794

        Modify exposure_id calculations in obs_lsst

        1 Tim Jenness Done
         
        DM-17669

        Reorganize how TS8 data are handled inside obs_lsst

        4 Tim Jenness Done
         
        DM-17547

        Integrate obs_lsst camera instructions into sphinx docs

        1 Tim Jenness Done
         
        DM-17375

        Change license of astro_metadata_translator

        2 Tim Jenness Done
         
        DM-17369

        Enable sphinx documentation in obs_lsst

        2 Tim Jenness Done
         
        DM-17437

        Add UC Davis camera support to obs_lsst

        2 Tim Jenness Done
         
        DM-17530

        obs_lsst no longer supports TS3 test data

        6 Tim Jenness Done
         
        DM-17228

        Use astro_metadata_translator in obs_lsst gen2 translator

        4 Tim Jenness Done
         
        DM-17292

        Add obs_lsst camera package unit tests

        6 Tim Jenness Done
         
        DM-17095

        Tests are very slow in obs_lsst

        0.5 Tim Jenness Done
         
        DM-17025

        Improve filename template mechanisms in PosixDatastore and Butler

        7 Tim Jenness Done
         
        DM-16554

        Write metadata translators for obs_lsst cameras

        10 Tim Jenness Done
         
        DM-16292

        Calibration products need more metadata

        4 Tim Jenness Done
         
        DM-9873

        PropertySet does not support values of None

        3 Tim Jenness Done
         
        DM-14459

        Add check to (Posix)Datastore that prevents silent overwrite

        1 Tim Jenness Done
         
        DM-15374

        Per dataset allow/deny lists for datastores

        3 Tim Jenness Done

          Activity

          Hide
          tjenness Tim Jenness added a comment -

          Lots of progress was made on daf_butler datastores, obs_lsst, and astro_metadata_translator. This used almost twice the amount of allocated SPs for the epic, although a lot of LOE time was spent on this.

          Show
          tjenness Tim Jenness added a comment - Lots of progress was made on daf_butler datastores, obs_lsst, and astro_metadata_translator. This used almost twice the amount of allocated SPs for the epic, although a lot of LOE time was spent on this.
          Hide
          ktl Kian-Tat Lim added a comment -

          That was a lot of work! Thanks for all your effort on this.

          Show
          ktl Kian-Tat Lim added a comment - That was a lot of work! Thanks for all your effort on this.

            People

            • Assignee:
              tjenness Tim Jenness
              Reporter:
              ktl Kian-Tat Lim
              Reviewers:
              Kian-Tat Lim
              Watchers:
              Kian-Tat Lim, Tim Jenness
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel