Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11783

Replace meas_mosaic with Jointcal

    Details

    • Epic Name:
      s18-ap-jointcal-mosaic
    • Story Points:
      52
    • WBS:
      02C.03.07
    • Team:
      Alert Production
    • Cycle:
      Spring 2018

      Description

      This epic is for the work required to determine if we are ready to replace meas_mosaic with jointcal. The test plan is described in DM-10730, and will require that jointcal's output be able to be run through validate_drp.

      The length of this epic depends in part on whether changes to jointcal-to improve its models, etc.-are necessary after the first comparison is done.

        Attachments

          Issue Links

          Stories in Epic (Custom Issue Matrix)

          Key Summary Story Points Assignee Status
           
          DM-16235

          Jointcal PhotoCalib returns negative calibrations

          8 John Parejko Done
           
          DM-13669

          Track and log measurement/reference outliers separately

          2 John Parejko Done
           
          DM-13575

          fix minor bug in photometry ipynb

          0.5 John Parejko Done
           
          DM-13674

          Save full astrometry model as astShim.FrameDict

          6 John Parejko Done
           
          DM-13671

          Add option to skip cholmod.update step

          4 John Parejko Done
           
          DM-13670

          Check residual vs. errors for photometry

          6 John Parejko Done
           
          DM-13272

          Confirm jointcal's astrometry output

          16 John Parejko Done
           
          DM-13269

          Improve jointcal debugging output

          8 John Parejko Done
           
          DM-13187

          jointcal selected_*_refStars is not correctly computed

          2 John Parejko Done
           
          DM-12918

          investigate and merge u/fix_outliers

          4 John Parejko Done
           
          DM-12906

          Generate summary tables for jointcal/meas_mosaic comparison

          4 John Parejko Done
           
          DM-14612

          Fix race condition in new jointcal matrix dump test

          0.5 John Parejko Done
           
          DM-14574

          implement magnitude-based photometric model

          6 John Parejko Done
           
          DM-14510

          Implement line search

          8 John Parejko Done
           
          DM-15713

          Run jointcal comparison with higher order polynomial

          1 John Parejko Done
           
          DM-15695

          Measure jointcal acceptance statistics

          2 John Parejko Done
           
          DM-15617

          Process HSC PDR1 data with Jointcal as of 2018-08-31.

          2 John Parejko Done
           
          DM-15431

          Add Jacobian BoundedField accessor to SkyWcs

          6 John Parejko Done
           
          DM-15421

          Change to use constrained models by default

          1 John Parejko Done
           
          DM-15254

          Option to normalize warps by Jointcal's PhotoCalib

          8 John Parejko Done
           
          DM-15241

          log error when final chi2 is large

          0.5 John Parejko Done
           
          DM-15011

          implement separate Visit and Chip fitting for photometry

          2 John Parejko Done
           
          DM-15751

          Configure ability to build coadds with either Jointcal or meas_mosaic

          2 John Parejko Done
           
          DM-14786

          Implement ConstrainedMagnitude model

          10 John Parejko Done
           
          DM-12310

          Figure out why most HSC datasets start with photometry chi2==NaN

          4 John Parejko Done
           
          DM-12254

          Switch jointcal to default to IndexedRefObj instead of a.net

          1 John Parejko Done
           
          DM-11787

          First comparison of jointcal/meas_mosaic in validate_drp

          4 John Parejko Done
           
          DM-11785

          Run jointcal on acceptance data

          8 John Parejko Done
           
          DM-11784

          Find all the necessary data for jointcal/meas_mosaic comparison

          1 John Parejko Done
           
          DM-12029

          recompress jointcal's testdata zeroed images with fpack

          2 John Parejko Done
           
          DM-11931

          Fix jointcal exit status and doRaise handling

          2 John Parejko Done
           
          DM-11571

          Complete and test use of jointcal results in validate_drp

          4 John Parejko Done
           
          DM-14767

          fix photometric error calculation

          4 John Parejko Invalid
           
          DM-14020

          Allow inclusion of astrometric jacobian in photoCalib

          John Parejko Invalid

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            This was started in September.

            Show
            Parejkoj John Parejko added a comment - This was started in September.
            Hide
            swinbank John Swinbank added a comment -

            As of DM-15751, it is possible to run complete (as in, through coadd construction & processing) pipelines based on Jointcal. I regard the success criteria for this epic as having been met.

            Per discussion with the team in Princeton, Jointcal will not be made the default for HSC processing until the completion of DM-16596. That ticket is a DRP deliverable, though, so doesn't block the completion of this epic.

            The HSC team have also requested that meas_mosaic not be dropped from lsst_distrib entirely until after the next HSC data release, currently scheduled for July 2019. They will be responsible for its continued maintenance to that date.

            Show
            swinbank John Swinbank added a comment - As of DM-15751 , it is possible to run complete (as in, through coadd construction & processing) pipelines based on Jointcal. I regard the success criteria for this epic as having been met. Per discussion with the team in Princeton, Jointcal will not be made the default for HSC processing until the completion of DM-16596 . That ticket is a DRP deliverable, though, so doesn't block the completion of this epic. The HSC team have also requested that meas_mosaic not be dropped from lsst_distrib entirely until after the next HSC data release, currently scheduled for July 2019. They will be responsible for its continued maintenance to that date.
            Hide
            Parejkoj John Parejko added a comment -

            Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

            Show
            Parejkoj John Parejko added a comment - Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests
            Hide
            Parejkoj John Parejko added a comment -

            Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

            Show
            Parejkoj John Parejko added a comment - Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Michael Wood-Vasey, Pierre Astier, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel