Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11783

Replace meas_mosaic with Jointcal

    XMLWordPrintable

    Details

    • Epic Name:
      s18-ap-jointcal-mosaic
    • Story Points:
      52
    • WBS:
      02C.03.07
    • Team:
      Alert Production
    • Cycle:
      Spring 2018

      Description

      This epic is for the work required to determine if we are ready to replace meas_mosaic with jointcal. The test plan is described in DM-10730, and will require that jointcal's output be able to be run through validate_drp.

      The length of this epic depends in part on whether changes to jointcal-to improve its models, etc.-are necessary after the first comparison is done.

        Attachments

          Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. DM-12796 jointcal butler templates don't include filter

          • Invalid

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11571 Complete and test use of jointcal results in validate_drp

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11800 Update meas_mosaic to new transform-based API for afw::cameraGeom

          • Done

          Technical task - Created by JIRA Agile - do not edit or delete. Issue type for a technical task. DM-10730 Near-term jointcal acceptance: write test description

          • Done
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. DM-15241 log error when final chi2 is large

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-15221 parallelize validate_drp ingest

          • Won't Fix
          Wiki Page

          Wiki Page Loading...

          Stories in Epic (Custom Issue Matrix)

          Key Summary Story Points Assignee Status
           
          		DM-15713

          Run jointcal comparison with higher order polynomial

          		 1 John Parejko Done
           
          		DM-15695

          Measure jointcal acceptance statistics

          		 2 John Parejko Done
           
          		DM-16235

          Jointcal PhotoCalib returns negative calibrations

          		 8 John Parejko Done
           
          		DM-15751

          Configure ability to build coadds with either Jointcal or meas_mosaic

          		 2 John Parejko Done
           
          		DM-15617

          Process HSC PDR1 data with Jointcal as of 2018-08-31.

          		 2 John Parejko Done
           
          		DM-14612

          Fix race condition in new jointcal matrix dump test

          		 0.5 John Parejko Done
           
          		DM-14574

          implement magnitude-based photometric model

          		 6 John Parejko Done
           
          		DM-11787

          First comparison of jointcal/meas_mosaic in validate_drp

          		 4 John Parejko Done
           
          		DM-11785

          Run jointcal on acceptance data

          		 8 John Parejko Done
           
          		DM-11784

          Find all the necessary data for jointcal/meas_mosaic comparison

          		 1 John Parejko Done
           
          		DM-12029

          recompress jointcal's testdata zeroed images with fpack

          		 2 John Parejko Done
           
          		DM-11931

          Fix jointcal exit status and doRaise handling

          		 2 John Parejko Done
           
          		DM-15254

          Option to normalize warps by Jointcal's PhotoCalib

          		 8 John Parejko Done
           
          		DM-15241

          log error when final chi2 is large

          		 0.5 John Parejko Done
           
          		DM-15431

          Add Jacobian BoundedField accessor to SkyWcs

          		 6 John Parejko Done
           
          		DM-15421

          Change to use constrained models by default

          		 1 John Parejko Done
           
          		DM-14786

          Implement ConstrainedMagnitude model

          		 10 John Parejko Done
           
          		DM-11571

          Complete and test use of jointcal results in validate_drp

          		 4 John Parejko Done
           
          		DM-15011

          implement separate Visit and Chip fitting for photometry

          		 2 John Parejko Done
           
          		DM-12918

          investigate and merge u/fix_outliers

          		4 John Parejko Done
           
          		DM-12906

          Generate summary tables for jointcal/meas_mosaic comparison

          		 4 John Parejko Done
           
          		DM-13272

          Confirm jointcal's astrometry output

          		 16 John Parejko Done
           
          		DM-13269

          Improve jointcal debugging output

          		 8 John Parejko Done
           
          		DM-12310

          Figure out why most HSC datasets start with photometry chi2==NaN

          		 4 John Parejko Done
           
          		DM-12254

          Switch jointcal to default to IndexedRefObj instead of a.net

          		 1 John Parejko Done
           
          		DM-14510

          Implement line search

          		 8 John Parejko Done
           
          		DM-13674

          Save full astrometry model as astShim.FrameDict

          		 6 John Parejko Done
           
          		DM-13671

          Add option to skip cholmod.update step

          		 4 John Parejko Done
           
          		DM-13670

          Check residual vs. errors for photometry

          		 6 John Parejko Done
           
          		DM-13669

          Track and log measurement/reference outliers separately

          		 2 John Parejko Done
           
          		DM-13575

          fix minor bug in photometry ipynb

          		 0.5 John Parejko Done
           
          		DM-13187

          jointcal selected_*_refStars is not correctly computed

          		 2 John Parejko Done
           
          		DM-14767

          fix photometric error calculation

          		 4 John Parejko Invalid
           
          		DM-14020

          Allow inclusion of astrometric jacobian in photoCalib

          		 John Parejko Invalid

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            This was started in September.

            Show
            Parejkoj John Parejko added a comment - This was started in September.
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            As of DM-15751, it is possible to run complete (as in, through coadd construction & processing) pipelines based on Jointcal. I regard the success criteria for this epic as having been met.

            Per discussion with the team in Princeton, Jointcal will not be made the default for HSC processing until the completion of DM-16596. That ticket is a DRP deliverable, though, so doesn't block the completion of this epic.

            The HSC team have also requested that meas_mosaic not be dropped from lsst_distrib entirely until after the next HSC data release, currently scheduled for July 2019. They will be responsible for its continued maintenance to that date.

            Show
            swinbank John Swinbank added a comment - As of DM-15751 , it is possible to run complete (as in, through coadd construction & processing) pipelines based on Jointcal. I regard the success criteria for this epic as having been met. Per discussion with the team in Princeton, Jointcal will not be made the default for HSC processing until the completion of DM-16596 . That ticket is a DRP deliverable, though, so doesn't block the completion of this epic. The HSC team have also requested that meas_mosaic not be dropped from lsst_distrib entirely until after the next HSC data release, currently scheduled for July 2019. They will be responsible for its continued maintenance to that date.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

            Show
            Parejkoj John Parejko added a comment - Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

            Show
            Parejkoj John Parejko added a comment - Linking the page describing the acceptance criteria, so that it is easier to find in the future: https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Watchers:
              Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Michael Wood-Vasey, Pierre Astier, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.