Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13054

Add colorterm config support to jointcal

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Epic Link:
    • Sprint:
      AP S18-6, AP F18-1, AP F18-2, AP F18-3, AP F18-6, AP S19-2, AP S19-3
    • Team:
      Alert Production

      Description

      Dominique Boutigny's work on color terms in u/fix_outliers should help improve the photometry models.

      This ticket is to port over Dominique Boutigny's work in the u/fix_outliers branch (specifically this commit: https://github.com/lsst/jointcal/commit/e1d0292c71275315a6b17b67ecaf47acd2ffbee8 ) to use ReferenceSourceSelectorTask, add a unittest to demonstrate that the color terms do affect the fit (for both the simple, and constrained models), and update the other tests to reflect any changes in the fit metrics.

        Attachments

          Issue Links

          blocks

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. DM-12793 implement color terms for jointcal photometry

          • Invalid

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-16702 Add reference stars to fgcmcal fit as an option

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-17743 Shepherd tickets for HSC release candidate

          • Done
          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11395 Update testdata_jointcal refcats to new Indexed format

          • Done
          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-17692 Examine ref and source flux error propagation in PhotoCalTask and colorterms

          • To Do
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-14061 Remove refCat.copy() lines from jointcal

          • Won't Fix

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-12918 investigate and merge u/fix_outliers

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-13053 allow refcat selection improve fitting

          • Done

            Activity

            Ascending order - Click to sort in descending order
            15 older comments
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            For the remaining small offset, we should look in more detail at how that is being calculated vs. the reference fluxes: could the analysis code be using incorrect colorterms? Yusra AlSayyad: do you want to file a ticket about investigating that further?

            Show
            Parejkoj John Parejko added a comment - For the remaining small offset, we should look in more detail at how that is being calculated vs. the reference fluxes: could the analysis code be using incorrect colorterms? Yusra AlSayyad : do you want to file a ticket about investigating that further?
            Hide
            Permalink
            erykoff Eli Rykoff added a comment -

            Are we talking about the 5mmag offset in new colortermified jointcal, and 2-3 mmag offset in meas_mosaic?

            Show
            erykoff Eli Rykoff added a comment - Are we talking about the 5mmag offset in new colortermified jointcal, and 2-3 mmag offset in meas_mosaic?
            Hide
            Permalink
            Parejkoj John Parejko added a comment - - edited

            One final jenkins run since I had to rebase things: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29415/pipeline

            Show
            Parejkoj John Parejko added a comment - - edited One final jenkins run since I had to rebase things: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29415/pipeline
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Eli Rykoff, Yes, that:

            and the 3 mmag shift in the `w` principle component from jointcal to meas_mosaic  new_jointcal_gri  which is probably a manifestation of the offsets above (and just another way of looking at it). Opening a ticket with this text now. 

            Show
            yusra Yusra AlSayyad added a comment - Eli Rykoff , Yes, that: the shift that jumps from 3 to 5mmag from the meas_mosaic to colortermified jointcal in HSC-I meas_mosaic  vs. jointcal_after 6 to 12mmag in HSC-R meas_mosaic_R vs. jointcal_R 8 to 8mmag (no change in HSC-G!) meas_mosaic_G vs. jointcal_G and the 3 mmag shift in the `w` principle component from jointcal to meas_mosaic   new_jointcal_gri   which is probably a manifestation of the offsets above (and just another way of looking at it). Opening a ticket with this text now. 
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thank you for the review and running it on the large HSC data.

            I had to cleanup the jointcal hsc tests because of the new hsc default config, but that finally passed the above jenkins run.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thank you for the review and running it on the large HSC data. I had to cleanup the jointcal hsc tests because of the new hsc default config, but that finally passed the above jenkins run. Merged and done.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Eli Rykoff
              Watchers:
              Dominique Boutigny, Eli Rykoff, Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Pierre Astier, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.