Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25849

Update HSC to use jointcal refcat defaults

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: jointcal
    • Labels:
      None
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      AP F20-5 (October), AP F20-6 (November), AP S21-1 (December)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      HSC still has PS1 overrides when running jointcal. This means that it is not using the new Gaia DR2 and proper motion updates, and will not follow along with any other jointcal refcat improvements.

      I can provide what I believe are appropriate fixes (mostly removals) to the HSC jointcal configs, but I do not have the expertise to determine whether the changes are what HSC really wants. In particular, the narrow band filters may require additional overrides for e.g. filterMaps for Gaia, and the Gaia data may or may not be sufficient for those filters due to depth or other properties.

      I do not know how to estimate the time required to validate such a change, but I'm creating this ticket to have a place to put updates to obs_subaru: I have a first cut ready to go.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-25807 refcat requireProperMotion=True config option bypassed for misconfigured catalog

          • Done
          is triggered by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24277 Apply proper motion and parallax while loading refcats in Jointcal

          • Done

            Activity

            Ascending order - Click to sort in descending order
            5 older comments
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment - - edited

            Some results from `/datasets/hsc/repo/rerun/private/yusra/RC2/DM-25849`

            https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AM1.png
            https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AF1_design.png
            Mostly not much of a change. Slightly better in some tract/filter combos. HSC-Z and HSC-Y get a little worse on 9813 (the ultra deep minimally dithered tract).

            Log for 9813, HSC-Y is in /project/yusra/datasets/hsc/rerun/private/yusra/RC2/DM-25849/logs/jointcal-9813-HSC-Y-28002.log

            for 20' pairs:
            https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AM2.png
            https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AF2_design.png

            Aside from HSC-Y 9813, seems to not do any harm. Thoughts?

            Show
            yusra Yusra AlSayyad added a comment - - edited Some results from `/datasets/hsc/repo/rerun/private/yusra/RC2/ DM-25849 ` https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AM1.png https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AF1_design.png Mostly not much of a change. Slightly better in some tract/filter combos. HSC-Z and HSC-Y get a little worse on 9813 (the ultra deep minimally dithered tract). Log for 9813, HSC-Y is in /project/yusra/datasets/hsc/rerun/private/yusra/RC2/ DM-25849 /logs/jointcal-9813-HSC-Y-28002.log for 20' pairs: https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AM2.png https://lsst.ncsa.illinois.edu/~yusra/download/DM-25849-AF2_design.png Aside from HSC-Y 9813, seems to not do any harm. Thoughts?
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Also, see convo on #dm-hsc-reprocessing.

            Next official RC2 HSC rerun is w_2020_50, which is tagged tomorrow night. Think you'd be able to merge tomorrow to get it into w_2020_50?

            Show
            yusra Yusra AlSayyad added a comment - Also, see convo on #dm-hsc-reprocessing. Next official RC2 HSC rerun is w_2020_50, which is tagged tomorrow night. Think you'd be able to merge tomorrow to get it into w_2020_50?
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Would someone please do a review of the two PRs? Should be trivial.

            Show
            Parejkoj John Parejko added a comment - Would someone please do a review of the two PRs? Should be trivial.
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Done. See comment on the obs_subaru PR about a more dramatic commit message. When I hear 'cleanup' I think 'no behavior changes to see here.'

            Show
            yusra Yusra AlSayyad added a comment - Done. See comment on the obs_subaru PR about a more dramatic commit message. When I hear 'cleanup' I think 'no behavior changes to see here.'
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for the processing run and code review Yusra AlSayyad. I fix that commit message.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks for the processing run and code review Yusra AlSayyad . I fix that commit message. Merged and done.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Yusra AlSayyad
              Watchers:
              John Parejko, Lauren MacArthur, Paul Price, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.