Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28214

Adapt pipe_analysis scripts to allow loading of Gaia reference catalogs for astrometric matching

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Epic Link:
    • Sprint:
      DRP S21a (Dec Jan)
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      We have the ability to use either Gaia or PanSTARRS reference catalogs for astrometric fitting in jointcal.  As of DM-17597, the default was set to the Gaia reference catalogs (for astrometric fitting only, the default for photometry is still PanSTARRS PS1), but obs_subaru was still overriding to use PS1.  As of DM-25849, this override has been removed so that HSC processing now uses the jointcal defaults.  The scripts in pipe_analysis create plots and measure metrics based on a matching of pipeline sources to reference catalog sources and had PS1 hard-coded as the reference catalog that gets used in both types of calibration metrics.  Update the scripts to be able to use either Gaia or PS1 for astrometry.  PS1 is still the only contender for photometry (for both jointcal or fgcm bases photometric solutions).

        Attachments

          Issue Links

            Activity

            No builds found.
            lauren Lauren MacArthur created issue -
            lauren Lauren MacArthur made changes -
            Field Original Value New Value
            Status To Do [ 10001 ] In Progress [ 3 ]
            lauren Lauren MacArthur made changes -
            Summary Adapt pipe_analysis scripts to allow loading of Gaia reference catalogs for astrometic matching Adapt pipe_analysis scripts to allow loading of Gaia reference catalogs for astrometric matching
            lauren Lauren MacArthur made changes -
            lauren Lauren MacArthur made changes -
            lauren Lauren MacArthur made changes -
            Hide
            lauren Lauren MacArthur added a comment - - edited

            Ok, this is all working now.  There are now separate loaders for astrometry and photometry.  The reference catalog for either one can be set by setting a config: -c astromRefCat="gaia" or "ps1" for astrometric plots/metrics (or photomRefCat for photometry, but we only ever use ps1 for photometric calibration at this point...and the code will let you know that if you try setting it otherwise!). The defaults match our current settings for processing, so gaia_dr2_20200414 for astrometry and ps1_pv3_3pi_20170110 for photometry. The match plots indicate which catalog was used for the reference catalog so it will be clear what you are looking at. For example, this astrometric metric used gaia:

            whereas this photometric one used ps1:

            I am running the scripts on the w_2021_02 RC2 run and all looks good with the jointcal fix of DM-28233. The following compares it with the v21_0_0_rc1 run and demonstrates (in a slightly more round-about route than Eli Rykoff showed on DM-28173!) the offsets between the two reference catalogs

            If there are any other plots you would like to look at, they can be found here which was run using the defaults. Just to make sure it works, I also ran with -c astromRefCat="ps1" and those plots can be found here.

            Show
            lauren Lauren MacArthur added a comment - - edited Ok, this is all working now.  There are now separate loaders for astrometry and photometry.  The reference catalog for either one can be set by setting a config: -c astromRefCat="gaia" or "ps1" for astrometric plots/metrics (or  photomRefCat for photometry, but we only ever use ps1 for photometric calibration at this point...and the code will let you know that if you try setting it otherwise!). The defaults match our current settings for processing, so gaia_dr2_20200414 for astrometry and ps1_pv3_3pi_20170110 for photometry. The match plots indicate which catalog was used for the reference catalog so it will be clear what you are looking at. For example, this astrometric metric used gaia: whereas this photometric one used ps1: I am running the scripts on the w_2021_02 RC2 run and all looks good with the jointcal fix of DM-28233 . The following compares it with the v21_0_0_rc1 run and demonstrates (in a slightly more round-about route than Eli Rykoff showed on DM-28173 !) the offsets between the two reference catalogs If there are any other plots you would like to look at, they can be found here which was run using the defaults. Just to make sure it works, I also ran with -c astromRefCat="ps1" and those plots can be found here .
            Hide
            lauren Lauren MacArthur added a comment -

            Could you give this a look when you get a chance?

            Show
            lauren Lauren MacArthur added a comment - Could you give this a look when you get a chance?
            lauren Lauren MacArthur made changes -
            Reviewers Yusra AlSayyad [ yusra ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            lauren Lauren MacArthur made changes -
            Remote Link This issue links to "Page (Confluence)" [ 27298 ]
            lauren Lauren MacArthur made changes -
            Story Points 8
            lauren Lauren MacArthur made changes -
            Remote Link This issue links to "Page (Confluence)" [ 27721 ]
            Hide
            yusra Yusra AlSayyad added a comment -

            obs_base and obs_subaru fine, and I'm feeling very unpicky about pipe_analysis today. Good to merge

            Show
            yusra Yusra AlSayyad added a comment - obs_base and obs_subaru fine, and I'm feeling very unpicky about pipe_analysis today. Good to merge
            yusra Yusra AlSayyad made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            lauren Lauren MacArthur added a comment -

            Awesome, thanks again!  Jenkins + ci_hsc passed.  Merged and done.

            Show
            lauren Lauren MacArthur added a comment - Awesome, thanks again!  Jenkins + ci_hsc passed .  Merged and done.
            lauren Lauren MacArthur made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Lauren MacArthur, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.