Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19473

Create Gaia DR2 reference catalog

    Details

    • Epic Name:
      f19b-ap-gaia
    • Story Points:
      30
    • WBS:
      02C.03.05
    • Team:
      Alert Production
    • Cycle:
      Fall 2019

      Description

      In order to make proper motion corrections, we need a reference catalog with proper motions. We also need new style refcats that have nJy fluxes and all of the other nice new fields. Gaia DR2 came out last year, and can give us all of these things. This epic is to hold the various steps necessary to put together a new Gaia DR2 reference catalog.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Jim Bosch had commented in private conversation that he wanted the resulting Gaia DR2 catalog to have the same HTM depth as the existing catalogs, for butler gen3 simplification. Unfortunately, Gaia DR1 uses depth=8, while PS1_PV3 uses depth=7. Is there a preference here?

            Show
            Parejkoj John Parejko added a comment - Jim Bosch had commented in private conversation that he wanted the resulting Gaia DR2 catalog to have the same HTM depth as the existing catalogs, for butler gen3 simplification. Unfortunately, Gaia DR1 uses depth=8 , while PS1_PV3 uses depth=7 . Is there a preference here?
            Hide
            Parejkoj John Parejko added a comment -

            Once this is complete, we can retire the subaru, decam, and lsst jointcal astrometryReferenceErr configs that were defined because the PS1 refcats don't have coordinate errors.

            Show
            Parejkoj John Parejko added a comment - Once this is complete, we can retire the subaru, decam, and lsst jointcal astrometryReferenceErr configs that were defined because the PS1 refcats don't have coordinate errors.
            Hide
            swinbank John Swinbank added a comment -

            Per discussion with John Parejko of 2019-07-30, the goal for this epic is to generate a reference catalog with HTM depth 7 based on the complete Gaia DR2 release, and containing the following fields:

            • id: Identifier provided by the Gaia catalog source_id.
            • coord_ra, coord_dec: As provided by Gaia's ra, dec.
            • coord_raErr, coord_decErr: As provided by Gaia's ra_error, dec_error.
            • epoch: As provided by Gaia's ref_epoch.
            • phot_g_mean_flux: Converted from Gaia's phot_g_mean_mag using existing code (this is not the same as Gaia's phot_g_mean_flux, which does not have physical units).
            • phot_g_mean_fluxErr: Computed from (LSST's) phot_g_mean_flux / (Gaia's) phot_g_mean_flux_over_error.
            • phot_bp_mean_flux, phot_rp_mean_flux, and equivalent error fields, computed as for phot_g_mean_flux.
            • pm_ra, pm_raErr: As provided by Gaia's pmra, pmra_error.
            • pm_dec, pm_decErr: As provided by Gaia's pmdec, pmdec_error.
            • pm_flag: Always False.
            • parallax, parallaxErr: As provided by Gaia's parallax, parallax_error.
            • parallax_flag: Always False.
            • astrometric_excess_noise: As provided by Gaia's astrometric_excess_noise.
            • phot_variable_flag: As provided by Gaia's phot_variable_flag.

            It will be made available on lsst-dev in the location specified in the Developer Guide.

            Robert Lupton, Jim Bosch — can you confirm that you are happy with the above, please?

            Show
            swinbank John Swinbank added a comment - Per discussion with John Parejko of 2019-07-30, the goal for this epic is to generate a reference catalog with HTM depth 7 based on the complete Gaia DR2 release, and containing the following fields: id : Identifier provided by the Gaia catalog source_id . coord_ra , coord_dec : As provided by Gaia's ra , dec . coord_raErr , coord_decErr : As provided by Gaia's ra_error , dec_error . epoch : As provided by Gaia's ref_epoch . phot_g_mean_flux : Converted from Gaia's phot_g_mean_mag using existing code (this is not the same as Gaia's phot_g_mean_flux , which does not have physical units). phot_g_mean_fluxErr : Computed from (LSST's) phot_g_mean_flux / (Gaia's) phot_g_mean_flux_over_error . phot_bp_mean_flux , phot_rp_mean_flux , and equivalent error fields, computed as for phot_g_mean_flux . pm_ra , pm_raErr : As provided by Gaia's pmra , pmra_error . pm_dec , pm_decErr : As provided by Gaia's pmdec , pmdec_error . pm_flag : Always False . parallax , parallaxErr : As provided by Gaia's parallax , parallax_error . parallax_flag : Always False . astrometric_excess_noise : As provided by Gaia's astrometric_excess_noise . phot_variable_flag : As provided by Gaia's phot_variable_flag . It will be made available on lsst-dev in the location specified in the Developer Guide. Robert Lupton , Jim Bosch — can you confirm that you are happy with the above, please?
            Hide
            rhl Robert Lupton added a comment -

            I think that looks fine.  I'm a bit concerned about getting the mean flux from the mags as you have to worry about Vega and AB mags, and the procedure by which I think they went from flux to mag seems to described on  https://gea.esac.esa.int/archive/documentation/GDR2/Data_processing/chap_cu5pho/sec_cu5pho_calibr/ssec_cu5pho_calibr_extern.html (e.g. just above equation 5.36) and it would be good to check that we do understand their gyrations.

            The use of errors in ra and dec (rather than the covariance matrix) is a bit of a concern, as they in fact do provide the correlation coefficients, but that can wait for another pass over the catalogues.

            Show
            rhl Robert Lupton added a comment - I think that looks fine.  I'm a bit concerned about getting the mean flux from the mags as you have to worry about Vega and AB mags, and the procedure by which I think they went from flux to mag seems to described on  https://gea.esac.esa.int/archive/documentation/GDR2/Data_processing/chap_cu5pho/sec_cu5pho_calibr/ssec_cu5pho_calibr_extern.html  (e.g. just above equation 5.36) and it would be good to check that we do understand their gyrations. The use of errors in ra and dec (rather than the covariance matrix) is a bit of a concern, as they in fact do provide the correlation coefficients, but that can wait for another pass over the catalogues.
            Hide
            jbosch Jim Bosch added a comment -

            Fine with me.  I do think we'll want to revisit reference catalog formats in the future, but I don't want that to get in the way of anything right now.

            Show
            jbosch Jim Bosch added a comment - Fine with me.  I do think we'll want to revisit reference catalog formats in the future, but I don't want that to get in the way of anything right now.
            Hide
            Parejkoj John Parejko added a comment -

            Damn it. I found the description of their magnitudes, and they're Vega, not AB. I'm going to have to rethink that part.

            Show
            Parejkoj John Parejko added a comment - Damn it. I found the description of their magnitudes, and they're Vega, not AB. I'm going to have to rethink that part.
            Hide
            Parejkoj John Parejko added a comment -

            The reference catalog is now available on lsst-dev with a summarizing README. See this Community post for details: https://community.lsst.org/t/gaia-dr2-reference-catalog-in-lsst-format/3901

            Show
            Parejkoj John Parejko added a comment - The reference catalog is now available on lsst-dev with a summarizing README. See this Community post for details: https://community.lsst.org/t/gaia-dr2-reference-catalog-in-lsst-format/3901

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Meredith Rawls, Paul Price, Robert Lupton, Simon Krughoff, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel