Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8828

Support proper motions in reference catalogs

    Details

    • Story Points:
      14
    • Epic Link:
    • Sprint:
      AP F18-2, AP F18-3, AP F18-4
    • Team:
      Alert Production

      Description

      Neglect of proper motion in the reference catalog can cause zonal astrometric errors in wide-field surveys (Tholen, Micheli & Elliott 2013). We therefore require a means to transform reference catalog positions to the epoch of observation.

      The PS1 PV3 catalog contains columns with the proper motions for RA and Dec in arcsec/year (pmRa, pmDec) with associated errors, and the mean epoch in seconds relative to the UNIX epoch (epoch). It would be good to support this scheme even if another is chosen.

        Attachments

          Issue Links

            Activity

            Hide
            rowen Russell Owen added a comment -

            Would it be practical to add support for ingesting parallax on this same ticket (while deferring applying parallax to a different ticket)? The advantages I see are that the reference catalog schema and ingest will be finished (assuming we go with Robert's desire to not support radial velocity, which is not my first choice but seems acceptable). Naively, at least, it looks like very little additional work.

            Show
            rowen Russell Owen added a comment - Would it be practical to add support for ingesting parallax on this same ticket (while deferring applying parallax to a different ticket)? The advantages I see are that the reference catalog schema and ingest will be finished (assuming we go with Robert's desire to not support radial velocity, which is not my first choice but seems acceptable). Naively, at least, it looks like very little additional work.
            Hide
            swinbank John Swinbank added a comment -

            Would it be practical to add support for ingesting parallax on this same ticket (while deferring applying parallax to a different ticket)?

            We discussed this on 2018-08-07 (with Eric Bellm); the answer is “yes”.

            Show
            swinbank John Swinbank added a comment - Would it be practical to add support for ingesting parallax on this same ticket (while deferring applying parallax to a different ticket)? We discussed this on 2018-08-07 (with Eric Bellm ); the answer is “yes”.
            Hide
            ebellm Eric Bellm added a comment -

            Russell Owen, I believe this page has the appropriate information about error propagation using the covariances: http://gea.esac.esa.int/archive/documentation/GDR2/Data_processing/chap_cu3ast/sec_cu3ast_intro/ssec_cu3ast_intro_tansforms.html

            Show
            ebellm Eric Bellm added a comment - Russell Owen , I believe this page has the appropriate information about error propagation using the covariances: http://gea.esac.esa.int/archive/documentation/GDR2/Data_processing/chap_cu3ast/sec_cu3ast_intro/ssec_cu3ast_intro_tansforms.html
            Hide
            rowen Russell Owen added a comment - - edited

            Changes:

            Add support for position errors, proper motion (with error) and parallax (with error) to reference catalogs. LoadReferenceCatalogTask schema documentation describes the extra fields and makeMinimalSchema can create them. IngestIndexedReferenceTask supports the new fields as well, though not the off-diagonal error terms. That limitation is documented and I don't see an easy way to fix it. Ingesting Gaia catalogs may require a redesign, perhaps moving the parsing of the foreign catalog into a subtask of IngestIndexedReferenceTask.

            I also modified IngestIndexedReferenceTask to create SimpleCatalog}}s instead of {{SourceCatalog}}s which required a corresponding simple update to {{obs_base. We don't need the extra fields in source catalogs, and reference catalogs were always supposed to be instances of SimpleCatalog, as documented in LoadReferenceCatalogTask.

            Note that initially this ticket included a loader for non-standard PanSTARRs catalog, but after discussion with Paul Price I removed that, thus eliminating the need for about half of the pull requests (though all were very small). I updated those pull requests to be the same as master, so I could run Jenkins. Our intent is to create a new version of this catalog (after figuring out which fields we are allowed to include – a process that I have started). Meanwhile the current one can be read, but proper motions will not be applied.

            Paul Price has also kindly agreed to take look over the code (not a full review) since he wrote the initial version of this code.

            Show
            rowen Russell Owen added a comment - - edited Changes: Add support for position errors, proper motion (with error) and parallax (with error) to reference catalogs. LoadReferenceCatalogTask schema documentation describes the extra fields and makeMinimalSchema can create them. IngestIndexedReferenceTask supports the new fields as well, though not the off-diagonal error terms. That limitation is documented and I don't see an easy way to fix it. Ingesting Gaia catalogs may require a redesign, perhaps moving the parsing of the foreign catalog into a subtask of IngestIndexedReferenceTask . I also modified IngestIndexedReferenceTask to create SimpleCatalog}}s instead of {{SourceCatalog}}s which required a corresponding simple update to {{obs_base . We don't need the extra fields in source catalogs, and reference catalogs were always supposed to be instances of SimpleCatalog , as documented in LoadReferenceCatalogTask . Note that initially this ticket included a loader for non-standard PanSTARRs catalog, but after discussion with Paul Price I removed that, thus eliminating the need for about half of the pull requests (though all were very small). I updated those pull requests to be the same as master, so I could run Jenkins. Our intent is to create a new version of this catalog (after figuring out which fields we are allowed to include – a process that I have started). Meanwhile the current one can be read, but proper motions will not be applied. Paul Price has also kindly agreed to take look over the code (not a full review) since he wrote the initial version of this code.
            Hide
            krughoff Simon Krughoff added a comment -

            Overall, this looks good. My main concern is that the actual application of the proper motion correction is not delegated to a utility function (perhaps on the catalogs class?) that in turn delegates to a third party package, but that is fairly minor, I guess.

            Show
            krughoff Simon Krughoff added a comment - Overall, this looks good. My main concern is that the actual application of the proper motion correction is not delegated to a utility function (perhaps on the catalogs class?) that in turn delegates to a third party package, but that is fairly minor, I guess.

              People

              • Assignee:
                rowen Russell Owen
                Reporter:
                price Paul Price
                Reviewers:
                Simon Krughoff
                Watchers:
                Chris Walter, Eric Bellm, Jim Bosch, John Parejko, John Swinbank, Michael Wood-Vasey, Paul Price, Russell Owen, Simon Krughoff, Wil O'Mullane
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel