Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10826

fix jointcal collectRefStar handling of position errors

    XMLWordPrintable

Details

    • 4
    • AP S19-5, AP S19-6
    • Alert Production

    Description

      Jointcal does some odd things when building its RefStar catalog (Associations::collectRefStars), including creating fake errors (maybe because the original reference catalogs didn't have useful errors?), making a magnitude from the flux (is that really necessary? Why can't we work in flux units here?), and making BaseStars followed by a cast, when it should just be able to make RefStars.

      Attachments

        Issue Links

          Activity

            Pasted from github and private slack communication:

            Running with default configs should do something sensible. Our current default reference catalogs have '0' for their ra/dec errors.

            See two logs in /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_06_DM-10826/logs

            • HSC-G-9615-attempt1.log run with defaults:

              [yusra@lsst-dev01 logs]$ tail HSC-G-9615-attempt1.log
              jointcal.Associations INFO: Unmatched objects: 12
              jointcal.Associations INFO: Matched 146 objects in 26094_103
              jointcal.Associations INFO: Unmatched objects: 14
              jointcal.astrometryReferenceSelector INFO: Selected 1281797/1281797 references
              jointcal INFO: Applying color terms for filterName='g' reference catalog=ps1_pv3_3pi_20170110
              jointcal FATAL: Failed processing tract 9615, TypeError: 
                File "src/table/Schema.cc", line 211, in lsst::afw::table::SchemaItem<T> lsst::afw::table::detail::SchemaImpl::find(const string&) const [with T = double; std::string = std::basic_string<char>]
                  Field 'coord_ra_err' does not have the given type. {0}
              lsst::pex::exceptions::TypeError: 'Field 'coord_ra_err' does not have the given type.'
              

            • HSC-G-9615-attempt2.log was run with --config astrometryReferenceErr=10 and it completes.

            How to reproduce (note replacement of username):
            jointcal.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2019_06/DM-17400-sfm:private/parejkoj/RC2/w_2019_06_DM-10826 --id ccd=0..8^10..103 visit=26024^26028^26032^26036^26044^26046^26048^26050^26058^26060^26062^26070^26072^26074^26080^26084^26094 filter=HSC-G tract=9615 --no-versions --clobber-config &> HSC-G-9615.log

            yusra Yusra AlSayyad added a comment - Pasted from github and private slack communication: Running with default configs should do something sensible. Our current default reference catalogs have '0' for their ra/dec errors. See two logs in /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_06_ DM-10826 /logs HSC-G-9615-attempt1.log run with defaults: [yusra@lsst-dev01 logs]$ tail HSC-G-9615-attempt1.log jointcal.Associations INFO: Unmatched objects: 12 jointcal.Associations INFO: Matched 146 objects in 26094_103 jointcal.Associations INFO: Unmatched objects: 14 jointcal.astrometryReferenceSelector INFO: Selected 1281797/1281797 references jointcal INFO: Applying color terms for filterName='g' reference catalog=ps1_pv3_3pi_20170110 jointcal FATAL: Failed processing tract 9615, TypeError: File "src/table/Schema.cc", line 211, in lsst::afw::table::SchemaItem<T> lsst::afw::table::detail::SchemaImpl::find(const string&) const [with T = double; std::string = std::basic_string<char>] Field 'coord_ra_err' does not have the given type. {0} lsst::pex::exceptions::TypeError: 'Field 'coord_ra_err' does not have the given type.' HSC-G-9615-attempt2.log was run with --config astrometryReferenceErr=10 and it completes. How to reproduce (note replacement of username): jointcal.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2019_06/ DM-17400 -sfm:private/parejkoj/RC2/w_2019_06_ DM-10826 --id ccd=0..8^10..103 visit=26024^26028^26032^26036^26044^26046^26048^26050^26058^26060^26062^26070^26072^26074^26080^26084^26094 filter=HSC-G tract=9615 --no-versions --clobber-config &> HSC-G-9615.log
            Parejkoj John Parejko added a comment - - edited

            I've fixed the above issue and rebased/squashed. Are you happy with this, yusra? Should we have another ticket to set the subaru/lsst refcat coord errors?

            New Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29790/pipeline

            Parejkoj John Parejko added a comment - - edited I've fixed the above issue and rebased/squashed. Are you happy with this, yusra ? Should we have another ticket to set the subaru/lsst refcat coord errors? New Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29790/pipeline

            Please add the config for obs_subaru/decam (10mas) and obs_lsst (1mas) on this ticket. jointcal needs to do the thing (without commandline overrides).

            yusra Yusra AlSayyad added a comment - Please add the config for obs_subaru/decam (10mas) and obs_lsst (1mas) on this ticket. jointcal needs to do the thing (without commandline overrides).
            Parejkoj John Parejko added a comment -

            I added config overrides for lsst, decam, and subaru (and a ticket to remove them once the Gaia DR2 catalog is in place), and did a trial run with your command above, which seems to have worked. Do you want to take a quick look at the config PRs?

            Parejkoj John Parejko added a comment - I added config overrides for lsst, decam, and subaru (and a ticket to remove them once the Gaia DR2 catalog is in place), and did a trial run with your command above, which seems to have worked. Do you want to take a quick look at the config PRs?
            Parejkoj John Parejko added a comment -

            Thank you for the review comments, yusra and salnikov! I filed DM-19640 to remind us to remove these overrides once Gaia DR2 is in place.

            Merged and done.

            Parejkoj John Parejko added a comment - Thank you for the review comments, yusra and salnikov ! I filed DM-19640 to remind us to remove these overrides once Gaia DR2 is in place. Merged and done.

            People

              Parejkoj John Parejko
              Parejkoj John Parejko
              Yusra AlSayyad
              Andy Salnikov, Dominique Boutigny, Hisanori Furusawa, John Parejko, John Swinbank, Pierre Astier, Simon Krughoff (Inactive), Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.