Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-575

Convert old refcats to nJy for RFC-549

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      As fallout from RFC-549, we have to transition our existing reference catalogs from having fluxes in Jansky (most only implicitly: most don't have units on their flux fields at all!) to nanoJansky. We do not have versioning for reference catalogs (the afw.table version has a different purpose and is not propagated to the butler), so we have no way to identify "old" refcats other than attempting to guess from the various fields they do or don't contain.

      I propose to do this by writing a script that will convert refcat files to have nJy units and having LoadReferenceObjectsTask.loadSkyCircle raise an exception pointing to that script if it encounters a refcat that has anything other than "nJy" for flux units. This will greatly simplify this transition and will allow us to also fix the fluxSigma->fluxErr values in those refcats, while maintaining the old refcats in-place for older versions of the stack.

      For example, on lsst-dev, we would copy each of the existing directories in /datasets/refcats/htm, appending "-nJy"; run the script on those new directories; and update the obs package config defaults to point to the new refcats. Those old refcats could then be kept in place until at least v18.

      All newly created refcats will have fluxes in units of nJy.

      This would be made much easier if we could drop support for astrometry.net reference catalogs at the same time (RFC-562): a.net refcats are persisted in a way that makes them much harder to modify in-place. I believe the only important a.net refcat that we are still using is in testdata_jointcal, and I have a plan for dealing with it soon.

        Attachments

          Issue Links

            Activity

            Parejkoj John Parejko created issue -
            Parejkoj John Parejko made changes -
            Field Original Value New Value
            Link This issue is triggered by RFC-549 [ RFC-549 ]
            Parejkoj John Parejko made changes -
            Watchers Jim Bosch, John Parejko, John Swinbank, Meredith Rawls, Paul Price, Simon Krughoff, Yusra AlSayyad [ Jim Bosch, John Parejko, John Swinbank, Meredith Rawls, Paul Price, Simon Krughoff, Yusra AlSayyad ] Jim Bosch, John Parejko, John Swinbank, Leanne Guy, Meredith Rawls, Paul Price, Simon Krughoff, Yusra AlSayyad [ Jim Bosch, John Parejko, John Swinbank, Leanne Guy, Meredith Rawls, Paul Price, Simon Krughoff, Yusra AlSayyad ]
            Parejkoj John Parejko made changes -
            Link This issue relates to RFC-573 [ RFC-573 ]
            swinbank John Swinbank made changes -
            Status Proposed [ 10805 ] Flagged [ 10606 ]
            tjenness Tim Jenness made changes -
            Watchers Jim Bosch, John Parejko, John Swinbank, Leanne Guy, Meredith Rawls, Paul Price, Simon Krughoff, Yusra AlSayyad [ Jim Bosch, John Parejko, John Swinbank, Leanne Guy, Meredith Rawls, Paul Price, Simon Krughoff, Yusra AlSayyad ] Colin Slater, Gabriele Comoretto, Jim Bosch, John Parejko, John Swinbank, Kian-Tat Lim, Leanne Guy, Margaret Gelman, Meredith Rawls, Paul Price, Robert Lupton, Simon Krughoff, Tim Jenness, Wil O'Mullane, Yusra AlSayyad [ Colin Slater, Gabriele Comoretto, Jim Bosch, John Parejko, John Swinbank, Kian-Tat Lim, Leanne Guy, Margaret Gelman, Meredith Rawls, Paul Price, Robert Lupton, Simon Krughoff, Tim Jenness, Wil O'Mullane, Yusra AlSayyad ]
            tjenness Tim Jenness made changes -
            Link This issue relates to RFC-562 [ RFC-562 ]
            tjenness Tim Jenness made changes -
            Link This issue relates to RFC-535 [ RFC-535 ]
            Parejkoj John Parejko made changes -
            Planned End 22/Feb/19 12:18 AM 23/Feb/19 12:18 AM
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 19820 ]
            lguy Leanne Guy made changes -
            Assignee John Parejko [ parejkoj ] Leanne Guy [ lguy ]
            swinbank John Swinbank made changes -
            Assignee Leanne Guy [ lguy ] John Parejko [ parejkoj ]
            lguy Leanne Guy made changes -
            Planned End 23/Feb/19 12:18 AM 07/Mar/19 12:18 AM
            rgill Ranpal Gill made changes -
            Assignee John Parejko [ parejkoj ] Ranpal Gill [ rgill ]
            rgill Ranpal Gill made changes -
            Assignee Ranpal Gill [ rgill ] John Parejko [ parejkoj ]
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 19945 ]
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-18303 [ DM-18303 ]
            womullan Wil O'Mullane made changes -
            Status Flagged [ 10606 ] Board Recommended [ 11405 ]
            Parejkoj John Parejko made changes -
            Link This issue is triggering DM-17029 [ DM-17029 ]
            Parejkoj John Parejko made changes -
            Status Board Recommended [ 11405 ] Adopted [ 10806 ]
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 20030 ]
            Parejkoj John Parejko made changes -
            Resolution Done [ 10000 ]
            Status Adopted [ 10806 ] Implemented [ 11105 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue relates to DM-19150 [ DM-19150 ]
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 20326 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue relates to DM-19580 [ DM-19580 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-20284 [ DM-20284 ]
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-34793 [ DM-34793 ]
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-34792 [ DM-34792 ]

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Watchers:
              Colin Slater, Gabriele Comoretto [X] (Inactive), Jim Bosch, John Parejko, John Swinbank, Kian-Tat Lim, Leanne Guy, Margaret Gelman, Meredith Rawls, Paul Price, Robert Lupton, Simon Krughoff, Tim Jenness, Wil O'Mullane, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.