Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3445

Overhaul the way the right colorterm dictionary is selected

    XMLWordPrintable

    Details

    • Team:
      Data Access and Database

      Description

      There are some configuration issues associated with reference catalogs, including:

      1) Determining the source of the reference catalog. This information is used in two ways:

      • mapping the filter used to take an exposure to a filter in a reference catalog, for matching sources to reference objects (e.g. for astrometry and photometry)
      • picking the correct set of colorterms based on the filter and type of CCD used for the science image and filters in the reference catalog
        At present this information is encoded in the eups version name of the astrometry_net_data package used. This requires the use of eups and assumes astrometry.net index files. This is the most serious immediate issue.

      2) Colorterm correction data is saved in obs_* packages as config files. This makes it harder to include a temporal component.

      This ticket is a request for a cleaner solution.

      The plan has been to wait for "the new butler", e.g. DM-2404 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility for fixing the first problem is to put the source of the reference catalog into metadata attached to the reference catalog.

        Attachments

          Issue Links

            Activity

            rowen Russell Owen created issue -
            rowen Russell Owen made changes -
            Field Original Value New Value
            Link This issue relates to DM-3442 [ DM-3442 ]
            rowen Russell Owen made changes -
            Description There are some configuration issues associated with reference catalogs, including:
            - mapping the right filter used to take an exposure to the right filter in a reference catalog for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and similar data for the reference catalog

            At present this is done using the version name of the astrometry.net data file being used. This requires the use of eups and assumes astrometry.net index files. Also, much of the data ends up in a frozen config, which may not be a viable long-term solution; in particular, we hope to get the necessary information from the butler, at which point it will likely be too late to modify a config.

            This ticket is a request for a cleaner solution.

            We hope we can wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility is that we associate metadata with the reference catalog (something the reference object loader can do) and use that metadata to determine the mappings above.
            There are some configuration issues associated with reference catalogs, including:
            - mapping the right filter used to take an exposure to the right filter in a reference catalog for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and similar data for the reference catalog

            At present this is done using the eups version name of the astrometry.net data file being used. This requires the use of eups and assumes astrometry.net index files. Also, much of the data ends up in a frozen config, which may not be a viable long-term solution; in particular, we hope to get the necessary information from the butler, at which point it will likely be too late to modify a config.

            This ticket is a request for a cleaner solution.

            We hope we can wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility is that we associate metadata with the reference catalog (something the reference object loader can do) and use that metadata to determine the mappings above.
            rowen Russell Owen made changes -
            Description There are some configuration issues associated with reference catalogs, including:
            - mapping the right filter used to take an exposure to the right filter in a reference catalog for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and similar data for the reference catalog

            At present this is done using the eups version name of the astrometry.net data file being used. This requires the use of eups and assumes astrometry.net index files. Also, much of the data ends up in a frozen config, which may not be a viable long-term solution; in particular, we hope to get the necessary information from the butler, at which point it will likely be too late to modify a config.

            This ticket is a request for a cleaner solution.

            We hope we can wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility is that we associate metadata with the reference catalog (something the reference object loader can do) and use that metadata to determine the mappings above.
            There are some configuration issues associated with reference catalogs, including:

            1) Determining the source of the reference catalog. This information is used in two ways:
            - mapping the filter used to take an exposure to a filter in a reference catalog, for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and filters in the reference catalog
            At present this information is encoded in the eups version name of the astrometry_net_data package used. This requires the use of eups and assumes astrometry.net index files.

            2) Colorterm correction data is saved in obs_* packages as config files. This makes it harder to include a temporal component.

            This ticket is a request for a cleaner solution.

            The plan has been to wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility is that we associate metadata with the reference catalog (something the reference object loader can do) and use that metadata to determine the mappings above.
            rowen Russell Owen made changes -
            Watchers Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Robert Lupton, Russell Owen, Simon Krughoff, Tim Jenness [ Jim Bosch, John Swinbank, Kian-Tat Lim, Paul Price, Robert Lupton, Russell Owen, Simon Krughoff, Tim Jenness ] Jim Bosch, John Swinbank, Kian-Tat Lim, Nate Pease, Paul Price, Robert Lupton, Russell Owen, Simon Krughoff, Tim Jenness [ Jim Bosch, John Swinbank, Kian-Tat Lim, Nate Pease, Paul Price, Robert Lupton, Russell Owen, Simon Krughoff, Tim Jenness ]
            rowen Russell Owen made changes -
            Description There are some configuration issues associated with reference catalogs, including:

            1) Determining the source of the reference catalog. This information is used in two ways:
            - mapping the filter used to take an exposure to a filter in a reference catalog, for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and filters in the reference catalog
            At present this information is encoded in the eups version name of the astrometry_net_data package used. This requires the use of eups and assumes astrometry.net index files.

            2) Colorterm correction data is saved in obs_* packages as config files. This makes it harder to include a temporal component.

            This ticket is a request for a cleaner solution.

            The plan has been to wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility is that we associate metadata with the reference catalog (something the reference object loader can do) and use that metadata to determine the mappings above.
            There are some configuration issues associated with reference catalogs, including:

            1) Determining the source of the reference catalog. This information is used in two ways:
            - mapping the filter used to take an exposure to a filter in a reference catalog, for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and filters in the reference catalog
            At present this information is encoded in the eups version name of the astrometry_net_data package used. This requires the use of eups and assumes astrometry.net index files. This is the most serious immediate issue.

            2) Colorterm correction data is saved in obs_* packages as config files. This makes it harder to include a temporal component.

            This ticket is a request for a cleaner solution.

            The plan has been to wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility for fixing the first problem is to put the source of the reference catalog into metadata attached to the reference catalog.
            tjenness Tim Jenness made changes -
            Link This issue is duplicated by DM-167 [ DM-167 ]
            rowen Russell Owen made changes -
            Description There are some configuration issues associated with reference catalogs, including:

            1) Determining the source of the reference catalog. This information is used in two ways:
            - mapping the filter used to take an exposure to a filter in a reference catalog, for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and filters in the reference catalog
            At present this information is encoded in the eups version name of the astrometry_net_data package used. This requires the use of eups and assumes astrometry.net index files. This is the most serious immediate issue.

            2) Colorterm correction data is saved in obs_* packages as config files. This makes it harder to include a temporal component.

            This ticket is a request for a cleaner solution.

            The plan has been to wait for butler 2 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility for fixing the first problem is to put the source of the reference catalog into metadata attached to the reference catalog.
            There are some configuration issues associated with reference catalogs, including:

            1) Determining the source of the reference catalog. This information is used in two ways:
            - mapping the filter used to take an exposure to a filter in a reference catalog, for matching sources to reference objects (e.g. for astrometry and photometry)
            - picking the correct set of colorterms based on the filter and type of CCD used for the science image and filters in the reference catalog
            At present this information is encoded in the eups version name of the astrometry_net_data package used. This requires the use of eups and assumes astrometry.net index files. This is the most serious immediate issue.

            2) Colorterm correction data is saved in obs_* packages as config files. This makes it harder to include a temporal component.

            This ticket is a request for a cleaner solution.

            The plan has been to wait for "the new butler", e.g. DM-2404 (taking this use case into account in its design) before designing the solution. However, if we need something sooner, one possibility for fixing the first problem is to put the source of the reference catalog into metadata attached to the reference catalog.
            rowen Russell Owen made changes -
            Link This issue relates to DM-2404 [ DM-2404 ]
            frossie Frossie Economou made changes -
            Assignee Kian-Tat Lim [ ktl ]
            tjenness Tim Jenness made changes -
            Assignee Kian-Tat Lim [ ktl ] Nate Pease [ npease ]
            tjenness Tim Jenness made changes -
            Team Data Access and Database [ 10204 ]
            gcomoretto Gabriele Comoretto [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 21439 ]
            tjenness Tim Jenness made changes -
            Component/s obs_lsstSim [ 10764 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Invalid [ 11005 ]

              People

              Assignee:
              npease Nate Pease [X] (Inactive)
              Reporter:
              rowen Russell Owen
              Watchers:
              Frossie Economou, Jim Bosch, John Swinbank, Kian-Tat Lim, Nate Pease [X] (Inactive), Paul Price, Robert Lupton, Russell Owen, Simon Krughoff, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.