Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16023

Include alias maps in output schema when denormalizing matches

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw, meas_astrom
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      DRP F18-5
    • Team:
      Data Release Production

      Description

      There are two functions in the stack that effectively generate a denormalized catalog of matches.  The matches between a reference and source catalog are "joined' into a single "match" catalog, putting a ref_ or src_ prefix on the field names for the reference and source fields, respectively.  However, when the schema for the match catalog is created, it does not propagate any aliases that may be set, which will then break any code looking for a field by its alias name (often happens with aliases set for backwards compatibility).  This ticket is to update the following functions to include the aliases from the individual catalogs in the schema for the matched catalog:

      • matchesToCatalog in afw (here
      • denormalizeMatches in meas_astrom (here)

      I will leave it for a future tickets whether these two very similar functions should be consolidated into one.

        Attachments

          Activity

          Hide
          lauren Lauren MacArthur added a comment -

          Pair coding with Jim Bosch

          Show
          lauren Lauren MacArthur added a comment - Pair coding with Jim Bosch
          Hide
          lauren Lauren MacArthur added a comment - - edited

          Might you have a moment to give this a look on behalf of Jim and myself?  You may note that there is no formal test for the function in afw. I believe I may be the only consumer at the moment (I use it in the pipe_analysis scripts, and indeed it was in running these that revealed the issue) and can confirm it now runs successfully with both old (VERSION < 3) and new catalogs, whereas, prior to this fix the backwards compatibility was broken. As I note in the description, extra effort would be better placed in seeing if these very similar functions can be merged into one.

          PR for afw is here and Jenkins + ci_hsc passed

          Show
          lauren Lauren MacArthur added a comment - - edited Might you have a moment to give this a look on behalf of Jim and myself?  You may note that there is no formal test for the function in afw . I believe I may be the only consumer at the moment (I use it in the pipe_analysis scripts, and indeed it was in running these that revealed the issue) and can confirm it now runs successfully with both old (VERSION < 3) and new catalogs, whereas, prior to this fix the backwards compatibility was broken. As I note in the description, extra effort would be better placed in seeing if these very similar functions can be merged into one. PR for afw  is here and Jenkins + ci_hsc passed
          Hide
          sullivan Ian Sullivan added a comment -

          Looks good. I had one minor comment on wording in afw.

          Show
          sullivan Ian Sullivan added a comment - Looks good. I had one minor comment on wording in afw.
          Hide
          lauren Lauren MacArthur added a comment -

          Thanks for the speedy review.  I addressed your comment, rebased (some churn in afw – including a revert today), re-Jenkinsed (passed), and merged.

          Show
          lauren Lauren MacArthur added a comment - Thanks for the speedy review.  I addressed your comment, rebased (some churn in afw – including a revert today), re-Jenkinsed ( passed ), and merged.

            People

            Assignee:
            lauren Lauren MacArthur
            Reporter:
            lauren Lauren MacArthur
            Reviewers:
            Ian Sullivan
            Watchers:
            Ian Sullivan, Jim Bosch, Lauren MacArthur, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.