Data ManagementIcon indicating the project type

Data Management

Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16170

mosaic.py error "Field with name 'i_instFlux' not found"

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_mosaic
    • Labels:
      None

      Description

      With w_2018_41, meas_mosaic fails with errors like

      root WARN: Failed to read {'ccd': 24, 'visit': 1324, 'tract': 9615, 'filter': 'HSC-I'}: "Field with name 'i_instFlux' not found"

      and then Segmentation fault from zero matchList (Mosaic INFO: len(matchList) = 0 [] )

      One (longer than necessary) command to reproduce is 

      mosaic.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2018_41/DM-16011:private/usename/   --numCoresForReadSource=12 --id ccd=0..8^10..103 visit=26024^26028^26032^26036^26044^26046^26048^26050^26058^26060^26062^26070^26072^26074^26080^26084^26094 tract=9615

      The same command (with the same input data) works using the w_2018_39 stack.

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-16011 Reprocess RC2 with w_2018_41

          • Done

            Activity

            Ascending order - Click to sort in descending order
            12 older comments
            Hide
            Permalink
            lauren Lauren MacArthur added a comment -

            Is this really a workaround, or just reflective of the fact that reference catalogs don't have instFlux entries?

            Show
            lauren Lauren MacArthur added a comment - Is this really a workaround, or just reflective of the fact that reference catalogs don't have instFlux entries?
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            This is reverting the fix that was necessary because refcats were gaining an instFlux before Jim Bosch fixed the aliasing logic.

            Show
            Parejkoj John Parejko added a comment - This is reverting the fix that was necessary because refcats were gaining an instFlux before Jim Bosch fixed the aliasing logic.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for the quick review: merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks for the quick review: merged and done.
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            To further clarify for posterity:

            • Reference catalogs were incorrectly being read in with instFlux fields following the initial rename of flux to instFlux in source catalogs.
            • That was fixed in afw in DM-15857, but not before DM-15837 added a workaround to meas_mosaic to make it expect instFlux fields in reference catalogs.  The afw fix broke the meas_mosaic workaround; my question above was whether DM-15837 or any other ticket added any other workarounds that will have been broken by the more correct fix in afw.
            • DM-15891 also addressed some fallout from the instFlux change, but was always compatible with the afw fix in DM-15857.
            • DM-16068 augmented the afw fix from DM-15857 to handle some cases it had originally missed.
            • This ticket (DM-16170) reverts the DM-15837 workaround.
            Show
            jbosch Jim Bosch added a comment - To further clarify for posterity: Reference catalogs were incorrectly being read in with instFlux fields following the initial rename of flux to instFlux in source catalogs. That was fixed in afw in DM-15857 , but not before DM-15837 added a workaround to meas_mosaic to make it expect instFlux fields in reference catalogs.  The afw fix broke the meas_mosaic workaround; my question above was whether DM-15837 or any other ticket added any  other workarounds that will have been broken by the more correct fix in afw. DM-15891 also addressed some fallout from the instFlux change, but was always compatible with the afw fix in DM-15857 . DM-16068 augmented the afw fix from DM-15857 to handle some cases it had originally missed. This ticket ( DM-16170 ) reverts the DM-15837 workaround.
            Hide
            Permalink
            lauren Lauren MacArthur added a comment - - edited

            That said, I can't be sure that this doesn't break meas_mosaic master on pre-existing catalogs.

            I'll confirm this on DM-15916. UPDATE: confirmed with that branch 

            Show
            lauren Lauren MacArthur added a comment - - edited That said, I can't be sure that this doesn't break meas_mosaic master on pre-existing catalogs. I'll confirm this on DM-15916 . UPDATE : confirmed with that branch 

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Reviewers:
                Jim Bosch
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel