Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23992

Cannot load refcats in Gen 3 DECam processing

    Details

    • Story Points:
      2
    • Epic Link:
    • Sprint:
      DRP S20-5 (Apr)
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Attempting to run the Gen 3 single-frame pipeline as described in DM-23616 leads to AstrometryTask crashing with the error "No reference tables could be found for input region". Running with --show graph reports that no photometric or astrometric reference catalog shards are included in the quanta for CalibrateTask. However, they can be retrieved from the Butler using Registry.queryDatasets (only tested with no data ID constraints).

      The working hypothesis from #dm-middleware is that this is a problem in ingestion (i.e., in convert_gen2_repo_to_gen3.py), so I'm filing this as a bug in obs_base.

      Please fix the conversion(?) so that a repository created as described in DM-23616 can correctly match catalogs to image data IDs.

        Attachments

          Issue Links

            Activity

            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            I know HSC does that, but is there really no better way? It leads to a situation where configs are really easy to do incorrectly (especially since we'd have to modify every extant config that deals with refcats for DECam).

            Show
            krzys Krzysztof Findeisen added a comment - - edited I know HSC does that, but is there really no better way? It leads to a situation where configs are really easy to do incorrectly (especially since we'd have to modify every extant config that deals with refcats for DECam).
            Hide
            tjenness Tim Jenness added a comment -

            Not until the tasks are clever enough to distinguish physical filter from abstract filter. The Filter rewrite of afw was going to fix some of this since it would end up with that distinction being coded into the Filter class.

            Show
            tjenness Tim Jenness added a comment - Not until the tasks are clever enough to distinguish physical filter from abstract filter. The Filter rewrite of afw was going to fix some of this since it would end up with that distinction being coded into the Filter class.
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            Because of changes to the config, the workaround for the original bug is now config.defineVisits.computeVisitRegions.active.padding = 32768.

            Show
            krzys Krzysztof Findeisen added a comment - - edited Because of changes to the config, the workaround for the original bug is now config.defineVisits.computeVisitRegions.active.padding = 32768 .
            Hide
            nlust Nate Lust added a comment -

            I merged the small commits to unblock AP. Any future work around this topic should happen on a new ticket.

            Show
            nlust Nate Lust added a comment - I merged the small commits to unblock AP. Any future work around this topic should happen on a new ticket.
            Hide
            swinbank John Swinbank added a comment -

            Hey folks — much appreciated if you can add story point estimates to tickets, please. Thanks!

            Show
            swinbank John Swinbank added a comment - Hey folks — much appreciated if you can add story point estimates to tickets, please. Thanks!

              People

              • Assignee:
                nlust Nate Lust
                Reporter:
                krzys Krzysztof Findeisen
                Watchers:
                Dino Bektesevic, Jim Bosch, John Parejko, John Swinbank, Krzysztof Findeisen, Nate Lust, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel