Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13437

Stop calibration product generation from causing lots of warnings/errors due to inability to create pretty pictures

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_drivers
    • Labels:
      None
    • Team:
      External

      Description

      Creating flats for obs_comCam generates errors like the following as the code tries to make a pretty picture.  While this is a nice idea I don't think that the default should be to generate these warnings. 

      ...
      flat INFO: Writing {'filter': '550CutOn', 'ccd': 'S20', 'calibDate': '2017-05-23'} on lsst-dev01.ncsa.illinois.edu:4089146
      Unable to fit image for detector "S00" into image of camera
      Unable to fit image for detector "S01" into image of camera
      Unable to fit image for detector "S02" into image of camera
      Unable to fit image for detector "S10" into image of camera
      Unable to fit image for detector "S11" into image of camera
      Unable to fit image for detector "S12" into image of camera
      Unable to fit image for detector "S20" into image of camera
      Unable to fit image for detector "S21" into image of camera
      Unable to fit image for detector "S22" into image of camera
      flat WARN: Unable to create camera image: 'ComCamMapper' object has no attribute 'map_flat_camera'
      

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            I have improved the mechanics of constructing the full-focalplane images, which removes all of the chatter except for the final warning (Unable to create camera image), which I have removed by adding the datasets to obs_comCam. Robert Lupton, could you please verify this is what you had in mind?

            price@pap-laptop:~/LSST/afw (tickets/DM-13437 %=) $ git sub
            commit 7660d09479ae2b41f848c45f68061a490dce8558 (HEAD -> tickets/DM-13437, origin/tickets/DM-13437)
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Thu Feb 1 16:42:07 2018 -0500
             
                makeImageFromCamera: catch exceptions from retrieving image
                
                This allows the 'imageSource' to throw an exception rather than returning
                a nonsense input.
             
             python/lsst/afw/cameraGeom/utils.py | 6 +++++-
             1 file changed, 5 insertions(+), 1 deletion(-)
             
             
            price@pap-laptop:~/LSST/pipe_drivers (tickets/DM-13437=) $ git sub
            commit 8489765068486868910d0320ccbc5976b59ca2db
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Thu Feb 1 15:16:58 2018 -0500
             
                constructCalibs: fix camera image construction
                
                The dict of calibs has been indexed by the "ccdName" (a tuple of
                CCD-specific dataId values), but our ImageSource doesn't know
                how to construct one of those from the detector it's given.
                Instead, index the calibs by the detector ID.
             
             python/lsst/pipe/drivers/constructCalibs.py | 13 ++++++-------
             1 file changed, 6 insertions(+), 7 deletions(-)
             
            commit 7c519fb91fa09d4d9a0806229cdd7d212cb9845d (HEAD -> tickets/DM-13437, origin/tickets/DM-13437)
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Thu Feb 1 17:19:36 2018 -0500
             
                constructCalibs: use fake image for full-focal plane if unavailable
                
                We have been returning 1x1 images if there's no image, and that generates
                annoying warnings with cryptic messages. Instead, generate a full-size
                fake image.
             
             python/lsst/pipe/drivers/constructCalibs.py | 8 ++++++--
             1 file changed, 6 insertions(+), 2 deletions(-)
             
             
            price@pap-laptop:~/LSST/obs_comCam (tickets/DM-13437=) $ git sub
            commit edd5a182fd96fb6f1fb0db265d783adae21a61c3 (HEAD -> tickets/DM-13437, origin/tickets/DM-13437)
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Thu Feb 1 17:29:15 2018 -0500
             
                mapper: add full focal-plane images of calibs
                
                Very useful for getting a quick feel of the quality of what you just built.
             
             policy/comCamMapper.yaml | 25 +++++++++++++++++++++++++
             1 file changed, 25 insertions(+)
            

            Show
            price Paul Price added a comment - I have improved the mechanics of constructing the full-focalplane images, which removes all of the chatter except for the final warning ( Unable to create camera image ), which I have removed by adding the datasets to obs_comCam. Robert Lupton , could you please verify this is what you had in mind? price@pap-laptop:~/LSST/afw (tickets/DM-13437 %=) $ git sub commit 7660d09479ae2b41f848c45f68061a490dce8558 (HEAD -> tickets/DM-13437, origin/tickets/DM-13437) Author: Paul Price <price@astro.princeton.edu> Date: Thu Feb 1 16:42:07 2018 -0500   makeImageFromCamera: catch exceptions from retrieving image This allows the 'imageSource' to throw an exception rather than returning a nonsense input.   python/lsst/afw/cameraGeom/utils.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)     price@pap-laptop:~/LSST/pipe_drivers (tickets/DM-13437=) $ git sub commit 8489765068486868910d0320ccbc5976b59ca2db Author: Paul Price <price@astro.princeton.edu> Date: Thu Feb 1 15:16:58 2018 -0500   constructCalibs: fix camera image construction The dict of calibs has been indexed by the "ccdName" (a tuple of CCD-specific dataId values), but our ImageSource doesn't know how to construct one of those from the detector it's given. Instead, index the calibs by the detector ID.   python/lsst/pipe/drivers/constructCalibs.py | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)   commit 7c519fb91fa09d4d9a0806229cdd7d212cb9845d (HEAD -> tickets/DM-13437, origin/tickets/DM-13437) Author: Paul Price <price@astro.princeton.edu> Date: Thu Feb 1 17:19:36 2018 -0500   constructCalibs: use fake image for full-focal plane if unavailable We have been returning 1x1 images if there's no image, and that generates annoying warnings with cryptic messages. Instead, generate a full-size fake image.   python/lsst/pipe/drivers/constructCalibs.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)     price@pap-laptop:~/LSST/obs_comCam (tickets/DM-13437=) $ git sub commit edd5a182fd96fb6f1fb0db265d783adae21a61c3 (HEAD -> tickets/DM-13437, origin/tickets/DM-13437) Author: Paul Price <price@astro.princeton.edu> Date: Thu Feb 1 17:29:15 2018 -0500   mapper: add full focal-plane images of calibs Very useful for getting a quick feel of the quality of what you just built.   policy/comCamMapper.yaml | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
            Show
            price Paul Price added a comment - GitHub: https://github.com/lsst/afw/pull/318 https://github.com/lsst/pipe_drivers/pull/51 https://github.com/lsst/obs_comCam/pull/9
            Hide
            rhl Robert Lupton added a comment -

            Minor comments on GitHub

            Show
            rhl Robert Lupton added a comment - Minor comments on GitHub
            Hide
            price Paul Price added a comment -

            Addressed review comments and merged to master.

            Show
            price Paul Price added a comment - Addressed review comments and merged to master.

              People

              Assignee:
              price Paul Price
              Reporter:
              rhl Robert Lupton
              Reviewers:
              Robert Lupton
              Watchers:
              Paul Price, Robert Lupton
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.