Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12947

constructFlat.py fails on HSC

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
    • Team:
      External

      Description

      Mineo-san writes in HSC-1425:

      When I try to construct flat with hscPipe version 6.0-beta2 by:

      constructFlat.py $PWD --rerun=testflat --batch-type=smp --cores=17 --id field=DOMEFLAT ccd=49 filter=HSC-Y --config isr.doBias=False isr.doDark=False
      

      the command fails with the following error message:

      flat INFO: Scatter combination
      flat INFO: Combining {'ccd': 49, 'filter': 'HSC-Y', 'calibDate': '2014-03-25'} on alize.mtk.nao.ac.jp:31558
      flat.combination INFO: alize.mtk.nao.ac.jp:31558: Measured background of stack is 0.999997; adjusting to 1.000000
      application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
       
      Traceback (most recent call last):
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/6.0b1-hsc+6/python/lsst/ctrl/pool/pool.py", line 113, in wrapper
          return func(*args, **kwargs)
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/6.0b1-hsc+6/python/lsst/ctrl/pool/pool.py", line 237, in wrapper
          return func(*args, **kwargs)
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/6.0b1-hsc+6/python/lsst/ctrl/pool/pool.py", line 713, in reduce
          *args, **kwargs)
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/6.0b1-hsc+6/python/lsst/ctrl/pool/pool.py", line 570, in _reduceQueue
          resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/6.0b1-hsc+6/python/lsst/ctrl/pool/pool.py", line 570, in <listcomp>
          resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_drivers/6.0b2-hsc/python/lsst/pipe/drivers/constructCalibs.py", line 731, in combine
          finalScale=struct.scales.ccdScale)
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/obs_subaru/6.0b1-hsc+6/python/lsst/obs/hsc/calibs.py", line 48, in run
          self.maskVignetting(mi.getMask(), detector)
        File "/data2/hscpipe/6.0-beta2/stack_6.0-beta2/stack/miniconda3-4.3.21-10a4fa6/Linux64/obs_subaru/6.0b1-hsc+6/python/lsst/obs/hsc/calibs.py", line 63, in maskVignetting
          transform = detector.getTransformMap().get(detector.makeCameraSys(afwcg.FOCAL_PLANE))
      AttributeError: 'lsst.afw.cameraGeom._transformMap.TransformMap' object has no attribute 'get'
      

        Attachments

          Activity

          Hide
          price Paul Price added a comment -

          Jim Bosch, would you please have a look at this?

          pprice@tiger-sumire:~/LSST/obs/subaru (tickets/DM-12947 $%=) $ git sub
          commit 8e16056310c8f1a1d7d73fd26190443d35badf7f
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Thu Dec 7 13:48:38 2017 -0500
           
              calibs: update HSC vignetting mask use of Transform
              
              The API in afw for transforms between different coordinate frames
              (including detector pixels <--> focal plane) changed, so we need
              to adapt.
           
           python/lsst/obs/hsc/calibs.py | 7 ++++---
           1 file changed, 4 insertions(+), 3 deletions(-)
           
          commit 3d4fc2812402dceb347876101d83579db04ba8c6
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Thu Dec 7 13:50:26 2017 -0500
           
              calibs: optimise pixel-by-pixel mask
              
              Calculating the focal-plane offset of each pixel is very slow (it always
              has been, but it seems slower with the new Transform API), but the new
              Transform API allows us to vectorise the calculation.
           
           python/lsst/obs/hsc/calibs.py | 15 +++++++--------
           1 file changed, 7 insertions(+), 8 deletions(-)
          

          Show
          price Paul Price added a comment - Jim Bosch , would you please have a look at this? pprice@tiger-sumire:~/LSST/obs/subaru (tickets/DM-12947 $%=) $ git sub commit 8e16056310c8f1a1d7d73fd26190443d35badf7f Author: Paul Price <price@astro.princeton.edu> Date: Thu Dec 7 13:48:38 2017 -0500   calibs: update HSC vignetting mask use of Transform The API in afw for transforms between different coordinate frames (including detector pixels <--> focal plane) changed, so we need to adapt.   python/lsst/obs/hsc/calibs.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)   commit 3d4fc2812402dceb347876101d83579db04ba8c6 Author: Paul Price <price@astro.princeton.edu> Date: Thu Dec 7 13:50:26 2017 -0500   calibs: optimise pixel-by-pixel mask Calculating the focal-plane offset of each pixel is very slow (it always has been, but it seems slower with the new Transform API), but the new Transform API allows us to vectorise the calculation.   python/lsst/obs/hsc/calibs.py | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
          Hide
          jbosch Jim Bosch added a comment -

          Looks good! I'm curious how big some temporaries are (see PR), but it looks like that's not actually anything new (though I'd like that confirmed). Otherwise totally fine.

          Show
          jbosch Jim Bosch added a comment - Looks good! I'm curious how big some temporaries are (see PR), but it looks like that's not actually anything new (though I'd like that confirmed). Otherwise totally fine.
          Hide
          price Paul Price added a comment -

          Thanks Jim.

          Merged to master.

          Show
          price Paul Price added a comment - Thanks Jim. Merged to master.

            People

            • Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, Paul Price
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel