Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7548

constructBias.py crashes with errors about log

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ctrl_pool, ip_isr
    • Labels:
      None

      Description

      Andreas Ritter got this error when running constructBias.py. It seems pipe_drivers needs changes following the logging framework transition in pipe_base.

      bias WARN: Unable to process DataId(initialdata={'category': 'A', 'taiObs': '2015-12-04', 'visit': 6301, 'dateObs': '2015-12-04', 'site': 'S', 'filter': 'r', 'field': 'BIAS', 'spectrograph': 2, 'ccd': 5, 'arm': 'r'}, tag=set([])): Wrong number or type of arguments for overloaded function 'Log_log'.
        Possible C/C++ prototypes are:
          lsst::log::Log::log(lsst::log::Log,log4cxx::LevelPtr,log4cxx::spi::LocationInfo const &,char const *,...)
          lsst::log::Log::log(log4cxx::LevelPtr,log4cxx::spi::LocationInfo const &,char const *,...)
       
      NotImplementedError on tiger1:27163 in map: Wrong number or type of arguments for overloaded function 'Log_log'.
        Possible C/C++ prototypes are:
          lsst::log::Log::log(lsst::log::Log,log4cxx::LevelPtr,log4cxx::spi::LocationInfo const &,char const *,...)
          lsst::log::Log::log(log4cxx::LevelPtr,log4cxx::spi::LocationInfo const &,char const *,...)
       
      Traceback (most recent call last):
        File "/tigress/HSC/LSST/stack/Linux64/ctrl_pool/12.0+45/python/lsst/ctrl/pool/pool.py", line 89, in wrapper
          return func(*args, **kwargs)
        File "/tigress/HSC/LSST/stack/Linux64/ctrl_pool/12.0+45/python/lsst/ctrl/pool/pool.py", line 207, in wrapper
          return func(*args, **kwargs)
        File "/tigress/HSC/LSST/stack/Linux64/ctrl_pool/12.0+45/python/lsst/ctrl/pool/pool.py", line 524, in map
          return self._processQueue(context, func, zip(range(num), dataList), *args, **kwargs)
        File "/tigress/HSC/LSST/stack/Linux64/ctrl_pool/12.0+45/python/lsst/ctrl/pool/pool.py", line 421, in _processQueue
          return [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
        File "/tigress/HSC/LSST/stack/Linux64/pipe_drivers/12.0-3-g33a9219+35/python/lsst/pipe/drivers/constructCalibs.py", line 459, in process
          exposure = self.processSingle(sensorRef)
        File "/tigress/HSC/LSST/stack/Linux64/pipe_drivers/12.0-3-g33a9219+35/python/lsst/pipe/drivers/constructCalibs.py", line 477, in processSingle
          return self.isr.runDataRef(dataRef).exposure
        File "/tigress/HSC/LSST/stack/Linux64/obs_subaru/12.0-18-gb01a726+1/python/lsst/obs/subaru/isr.py", line 268, in runDataRef
          ccdExposure = self.assembleCcd.assembleCcd(ccdExposure)
        File "/tigress/HSC/LSST/stack/Linux64/ip_isr/12.0-7-ga6ffce9+2/python/lsst/ip/isr/assembleCcdTask.py", line 229, in assembleCcd
          self.postprocessExposure(outExposure=outExposure, inExposure=getNextExposure(ccd[0]))
        File "/tigress/HSC/LSST/stack/Linux64/ip_isr/12.0-7-ga6ffce9+2/python/lsst/ip/isr/assembleCcdTask.py", line 243, in postprocessExposure
          self.setWcs(outExposure = outExposure, inExposure = inExposure)
        File "/tigress/HSC/LSST/stack/Linux64/ip_isr/12.0-7-ga6ffce9+2/python/lsst/ip/isr/assembleCcdTask.py", line 280, in setWcs
          self.log.log(self.log.WARN, "No WCS found in input exposure")
        File "/tigress/HSC/LSST/stack/Linux64/log/12.0-5-g1df3bbf/python/lsst/log/logLib.py", line 249, in log
          return _logLib.Log_log(self, *args)
      NotImplementedError: Wrong number or type of arguments for overloaded function 'Log_log'.
        Possible C/C++ prototypes are:
          lsst::log::Log::log(lsst::log::Log,log4cxx::LevelPtr,log4cxx::spi::LocationInfo const &,char const *,...)
          lsst::log::Log::log(log4cxx::LevelPtr,log4cxx::spi::LocationInfo const &,char const *,...)
       
      application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
      

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Andreas Ritter may you please review this and see if you can run constructBias with the branch tickets/DM-7548? Changes are in ip_isr, where your processing broke, and ctrl_pool, where I made some modifications so the lsst.log log is saved to a file called "jobs.log" as well as outputted to console, similar to the pex.logging log. As both kinds of logs are in use, I intend to keep both in this ticket.

            I don't have the data repo to run constructBias as in your specific case, but I was able to run singleFrameDriver with some DECam data on my machine with 2 cores. Jenkins is being run, although I doubt if any changes actually got tested there...

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Andreas Ritter may you please review this and see if you can run constructBias with the branch tickets/ DM-7548 ? Changes are in ip_isr , where your processing broke, and ctrl_pool , where I made some modifications so the lsst.log log is saved to a file called "jobs.log" as well as outputted to console, similar to the pex.logging log. As both kinds of logs are in use, I intend to keep both in this ticket. I don't have the data repo to run constructBias as in your specific case, but I was able to run singleFrameDriver with some DECam data on my machine with 2 cores. Jenkins is being run , although I doubt if any changes actually got tested there...
            Hide
            aritter Andreas Ritter added a comment -

            Fix confirmed and code changes look good. Feel free to merge into master.

            Show
            aritter Andreas Ritter added a comment - Fix confirmed and code changes look good. Feel free to merge into master.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Running another Jenkins, including ci_hsc and lsst_ci, after rebasing: https://ci.lsst.codes/job/stack-os-matrix/label=centos-7,python=py2/15881//console

            Also I test-ran singleFrameDriver.py with some DECam data on my Mac machine with core=2. Problem running with core>=3 is known in DM-5748.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Running another Jenkins, including ci_hsc and lsst_ci , after rebasing: https://ci.lsst.codes/job/stack-os-matrix/label=centos-7,python=py2/15881//console Also I test-ran singleFrameDriver.py with some DECam data on my Mac machine with core=2. Problem running with core>=3 is known in DM-5748 .
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Merged.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Merged.

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Reviewers:
                Andreas Ritter
                Watchers:
                Andreas Ritter, Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel