Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4238

Fix integer casting error in numpy version 1.10 in obs subaru

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
      None
    • Story Points:
      0.5
    • Sprint:
      Science Pipelines DM-W16-3
    • Team:
      Data Release Production

      Description

      Fix type casting in obs_subaru in lates numpy in obs_subaru

        Attachments

          Issue Links

            Activity

            Hide
            nlust Nate Lust added a comment - - edited

            In case anyone has a similar error, here is the traceback that prompted this ticket. The solution involves explicitly casting the numpy array to dtype int.

            Traceback (most recent call last):
              File "/tigress/HSC/users/nate/repos/pipe_base/python/lsst/pipe/base/cmdLineTask.py", line 321, in __call__
                result = task.run(dataRef, **kwargs)
              File "/tigress/HSC/users/nate/repos/pipe_base/python/lsst/pipe/base/timer.py", line 118, in wrapper
                res = func(self, *args, **keyArgs)
              File "/tigress/HSC/users/nate/lsstsw/stack/Linux64/pipe_tasks/11.0-14-ga314014/python/lsst/pipe/tasks/processCcd.py", line 132, in run
                postIsrExposure = self.setPostIsrExposure(sensorRef)
              File "/tigress/HSC/users/nate/lsstsw/stack/Linux64/pipe_tasks/11.0-14-ga314014/python/lsst/pipe/tasks/processCcd.py", line 111, in setPostIsrExposure
                postIsrExposure = self.isr.runDataRef(sensorRef).exposure
              File "/tigress/HSC/users/nate/repos/obs_subaru/python/lsst/obs/subaru/isr.py", line 228, in runDataRef
                self.crosstalk.run(ccdExposure)
              File "/tigress/HSC/users/nate/repos/obs_subaru/python/lsst/obs/subaru/crosstalk.py", line 82, in run
                self.config.crosstalkMaskPlane)
              File "/tigress/HSC/users/nate/repos/obs_subaru/python/lsst/obs/subaru/crosstalk.py", line 270, in subtractXTalk
                np_msk[setThese][:] &= ~crosstalk
            TypeError: ufunc 'bitwise_and' output (typecode 'i') could not be coerced to provided output parameter (typecode 'H') according to the casting rule ''same_kind''
            

            Show
            nlust Nate Lust added a comment - - edited In case anyone has a similar error, here is the traceback that prompted this ticket. The solution involves explicitly casting the numpy array to dtype int . Traceback (most recent call last): File "/tigress/HSC/users/nate/repos/pipe_base/python/lsst/pipe/base/cmdLineTask.py", line 321, in __call__ result = task.run(dataRef, **kwargs) File "/tigress/HSC/users/nate/repos/pipe_base/python/lsst/pipe/base/timer.py", line 118, in wrapper res = func(self, *args, **keyArgs) File "/tigress/HSC/users/nate/lsstsw/stack/Linux64/pipe_tasks/11.0-14-ga314014/python/lsst/pipe/tasks/processCcd.py", line 132, in run postIsrExposure = self.setPostIsrExposure(sensorRef) File "/tigress/HSC/users/nate/lsstsw/stack/Linux64/pipe_tasks/11.0-14-ga314014/python/lsst/pipe/tasks/processCcd.py", line 111, in setPostIsrExposure postIsrExposure = self.isr.runDataRef(sensorRef).exposure File "/tigress/HSC/users/nate/repos/obs_subaru/python/lsst/obs/subaru/isr.py", line 228, in runDataRef self.crosstalk.run(ccdExposure) File "/tigress/HSC/users/nate/repos/obs_subaru/python/lsst/obs/subaru/crosstalk.py", line 82, in run self.config.crosstalkMaskPlane) File "/tigress/HSC/users/nate/repos/obs_subaru/python/lsst/obs/subaru/crosstalk.py", line 270, in subtractXTalk np_msk[setThese][:] &= ~crosstalk TypeError: ufunc 'bitwise_and' output (typecode 'i') could not be coerced to provided output parameter (typecode 'H') according to the casting rule ''same_kind''
            Hide
            price Paul Price added a comment -

            Does the astype('int') produce a copy or a view? If a copy, then the modification being made in the next line will be lost.

            I suggest adding an explanation in the commit message pointing to whatever changed in numpy.

            Show
            price Paul Price added a comment - Does the astype('int') produce a copy or a view? If a copy, then the modification being made in the next line will be lost. I suggest adding an explanation in the commit message pointing to whatever changed in numpy.
            Hide
            nlust Nate Lust added a comment - - edited

            You are right to check. I seem to remember astype returning a view, but it seems to return a copy now (unless it can be sure of compatible cast I think). I updated the solution to not cast the array, but cast the integer being compared.

            Show
            nlust Nate Lust added a comment - - edited You are right to check. I seem to remember astype returning a view, but it seems to return a copy now (unless it can be sure of compatible cast I think). I updated the solution to not cast the array, but cast the integer being compared.
            Hide
            tjenness Tim Jenness added a comment -

            I've linked this ticket to DM-4063, which covers the same ground.

            Show
            tjenness Tim Jenness added a comment - I've linked this ticket to DM-4063 , which covers the same ground.
            Hide
            tjenness Tim Jenness added a comment -

            This highlights why it would be good to have the obs_ packages in CI – we fixed all the casting problems we found from CI in DM-4063.

            Show
            tjenness Tim Jenness added a comment - This highlights why it would be good to have the obs_ packages in CI – we fixed all the casting problems we found from CI in DM-4063 .
            Hide
            price Paul Price added a comment -

            We delivered a CI package to the SQuARE team for integration into CI in DM-3663.

            Show
            price Paul Price added a comment - We delivered a CI package to the SQuARE team for integration into CI in DM-3663 .

              People

              • Assignee:
                nlust Nate Lust
                Reporter:
                nlust Nate Lust
                Reviewers:
                Paul Price
                Watchers:
                John Swinbank, Nate Lust, Paul Price, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: