Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19185

pipe_analysis needs updates for the Calib -> PhotoCalib changes

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      DRP S19-5
    • Team:
      Data Release Production

      Description

      DM-10156 changed Calib to PhotoCalib. It looks like some updates in pipe_analysis are needed. Running compareVisitAnalysis.py fails in the following error:

        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/17.0.1-2-g3e5d191+3/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/17.0.1-2-g3e5d191+3/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/visitAnalysis.py", line 768, in runDataRef
          postFix="_commonZp", **plotKwargs1)
        File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/coaddAnalysis.py", line 1337, in plotMags
          unitScale=self.unitScale),
        File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/utils.py", line 332, in __init__
          self.calib = afwImage.Calib()
      AttributeError: module 'lsst.afw.image' has no attribute 'Calib'
      

      An example to reproduce the above error is  compareVisitAnalysis.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2019_14/DM-18300-mosaic:private/user/name --rerun2 RC/w_2019_10/DM-17940 --no-versions --tract=9615 --id visit=26024

        Attachments

          Issue Links

            Activity

            No builds found.
            hchiang2 Hsin-Fang Chiang created issue -
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            (I forgot a second repo can be chained to --rerun too; description corrected) 

            Show
            hchiang2 Hsin-Fang Chiang added a comment - (I forgot a second repo can be chained to --rerun too; description corrected) 
            hchiang2 Hsin-Fang Chiang made changes -
            Field Original Value New Value
            Description DM-10156 changed Calib to PhotoCalib. It looks like some updates in {{pipe_analysis}} are needed. Running {{compareVisitAnalysis.py}} fails in the following error:
            {noformat} File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/17.0.1-2-g3e5d191+3/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
                result = self.runTask(task, dataRef, kwargs)
              File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/17.0.1-2-g3e5d191+3/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
                return task.runDataRef(dataRef, **kwargs)
              File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/visitAnalysis.py", line 768, in runDataRef
                postFix="_commonZp", **plotKwargs1)
              File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/coaddAnalysis.py", line 1337, in plotMags
                unitScale=self.unitScale),
              File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/utils.py", line 332, in __init__
                self.calib = afwImage.Calib()
            AttributeError: module 'lsst.afw.image' has no attribute 'Calib'
            {noformat}
            The above was from me running the command {{compareVisitAnalysis.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2019_14/DM-18300-mosaic --rerun2 RC/w_2019_10/DM-17940 --no-versions --tract=9615 --id visit=26024}}
             I expect permission errors if you run the command as-is, so some links in your space are probably needed. Using Butler in a more standard way and dumping outputs to a separate folder to make that easier is probably DM-14559.
            DM-10156 changed Calib to PhotoCalib. It looks like some updates in {{pipe_analysis}} are needed. Running {{compareVisitAnalysis.py}} fails in the following error:
            {noformat} File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/17.0.1-2-g3e5d191+3/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
                result = self.runTask(task, dataRef, kwargs)
              File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/17.0.1-2-g3e5d191+3/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
                return task.runDataRef(dataRef, **kwargs)
              File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/visitAnalysis.py", line 768, in runDataRef
                postFix="_commonZp", **plotKwargs1)
              File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/coaddAnalysis.py", line 1337, in plotMags
                unitScale=self.unitScale),
              File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/utils.py", line 332, in __init__
                self.calib = afwImage.Calib()
            AttributeError: module 'lsst.afw.image' has no attribute 'Calib'
            {noformat}
            An example to reproduce the above error is  {{compareVisitAnalysis.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2019_14/DM-18300-mosaic:private/user/name --rerun2 RC/w_2019_10/DM-17940 --no-versions --tract=9615 --id visit=26024}}
            lauren Lauren MacArthur made changes -
            Assignee Lauren MacArthur [ lauren ]
            lauren Lauren MacArthur made changes -
            Epic Link DM-16680 [ 235240 ]
            lauren Lauren MacArthur made changes -
            Sprint DRP S19-5 [ 898 ]
            Story Points 2
            lauren Lauren MacArthur made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue relates to DM-19128 [ DM-19128 ]
            Hide
            lauren Lauren MacArthur added a comment -

            Would you mind giving this a look? I have confirmed that the compare[Visit/Coadd]Analysis.py scripts now run and that the numbers coming out of the magnitude comparisons (esp. vs ref cats) are all self-consistent.

            Show
            lauren Lauren MacArthur added a comment - Would you mind giving this a look? I have confirmed that the compare [Visit/Coadd] Analysis.py scripts now run and that the numbers coming out of the magnitude comparisons (esp. vs ref cats) are all self-consistent.
            lauren Lauren MacArthur made changes -
            Reviewers Hsin-Fang Chiang [ hchiang2 ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Code changes look good to me. I didn't try running them but I'd take your words they are fine now  

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Code changes look good to me. I didn't try running them but I'd take your words they are fine now  
            hchiang2 Hsin-Fang Chiang made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            lauren Lauren MacArthur added a comment -

            Many thanks for another speedy review!

            Show
            lauren Lauren MacArthur added a comment - Many thanks for another speedy review!
            lauren Lauren MacArthur made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            yusra Yusra AlSayyad made changes -
            Story Points 2 4
            swinbank John Swinbank made changes -
            Epic Link DM-16680 [ 235240 ] DM-16678 [ 235238 ]

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Reviewers:
              Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, John Parejko, Lauren MacArthur, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.