Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14526

processCcd.py HSC data is broken with w_2018_20

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None

      Description

      $ setup lsst_distrib -t w_2018_20
      		 
      $ processCcd.py  /datasets/hsc/repo/ --output /scratch/whatever --id visit=34644 ccd=5
      		 
      root INFO: Loading config overrride file '/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/obs_subaru/15.0-8-gde10c556+2/config/processCcd.py'
      		 
      Traceback (most recent call last):
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-11-g8c8473f0/bin/processCcd.py", line 25, in <module>
      		 
          ProcessCcdTask.parseAndRun()
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-6-gfa9b38f+1/python/lsst/pipe/base/cmdLineTask.py", line 570, in parseAndRun
      		 
          parsedCmd = argumentParser.parse_args(config=config, args=args, log=log, override=cls.applyOverrides)
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-6-gfa9b38f+1/python/lsst/pipe/base/argumentParser.py", line 569, in parse_args
      		 
          self._applyInitialOverrides(namespace)
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-6-gfa9b38f+1/python/lsst/pipe/base/argumentParser.py", line 754, in _applyInitialOverrides
      		 
          namespace.config.load(filePath)
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pex_config/15.0-1-gae1598d+9/python/lsst/pex/config/config.py", line 558, in load
      		 
          self.loadFromStream(stream=code, root=root)
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pex_config/15.0-1-gae1598d+9/python/lsst/pex/config/config.py", line 578, in loadFromStream
      		 
          exec(stream, {}, local)
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/obs_subaru/15.0-8-gde10c556+2/config/processCcd.py", line 23, in <module>
      		 
          config.charImage.measurePsf.sourceSelector["objectSize"].sourceFluxField = 'base_PsfFlux_flux'
      		 
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pex_config/15.0-1-gae1598d+9/python/lsst/pex/config/configurableField.py", line 109, in __getattr__
      		 
          return getattr(self._value, name)
      		 
      AttributeError: 'MeasurePsfConfig' object has no attribute 'sourceSelector'

      Is this due to DM-10797?

        Attachments

          Issue Links

          is triggered by

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. DM-9832 Cleanup and unify star selector call signatures

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-10797 Refactor meas_algorithms/objectSizeStarSelector to new BaseClass

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-14339 Reprocess RC2 with w_2018_20

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            On master (~w_2018_20), MeasurePsfConfig does have starSelector, not sourceSelector:
            https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/measurePsf.py#L30-L31

            Show
            hchiang2 Hsin-Fang Chiang added a comment - On master (~w_2018_20), MeasurePsfConfig does have starSelector , not sourceSelector : https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/measurePsf.py#L30-L31
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Interesting. I thought I ran the full CI suite before merging DM-9832. Did ci_hsc not catch this?

            Looks like we need to revert two of the merges that were part of that, since I was over-zealous in my name changes:

            https://github.com/lsst/obs_lsstSim/pull/67/files
            https://github.com/lsst/obs_subaru/pull/129/files

            Show
            Parejkoj John Parejko added a comment - Interesting. I thought I ran the full CI suite before merging DM-9832 . Did ci_hsc not catch this? Looks like we need to revert two of the merges that were part of that, since I was over-zealous in my name changes: https://github.com/lsst/obs_lsstSim/pull/67/files https://github.com/lsst/obs_subaru/pull/129/files
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            If I read this Jenkins https://ci.lsst.codes/blue/organizations/jenkins/science-pipelines%2Flsst_distrib/activity/  correctly, ci_hsc has been failing since 7 day ago. 

            Show
            hchiang2 Hsin-Fang Chiang added a comment - If I read this Jenkins https://ci.lsst.codes/blue/organizations/jenkins/science-pipelines%2Flsst_distrib/activity/   correctly, ci_hsc has been failing since 7 day ago. 
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks, Hsin-Fang Chiang. Clearly I didn't run ci_hsc. I'm sorry!

            Does the new branch fix it for you?

            Show
            Parejkoj John Parejko added a comment - Thanks, Hsin-Fang Chiang . Clearly I didn't run ci_hsc. I'm sorry! Does the new branch fix it for you?
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Sadly, it looks like nobody is getting obvious notifications about the ci_hsc breakages?

            Show
            Parejkoj John Parejko added a comment - Sadly, it looks like nobody is getting obvious notifications about the ci_hsc breakages?
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Yes, changing the obs_subaru config as in your branch allows processCcd.py to run. 

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Yes, changing the obs_subaru config as in your branch allows processCcd.py to run. 
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Apparently nobody gets notifications about the ci_hsc breakage...  

            I thought the weekly release wouldn't go out if the CI fails. But we do have w_2018_20 from last night.  Apparently I was wrong...  

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Apparently nobody gets notifications about the ci_hsc breakage...   I thought the weekly release wouldn't go out if the CI fails. But we do have w_2018_20 from last night.  Apparently I was wrong...  
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for testing the fix. I've merged it now.

            I'll ask on #dm-square about making this kind of breakage more obvious.

            Show
            Parejkoj John Parejko added a comment - Thanks for testing the fix. I've merged it now. I'll ask on #dm-square about making this kind of breakage more obvious.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, John Parejko
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.