Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14257

matchedVisitMetrics.py is broken

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Starting with w_2018_16, Running matchedVisitMetrics.py failed with the following:

      Traceback (most recent call last):
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/15.0-5-gfa4ad3e/bin/matchedVisitMetrics.py", line 25, in <module>
          MatchedVisitMetricsTask.parseAndRun()
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-4-g654b129+10/python/lsst/pipe/base/cmdLineTask.py", line 596, in parseAndRun
          resultList = taskRunner.run(parsedCmd)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-4-g654b129+10/python/lsst/pipe/base/cmdLineTask.py", line 238, in run
          if self.precall(parsedCmd):
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-4-g654b129+10/python/lsst/pipe/base/cmdLineTask.py", line 352, in precall
          task = self.makeTask(parsedCmd=parsedCmd)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-4-g654b129+10/python/lsst/pipe/base/cmdLineTask.py", line 324, in makeTask
          return self.TaskClass(config=self.config, log=self.log)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/15.0-5-gfa4ad3e/python/lsst/validate/drp/matchedVisitMetricsTask.py", line 113, in __init__
          self.metrics = MetricSet._load_metrics_yaml(metricsFile)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/verify/15.0+9/python/lsst/verify/metricset.py", line 171, in _load_metrics_yaml
          metric = Metric.deserialize(name=name, **metric_doc)
      TypeError: deserialize() got an unexpected keyword argument 'operator'
      

      A quick command to reproduce is

      matchedVisitMetrics.py /datasets/hsc/repo/rerun/RC/w_2018_17/DM-14055/ --output  /home/your/folder/ --id tract=9697 visit=6320 
      

      A full command I tried to run was:

      matchedVisitMetrics.py /datasets/hsc/repo/rerun/RC/w_2018_17/DM-14055/ --output  /home/your/folder/ --config useJointCal=True --id tract=9697 visit=6320^34338^34342^34362^34366^34382^34384^34400^34402^34412^34414^34422^34424^34448^34450^34464^34468^34478^34480^34482^34484^34486 ccd=0..8^10..103
      

      It failed the same way with w_2018_17 too.
      The same command worked with w_2018_15.

        Attachments

          Issue Links

            Activity

            Hide
            afausti Angelo Fausti added a comment -

            Simon Krughoff Hsin-Fang Chiang I created DM-14328 to discuss what we need to send these results to SQuaSH.

            Show
            afausti Angelo Fausti added a comment - Simon Krughoff Hsin-Fang Chiang I created DM-14328 to discuss what we need to send these results to SQuaSH.
            Hide
            krughoff Simon Krughoff added a comment -

            It turns out to test this in CI, we need a repository with coadds. Adding that to the current CI is beyond the scope of this ticket. This triggers DM-14332 to add a coadd dataset and to add a test in CI for this script.

            As such, I'm going to merge.

            Show
            krughoff Simon Krughoff added a comment - It turns out to test this in CI, we need a repository with coadds. Adding that to the current CI is beyond the scope of this ticket. This triggers DM-14332 to add a coadd dataset and to add a test in CI for this script. As such, I'm going to merge.
            Hide
            krughoff Simon Krughoff added a comment -

            Merged. This has been tested by both Hsin-Fang Chiang and me.

            Show
            krughoff Simon Krughoff added a comment - Merged. This has been tested by both Hsin-Fang Chiang and me.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            FWIW, running matchedVisitMetrics.py should not require coadd products, or at least until DM-13839 or other new tickets require it. Michael Wood-Vasey would know more about the current status.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - FWIW, running matchedVisitMetrics.py should not require coadd products, or at least until DM-13839 or other new tickets require it. Michael Wood-Vasey would know more about the current status.
            Hide
            krughoff Simon Krughoff added a comment -

            It is looking for a skyMap and not finding one. I don't know exactly why.

            Show
            krughoff Simon Krughoff added a comment - It is looking for a skyMap and not finding one. I don't know exactly why.

              People

              Assignee:
              krughoff Simon Krughoff
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Angelo Fausti, Hsin-Fang Chiang, John Parejko, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.