Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9829

validate_drp broken on decam dataset as of 03/15

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      A test run for the decam dataset succeed yesterday morning but the periodic evening build failed with the stack trace below. Perhaps this is related to the daf_persistence changes yesterday?

      https://ci.lsst.codes/job/validate_drp/dataset=decam,label=centos-7,python=py2/863/

      Traceback (most recent call last):
        File "/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gb01c0af08e/bin/validateDrp.py", line 97, in <module>
          validate.run(args.repo, **kwargs)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gb01c0af08e/python/lsst/validate/drp/validate.py", line 104, in run
          **kwargs)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gb01c0af08e/python/lsst/validate/drp/validate.py", line 204, in runOneFilter
          verbose=verbose)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gb01c0af08e/python/lsst/validate/drp/matchreduce.py", line 148, in __init__
          repo, dataIds, matchRadius)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gb01c0af08e/python/lsst/validate/drp/matchreduce.py", line 208, in _loadAndMatchCatalogs
          calexpMetadata = butler.get("calexp_md", vId, immediate=True)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/lsstsw/stack/Linux64/daf_persistence/13.0-4-g886c4ba/python/lsst/daf/persistence/butler.py", line 945, in get
          raise NoResults("No locations for get:", datasetType, dataId)
      lsst.daf.persistence.butlerExceptions.NoResults: No locations for get: datasetType:calexp_md dataId:DataId(initialdata={'filter': 'z', 'visit': 176837, 'ccdnum': 14}, tag=set([]))
      cp: cannot stat ‘/home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/validate_drp/Decam_output_z.json’: No such file or directory
      /home/jenkins-slave/workspace/validate_drp/dataset/decam/label/centos-7/python/py2/archive/decam/processCcd.log: 5,072 B / 93.7 KiB = 0.053
      

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            Trivial changes here: https://github.com/lsst/validate_drp/pull/42

            Is there a way to test by running validate_drp through Jenkins? I'm lost as to which build might actually do that.

            I've "stolen" this ticket for DRP!

            Show
            swinbank John Swinbank added a comment - Trivial changes here: https://github.com/lsst/validate_drp/pull/42 Is there a way to test by running validate_drp through Jenkins? I'm lost as to which build might actually do that. I've "stolen" this ticket for DRP!
            Hide
            swinbank John Swinbank added a comment -

            Michael Wood-Vasey — I note your comment ("This looks good.") on GitHub, but this ticket is still in review. Do you want me to go ahead and merge, or are you intending to say more?

            Show
            swinbank John Swinbank added a comment - Michael Wood-Vasey — I note your comment ("This looks good.") on GitHub, but this ticket is still in review. Do you want me to go ahead and merge, or are you intending to say more?
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            I'm running a test on my local desktop before approving.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - I'm running a test on my local desktop before approving.
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            I confirm that this change fixes the problem on my desktop.

            Merge it.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - I confirm that this change fixes the problem on my desktop. Merge it.
            Hide
            swinbank John Swinbank added a comment -

            Merged. Thanks!

            Shutting this ticket down since the immediate problem is resolved, but I'd still appreciate it if Nate Pease could pitch in with his thoughts.

            Show
            swinbank John Swinbank added a comment - Merged. Thanks! Shutting this ticket down since the immediate problem is resolved, but I'd still appreciate it if Nate Pease could pitch in with his thoughts.

              People

              • Assignee:
                swinbank John Swinbank
                Reporter:
                jhoblitt Joshua Hoblitt
                Reviewers:
                Michael Wood-Vasey
                Watchers:
                John Swinbank, Joshua Hoblitt, Michael Wood-Vasey, Nate Pease
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel