Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31381

No connection called refDataset in CatalogMeasurementBaseConnections

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: faro
    • Labels:
      None
    • Team:
      DM Science
    • Urgent?:
      No

      Description

      There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
      and therefore faro doesn't run on w32:

      Traceback in:
      /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

      How to reproduce:

      pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/runs/test-med-1/w_2021_32/DM-31348  -o u/yourUsername/DC2/debug_w30  -p $OBS_FARO_DIR/pipelines/metrics_pipeline_tract.yaml

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31382 Run faro in ci_hsc and ci_imsim

          • Done
          is triggering

          Bug - A problem which impairs or prevents the functions of the product. DM-32069 Ensure that real-time log changes are upstreamed to pilot v3

          • Done
          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            yusra Yusra AlSayyad created issue -
            yusra Yusra AlSayyad made changes -
            Field Original Value New Value
            Rank Ranked higher
            yusra Yusra AlSayyad made changes -
            Link This issue blocks DM-31382 [ DM-31382 ]
            kbechtol Keith Bechtol made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            Permalink
            kbechtol Keith Bechtol added a comment -

            See the following PR:
            https://github.com/lsst/faro/pull/99

            Show
            kbechtol Keith Bechtol added a comment - See the following PR: https://github.com/lsst/faro/pull/99
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment - - edited

            Thanks Keith:

            Brock Brendal [X] Can you try running faro again with this branch a try on your w_2021_32 DC2 rerun? You can do that by setting up the stack, then checking out this ticket branch on faro like:

            cd faro
            		 
            git checkout tickets/DM-31381
            		 
            setup -j -r . 
            scons

            Show
            yusra Yusra AlSayyad added a comment - - edited Thanks Keith: Brock Brendal [X] Can you try running faro again with this branch a try on your w_2021_32 DC2 rerun? You can do that by setting up the stack, then checking out this ticket branch on faro like: cd faro git checkout tickets/DM-31381 setup -j -r . scons
            kbechtol Keith Bechtol made changes -
            Component/s faro [ 19203 ]
            Hide
            Permalink
            kbechtol Keith Bechtol added a comment - - edited

            I revised the PR about 20 minutes ago. I realized that I misinterpreted the problem at first.

            I have re-run the command posted by Yusra to reproduce the error and can confirm that the original error does not appear with this branch. Now I get a `RuntimeError: QuantumGraph is empty.` message instead.

            The passing Jenkins job is here:
            https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34754/pipeline

            Show
            kbechtol Keith Bechtol added a comment - - edited I revised the PR about 20 minutes ago. I realized that I misinterpreted the problem at first. I have re-run the command posted by Yusra to reproduce the error and can confirm that the original error does not appear with this branch. Now I get a `RuntimeError: QuantumGraph is empty.` message instead. The passing Jenkins job is here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34754/pipeline
            yusra Yusra AlSayyad made changes -
            Description There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
            and therefore faro doesn't run on w32:

            Traceback in:
            /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

            How to reproduce:
            {code}
            pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/defaults/test-med-1 --output 2.2i/runs/test-med-1/w_2021_32/DM-31348 -o u/yourUsername/DC2/debug_w30 -p /software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/faro/22.0.1-27-ga9ad9bc+382eb65532/pipelines/metrics_pipeline_tract.yaml
            {code}             		There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
            and therefore faro doesn't run on w32:

            Traceback in:
            /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

            How to reproduce:
            {code}
            pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/defaults/test-med-1 -o u/yourUsername/DC2/debug_w30 -p /software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/faro/22.0.1-27-ga9ad9bc+382eb65532/pipelines/metrics_pipeline_tract.yaml
            {code}
            yusra Yusra AlSayyad made changes -
            Description There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
            and therefore faro doesn't run on w32:

            Traceback in:
            /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

            How to reproduce:
            {code}
            pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/defaults/test-med-1 -o u/yourUsername/DC2/debug_w30 -p /software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/faro/22.0.1-27-ga9ad9bc+382eb65532/pipelines/metrics_pipeline_tract.yaml
            {code}             		There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
            and therefore faro doesn't run on w32:

            Traceback in:
            /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

            How to reproduce:
            {code}
            pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/defaults/test-med-1 -o u/yourUsername/DC2/debug_w30 -p $OBS_FARO_DIR/pipelines/metrics_pipeline_tract.yaml
            {code}
            yusra Yusra AlSayyad made changes -
            Description There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
            and therefore faro doesn't run on w32:

            Traceback in:
            /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

            How to reproduce:
            {code}
            pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/defaults/test-med-1 -o u/yourUsername/DC2/debug_w30 -p $OBS_FARO_DIR/pipelines/metrics_pipeline_tract.yaml
            {code}             		There's no connection called 'refDataset' in CatalogMeasurementBaseConnections
            and therefore faro doesn't run on w32:

            Traceback in:
            /scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_32/DM-31348/20210811T151210Z/quantumGraphGeneration.out

            How to reproduce:
            {code}
            pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/runs/test-med-1/w_2021_32/DM-31348 -o u/yourUsername/DC2/debug_w30 -p $OBS_FARO_DIR/pipelines/metrics_pipeline_tract.yaml
            {code}
            Hide
            Permalink
            kbechtol Keith Bechtol added a comment - - edited

            Verified that the following command runs to completion.

            ```pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/runs/test-med-1/w_2021_32/DM-31348 -o u/kbechtol/DC2/debug_w30 -p $FARO_DIR/pipelines/metrics_pipeline_tract.yaml```

            Merged ticket to master branch of faro. The PR had previously be reviewed by Jeffrey Carlin.

            Show
            kbechtol Keith Bechtol added a comment - - edited Verified that the following command runs to completion. ```pipetask run -d "instrument = 'LSSTCam-imSim' and tract in (3828) and skymap='DC2'" -b /repo/dc2/butler.yaml -i 2.2i/runs/test-med-1/w_2021_32/ DM-31348 -o u/kbechtol/DC2/debug_w30 -p $FARO_DIR/pipelines/metrics_pipeline_tract.yaml``` Merged ticket to master branch of faro. The PR had previously be reviewed by Jeffrey Carlin .
            kbechtol Keith Bechtol made changes -
            Reviewers Dan Taranu [ dtaranu ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            tjenness Tim Jenness made changes -
            Team DM Science [ 12218 ]
            Hide
            Permalink
            dtaranu Dan Taranu added a comment -

            I believe Brock Brendal [X] has successfully resubmitted jobs with the branch.

            Yusra AlSayyad filed DM-31382 to catch potential future regressions before they're merged.

            Show
            dtaranu Dan Taranu added a comment - I believe Brock Brendal [X] has successfully resubmitted jobs with the branch. Yusra AlSayyad filed DM-31382 to catch potential future regressions before they're merged.
            dtaranu Dan Taranu made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            kbechtol Keith Bechtol made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            jcarlin Jeffrey Carlin made changes -
            Remote Link This issue links to "Page (Confluence)" [ 29475 ]
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-32069 [ DM-32069 ]

              People

              Assignee:
              kbechtol Keith Bechtol
              Reporter:
              yusra Yusra AlSayyad
              Reviewers:
              Dan Taranu
              Watchers:
              Dan Taranu, Keith Bechtol, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.