Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-34845

Create ap_verify dataset for DC2

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • ap_verify
    • None
    • 10
    • AP F22-5 (October), AP F22-6 (November)
    • Alert Production
    • No

    Description

      It would be useful to be able to run performance tests in an easily portable manner (and to eventually run DC2 in CI). As a first step, take the data chosen in DM-34824 and package it into a ap_verify Dataset.

      Attachments

        Issue Links

          Activity

            Parejkoj John Parejko added a comment - - edited

            Redoing the `butler create` without `standalone` does result in a working repo. I've also fixed some other errors (apparently I hadn't run it on a machine other than rubin-dev: some of the repo files were `direct` transferred instead of `copy`). I'll post on middleware-support about what `standalone` is supposed to do.

            nima: the dataset repo should be working now. There are 4 PRs to review, and to process the data you'll need to checkout this ticket branch of ap_verify (as I noted above).

            Parejkoj John Parejko added a comment - - edited Redoing the `butler create` without `standalone` does result in a working repo. I've also fixed some other errors (apparently I hadn't run it on a machine other than rubin-dev: some of the repo files were `direct` transferred instead of `copy`). I'll post on middleware-support about what `standalone` is supposed to do. nima : the dataset repo should be working now. There are 4 PRs to review, and to process the data you'll need to checkout this ticket branch of ap_verify (as I noted above).
            krzys Krzysztof Findeisen added a comment - - edited

            In that case, one more comment and I'll stop hijacking nima's review: please update https://github.com/lsst-dm/jenkins-dm-jobs/blob/main/etc/scipipe/ap_verify.yaml to include the new dataset, per sullivan's 21 June comment.

            krzys Krzysztof Findeisen added a comment - - edited In that case, one more comment and I'll stop hijacking nima 's review: please update https://github.com/lsst-dm/jenkins-dm-jobs/blob/main/etc/scipipe/ap_verify.yaml to include the new dataset, per sullivan 's 21 June comment.

            I can confirm that the latest changes fixed the above error. ap_verify now runs with the accompanying dc2 CI dataset.

            Getting back to code review.

            nima Nima Sedaghat added a comment - I can confirm that the latest changes fixed the above error. ap_verify now runs with the accompanying dc2 CI dataset. Getting back to code review.

            On the pull request, there is a small comment from me, and one from krzys. I think after those are addressed, we can go ahead and close this ticket.

            nima Nima Sedaghat added a comment - On the pull request, there is a small comment from me, and one from krzys . I think after those are addressed, we can go ahead and close this ticket.
            Parejkoj John Parejko added a comment - - edited

            Assuming the docs and Jenkins CI (with the merged branches) runs tonight go through successfully, this should be ready to go!

            Parejkoj John Parejko added a comment - - edited Assuming the docs and Jenkins CI (with the merged branches) runs tonight go through successfully, this should be ready to go!

            People

              Parejkoj John Parejko
              ebellm Eric Bellm
              Nima Sedaghat
              Eric Bellm, Ian Sullivan, John Parejko, Krzysztof Findeisen, Meredith Rawls, Nima Sedaghat, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.