Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-34845

Create ap_verify dataset for DC2

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      10
    • Sprint:
      AP F22-5 (October), AP F22-6 (November)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      It would be useful to be able to run performance tests in an easily portable manner (and to eventually run DC2 in CI). As a first step, take the data chosen in DM-34824 and package it into a ap_verify Dataset.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment - - edited

            Redoing the `butler create` without `standalone` does result in a working repo. I've also fixed some other errors (apparently I hadn't run it on a machine other than rubin-dev: some of the repo files were `direct` transferred instead of `copy`). I'll post on middleware-support about what `standalone` is supposed to do.

            Nima Sedaghat: the dataset repo should be working now. There are 4 PRs to review, and to process the data you'll need to checkout this ticket branch of ap_verify (as I noted above).

            Show
            Parejkoj John Parejko added a comment - - edited Redoing the `butler create` without `standalone` does result in a working repo. I've also fixed some other errors (apparently I hadn't run it on a machine other than rubin-dev: some of the repo files were `direct` transferred instead of `copy`). I'll post on middleware-support about what `standalone` is supposed to do. Nima Sedaghat : the dataset repo should be working now. There are 4 PRs to review, and to process the data you'll need to checkout this ticket branch of ap_verify (as I noted above).
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            In that case, one more comment and I'll stop hijacking Nima Sedaghat's review: please update https://github.com/lsst-dm/jenkins-dm-jobs/blob/main/etc/scipipe/ap_verify.yaml to include the new dataset, per Ian Sullivan's 21 June comment.

            Show
            krzys Krzysztof Findeisen added a comment - - edited In that case, one more comment and I'll stop hijacking Nima Sedaghat 's review: please update https://github.com/lsst-dm/jenkins-dm-jobs/blob/main/etc/scipipe/ap_verify.yaml to include the new dataset, per Ian Sullivan 's 21 June comment.
            Hide
            nima Nima Sedaghat added a comment -

            I can confirm that the latest changes fixed the above error. ap_verify now runs with the accompanying dc2 CI dataset.

            Getting back to code review.

            Show
            nima Nima Sedaghat added a comment - I can confirm that the latest changes fixed the above error. ap_verify now runs with the accompanying dc2 CI dataset. Getting back to code review.
            Hide
            nima Nima Sedaghat added a comment -

            On the pull request, there is a small comment from me, and one from Krzysztof Findeisen. I think after those are addressed, we can go ahead and close this ticket.

            Show
            nima Nima Sedaghat added a comment - On the pull request, there is a small comment from me, and one from Krzysztof Findeisen . I think after those are addressed, we can go ahead and close this ticket.
            Hide
            Parejkoj John Parejko added a comment - - edited

            Assuming the docs and Jenkins CI (with the merged branches) runs tonight go through successfully, this should be ready to go!

            Show
            Parejkoj John Parejko added a comment - - edited Assuming the docs and Jenkins CI (with the merged branches) runs tonight go through successfully, this should be ready to go!

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              ebellm Eric Bellm
              Reviewers:
              Nima Sedaghat
              Watchers:
              Eric Bellm, Ian Sullivan, John Parejko, Krzysztof Findeisen, Meredith Rawls, Nima Sedaghat, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.