Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-36743

Add the real-bogus PipelineTask to ap_verify for the DC2 CI dataset

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ap_verify
    • Labels:
      None
    • Story Points:
      6
    • Epic Link:
    • Sprint:
      AP F22-6 (November), AP S23-2 (January), AP S23-3 (February), AP S23-5 (April), AP S23-6 (May)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Add the traditional real-bogus classifier PipelineTask to ApVerify.yaml and ApVerifyWithFakes.yaml in the new DC2 CI dataset defined in DM-34845. The real-bogus PipelineTask should be run by default when ap_verify is run with this dataset, on any supported machine.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-36744 Add real-bogus CI output to Chronograf

          • To Do
          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-36742 Add real-bogus trained model to DC2 CI repository

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-38480 Add torchvision to rubin-env

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-38481 Add meas_transiNet to ap_pipe

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-38981 Update the ap_verify tutorial according to recent changes: fetch model packages

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-39127 DM-36743 broke ap_verify

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-37687 Add torchvision to the stack

          • Invalid

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-39369 Add the real-bogus classification task to the ap_verify pipeline for Cosmos and Hits CI datasets

          • Done

            Activity

            Ascending order - Click to sort in descending order
            20 older comments
            Hide
            Permalink
            ebellm Eric Bellm added a comment -

            Nima Sedaghat presumably in a "manual" processing context outside of Jenkins you can adjust the modelPackageName to whatever you'd like, no?

            Show
            ebellm Eric Bellm added a comment - Nima Sedaghat presumably in a "manual" processing context outside of Jenkins you can adjust the modelPackageName to whatever you'd like, no?
            Hide
            Permalink
            nima Nima Sedaghat added a comment - - edited

            Eric Bellm, it is doable, as in editing the .yaml file. However, what gets merged into the main of ap_verify stays there – somehow related to the recent discussion on "parametrization of ap_verify" we had recently.

            Show
            nima Nima Sedaghat added a comment - - edited Eric Bellm , it is doable, as in editing the .yaml file. However, what gets merged into the main of ap_verify stays there – somehow related to the recent discussion on "parametrization of ap_verify" we had recently.
            Hide
            Permalink
            nima Nima Sedaghat added a comment -

            But of course we can fetch the data repository manually – and I think it suffices to do once, per-user at least.

            Show
            nima Nima Sedaghat added a comment - But of course we can fetch the data repository manually – and I think it suffices to do once, per-user at least.
            Hide
            Permalink
            sullivan Ian Sullivan added a comment -

            A new consideration: we should probably hold off on merging anything significant to ap_verify until after CI is working again. This is both because we won't have the benefits of regularly testing the new code until CI is working again, and to not add complications to the process of fixing it. It's probably not that much of an extra complication, though, so I am open to arguments that it is worth merging now anyway.

            Show
            sullivan Ian Sullivan added a comment - A new consideration: we should probably hold off on merging anything significant to ap_verify until after CI is working again. This is both because we won't have the benefits of regularly testing the new code until CI is working again, and to not add complications to the process of fixing it. It's probably not that much of an extra complication, though, so I am open to arguments that it is worth merging now anyway.
            Hide
            Permalink
            nima Nima Sedaghat added a comment - - edited

            I don't know much about this recent CI outage . But what I can add is that, yes, if we merge this, anything trying to touch ap_verify*.yaml (outside of jenkins) is going to fail. So I agree with holding off.

            Show
            nima Nima Sedaghat added a comment - - edited I don't know much about this recent CI outage . But what I can add is that, yes, if we merge this, anything trying to touch ap_verify*.yaml (outside of jenkins) is going to fail. So I agree with holding off.

              People

              Assignee:
              nima Nima Sedaghat
              Reporter:
              sullivan Ian Sullivan
              Reviewers:
              Ian Sullivan
              Watchers:
              Eric Bellm, Ian Sullivan, John Parejko, Krzysztof Findeisen, Nima Sedaghat, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.