Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14550

Add new command line option '--ignore-blobs' to dispatch_verify.py

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None

      Description

      For some use cases, for instance, the HSC reprocessing on the LDF environment, we need an option in dispatch_verify.py to skip the upload of data blobs to SQuaSH.

        Attachments

          Issue Links

            Activity

            Hide
            afausti Angelo Fausti added a comment - - edited

            Example of command to test the --ignore-jobs option:

            dispatch_verify.py --url https://squash-restful-api-demo.lsst.codes --user <user> --password <password> --env jenkins --ignore-lsstsw --ignore-blobs test_job.json
            verify.bin.dispatchverify.main INFO: Loading test_job.json
            verify.bin.dispatchverify.main INFO: Ignoring blobs from Job JSON test_job.json
            verify.bin.dispatchverify.main INFO: Refreshing metric definitions from verify_metrics
            verify.bin.dispatchverify.main INFO: Inserting Jenkins CI environment metadata.
            verify.bin.dispatchverify.main INFO: Uploading Job JSON to https://squash-restful-api-demo.lsst.codes.
            verify.squash.get INFO: GET https://squash-restful-api-demo.lsst.codes status: 200
            verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/auth status: 200
            verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/job status: 202
            

            Show
            afausti Angelo Fausti added a comment - - edited Example of command to test the --ignore-jobs option: dispatch_verify.py --url https://squash-restful-api-demo.lsst.codes --user <user> --password <password> --env jenkins --ignore-lsstsw --ignore-blobs test_job.json verify.bin.dispatchverify.main INFO: Loading test_job.json verify.bin.dispatchverify.main INFO: Ignoring blobs from Job JSON test_job.json verify.bin.dispatchverify.main INFO: Refreshing metric definitions from verify_metrics verify.bin.dispatchverify.main INFO: Inserting Jenkins CI environment metadata. verify.bin.dispatchverify.main INFO: Uploading Job JSON to https://squash-restful-api-demo.lsst.codes. verify.squash.get INFO: GET https://squash-restful-api-demo.lsst.codes status: 200 verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/auth status: 200 verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/job status: 202
            Hide
            afausti Angelo Fausti added a comment -

            In combination with DM-14357:

            dispatch_verify.py --url https://squash-restful-api-demo.lsst.codes --user <user> --password <password> --env ldf --ignore-blobs Cfht_output_r.json
            verify.bin.dispatchverify.main INFO: Loading Cfht_output_r.json
            verify.bin.dispatchverify.main INFO: Ignoring blobs from Job JSON Cfht_output_r.json
            verify.bin.dispatchverify.main INFO: Refreshing metric definitions from verify_metrics
            verify.bin.dispatchverify.main INFO: Inserting LSST Data Facility environment metadata.
            verify.bin.dispatchverify.main INFO: Uploading Job JSON to https://squash-restful-api-demo.lsst.codes.
            verify.squash.get INFO: GET https://squash-restful-api-demo.lsst.codes status: 200
            verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/auth status: 200
            verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/job status: 202
            

            Show
            afausti Angelo Fausti added a comment - In combination with DM-14357 : dispatch_verify.py --url https://squash-restful-api-demo.lsst.codes --user <user> --password <password> --env ldf --ignore-blobs Cfht_output_r.json verify.bin.dispatchverify.main INFO: Loading Cfht_output_r.json verify.bin.dispatchverify.main INFO: Ignoring blobs from Job JSON Cfht_output_r.json verify.bin.dispatchverify.main INFO: Refreshing metric definitions from verify_metrics verify.bin.dispatchverify.main INFO: Inserting LSST Data Facility environment metadata. verify.bin.dispatchverify.main INFO: Uploading Job JSON to https://squash-restful-api-demo.lsst.codes. verify.squash.get INFO: GET https://squash-restful-api-demo.lsst.codes status: 200 verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/auth status: 200 verify.squash.post INFO: POST https://squash-restful-api-demo.lsst.codes/job status: 202
            Hide
            afausti Angelo Fausti added a comment -
            Show
            afausti Angelo Fausti added a comment - See PR https://github.com/lsst/verify/pull/26

              People

              • Assignee:
                afausti Angelo Fausti
                Reporter:
                afausti Angelo Fausti
                Watchers:
                Angelo Fausti
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel