Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21916

SQuaSH upload of Gen 3 Measurements

    XMLWordPrintable

    Details

      Description

      The existing upload script (dispatch_verify.py) assumes that metric values have been persisted in Job objects. Once we can store metric values in Gen 3 repositories, we need to be able to read those as part of the upload process.

      Story point value assumes that using Gen 3 Butler inside something that's not a PipelineTask is difficult.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21888 Stand up Gen3 version of ap_verify HiTS (2015) CI test

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-28888 Allow SQuaSH uploads from ap_verify Gen 3

          • Done
          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21875 Add StorageClass and Formatter support necessary to persist lsst.verify.Measurement in Gen3 repos

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-28757 Factor upload format out of lsst.verify.Job

          • To Do

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21885 Convert concrete MetricTasks to PipelineTasks

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-25507 Tooling to report to SQuaSH

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-28351 Add faro to lsst_distrib

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24262 Run HSC AP processing in CI using Gen 3

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-12549 ap_pipe must call AssociationTask in a reproducible order

          • Done
          mentioned in

          Page Loading...

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            11 older comments
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment - - edited

            Simon Krughoff, I apparently don't have permissions to push ticket branches to lsst-dmsst/metric-pipeline-tasks. What's the SST procedure for dealing with this? Should I fork?

            For reference, I'm trying to follow https://developer.lsst.io/stack/transferring-code.html.

            Show
            krzys Krzysztof Findeisen added a comment - - edited Simon Krughoff , I apparently don't have permissions to push ticket branches to lsst-dmsst/metric-pipeline-tasks . What's the SST procedure for dealing with this? Should I fork? For reference, I'm trying to follow https://developer.lsst.io/stack/transferring-code.html .
            Hide
            Permalink
            krughoff Simon Krughoff added a comment -

            Sure, forking seems reasonable. FWIW, it's going to move to the lsst org in the next week or so, if you'd prefer to just wait.

            Show
            krughoff Simon Krughoff added a comment - Sure, forking seems reasonable. FWIW, it's going to move to the lsst org in the next week or so, if you'd prefer to just wait.
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            Thanks for the tip, but given that rebasing the double-branches is excruciatingly painful, I'd like to minimize the number of changes I need to account for when I do so. I'll open PRs as soon as Jenkins finishes.

            Show
            krzys Krzysztof Findeisen added a comment - Thanks for the tip, but given that rebasing the double-branches is excruciatingly painful, I'd like to minimize the number of changes I need to account for when I do so. I'll open PRs as soon as Jenkins finishes.
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment - - edited

            PRs are verify#77 and metric-pipeline-tasks#66.

            Show
            krzys Krzysztof Findeisen added a comment - - edited PRs are verify#77 and metric-pipeline-tasks#66 .
            Hide
            Permalink
            krughoff Simon Krughoff added a comment -

            Looks great. I have not tested it explicitly, but since the interfaces are all preserved in the mertric_pipeline_tasks repository, I don't see any reason why these changes would cause problems

            Show
            krughoff Simon Krughoff added a comment - Looks great. I have not tested it explicitly, but since the interfaces are all preserved in the mertric_pipeline_tasks repository, I don't see any reason why these changes would cause problems

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Simon Krughoff
              Watchers:
              Angelo Fausti, Eric Bellm, Ian Sullivan, Jim Bosch, Krzysztof Findeisen, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.