Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5943

Add Git refs to Jobs Table of QA Dashboard

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      The Level 0 QA DB should know what Stack Git refs correspond to each job. This will enable plots to filter jobs based on development ticket so that a developer can understand how a branch compares to master.

      This ticket will add jobs to the schema (http://sqr-009.lsst.io/en/latest/#level-0-qa) and create the necessary migration script.

        Attachments

          Issue Links

            Activity

            Hide
            jsick Jonathan Sick added a comment -

            Some effort on getting up to speed with the QA project is included in this ticket.

            Show
            jsick Jonathan Sick added a comment - Some effort on getting up to speed with the QA project is included in this ticket.
            Hide
            jsick Jonathan Sick added a comment -

            Angelo Fausti: I’m going to include some test data and a bootstrap script with this ticket.

            Show
            jsick Jonathan Sick added a comment - Angelo Fausti : I’m going to include some test data and a bootstrap script with this ticket.
            Hide
            jsick Jonathan Sick added a comment -

            Repeated from a git commit:

            Note that originally I attempted to re-factor the VersionedPackage table into both a Package table and a Package Version table. The Package model has records for every EUPS package in a stack. The PackageVersion has records for the version of a package seen in a job. Thus if there are N packages, there will be N rows in the PackageVersion table for every build. This was implemented as a many-to-many relationship between Jobs and Packages with the PackageVersion table as an intermediary. Unfortunately it was difficult (impossible?) to make this work with the Django REST Framework as a nested serialization.

            I probably wasted 2.5 days of effort struggling to get Django REST Framework to do the right thing. Ultimately I couldn’t. I wouldn’t advocate adopting this technology for future SQuaRE REST APIs without good justification.

            Show
            jsick Jonathan Sick added a comment - Repeated from a git commit: Note that originally I attempted to re-factor the VersionedPackage table into both a Package table and a Package Version table. The Package model has records for every EUPS package in a stack. The PackageVersion has records for the version of a package seen in a job. Thus if there are N packages, there will be N rows in the PackageVersion table for every build. This was implemented as a many-to-many relationship between Jobs and Packages with the PackageVersion table as an intermediary. Unfortunately it was difficult (impossible?) to make this work with the Django REST Framework as a nested serialization. I probably wasted 2.5 days of effort struggling to get Django REST Framework to do the right thing. Ultimately I couldn’t. I wouldn’t advocate adopting this technology for future SQuaRE REST APIs without good justification.
            Hide
            jsick Jonathan Sick added a comment -

            See pull request. In the end; a pretty simple implementation.

            Show
            jsick Jonathan Sick added a comment - See pull request. In the end; a pretty simple implementation.
            Hide
            tjenness Tim Jenness added a comment -

            This seems to be somewhat related to RFC-169.

            Show
            tjenness Tim Jenness added a comment - This seems to be somewhat related to RFC-169 .
            Hide
            jsick Jonathan Sick added a comment - - edited

            Tim Jenness: Our plan is to get version information directly from the lsstsw CI environment and then persist it into the Dashboard's DB in order to associate software metadata to QA results in real-time on the dashboard.

            Further discussion about integrating the dashboard in DM provenance work should happen on the thread https://community.lsst.org/t/tying-software-provenance-into-the-qa-system/762?u=jsick

            After my inquiry it seemed that nothing in HSC/RFC-169 was useful, and that it'll be more expediant to ship this as an MVP instead until the actual provenance API is available.

            Show
            jsick Jonathan Sick added a comment - - edited Tim Jenness : Our plan is to get version information directly from the lsstsw CI environment and then persist it into the Dashboard's DB in order to associate software metadata to QA results in real-time on the dashboard. Further discussion about integrating the dashboard in DM provenance work should happen on the thread https://community.lsst.org/t/tying-software-provenance-into-the-qa-system/762?u=jsick After my inquiry it seemed that nothing in HSC/ RFC-169 was useful, and that it'll be more expediant to ship this as an MVP instead until the actual provenance API is available.
            Hide
            afausti Angelo Fausti added a comment -

            Reviewed and successfully tested a job post request using measurements and packages, more details in the PR comments https://github.com/lsst-sqre/qa-dashboard/pull/4

            Show
            afausti Angelo Fausti added a comment - Reviewed and successfully tested a job post request using measurements and packages, more details in the PR comments https://github.com/lsst-sqre/qa-dashboard/pull/4

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              jsick Jonathan Sick
              Reviewers:
              Angelo Fausti, J Matt Peterson [X] (Inactive)
              Watchers:
              Angelo Fausti, J Matt Peterson [X] (Inactive), Jonathan Sick, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.