Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-6196

SQuaSH capability extension: multiple testdata service

    XMLWordPrintable

    Details

    • Epic Name:
      sqre-f16-squash-1
    • Story Points:
      92
    • WBS:
      1.02C.10.02
    • Team:
      SQuaRE
    • Cycle:
      Fall 2016

      Description

      This epic covers work to deliver the following improvements to the SQuaSH prototype stood up in X16:

      • drilldown 1 level (time series->histogram)
      • multiple testdata options (requires jenkins, backend, dashboard extension)
      • processccd + validate_drp pseudo-workflow
      • pseudo-provenance (track manifest.txt - real LSST provenance system will be swapped in for extensive functionality when available)

        Attachments

          Issue Links

          Stories in Epic (Custom Issue Matrix)

          Key Summary Story Points Assignee Status
           
          DM-5945

          Implement validate_drp static plots in Bokeh as proof-of-concept for SQUASH

          10 Angelo Fausti Done
           
          DM-8561

          Filter null measurements in post-qa

          1 Jonathan Sick Done
           
          DM-8414

          Investigate alternatives to ingest JSON blobs into the SQUASH database

          4 Angelo Fausti Done
           
          DM-8195

          deploy squash code changes feature

          1.5 Joshua Hoblitt Done
           
          DM-7612

          Separate the presentation layer from API access in the metrics app

          2 Angelo Fausti Done
           
          DM-7581

          Use box annotations to indicate missing jobs

          2.5 Angelo Fausti Done
           
          DM-7441

          Manage multiple bokeh apps

          5 Angelo Fausti Done
           
          DM-7216

          deploy squash bokeh driven views

          3 Joshua Hoblitt Done
           
          DM-7098

          Add two more env variables from jenkins in the JSON gerenated by post_qa

          0.5 Angelo Fausti Done
           
          DM-7082

          deploy django admin interface fix

          0.75 Joshua Hoblitt Done
           
          DM-8573

          Cleanup validate_base Datum input value handling

          0.1 Jonathan Sick Done
           
          DM-8501

          Update SQUASH design documentation for F16

          1 Angelo Fausti Done
           
          DM-7749

          Update squash production environment

          1 Joshua Hoblitt Done
           
          DM-7517

          Fix ordering when returning jobs from the api

          1 Angelo Fausti Done
           
          DM-7328

          Port validate_drp to Python 3

          0.5 Jonathan Sick Done
           
          DM-6992

          Use Bokeh views entirely for dashboard

          10 Angelo Fausti Done
           
          DM-6629

          validate_drp: design and implement an API for metric measurements and serializations

          22 Jonathan Sick Done
           
          DM-8557

          SWIG in Python 3 does not accept numpy.int64 types as an index

          0.6 Jonathan Sick Done
           
          DM-8558

          Make metrics.yaml installable with validate_drp

          0.2 Jonathan Sick Done
           
          DM-8398

          deploy squash improvements in code changes feature

          0.333 Joshua Hoblitt Done
           
          DM-8368

          Improvements on code change feature

          2 Angelo Fausti Done
           
          DM-7737

          Port of validate_base to Python 3

          0.7 Jonathan Sick Done
           
          DM-7682

          Create a new fixture for loading test data

          1.5 Angelo Fausti Done
           
          DM-7427

          Improve display of metric thresholds

          4 Angelo Fausti Done
           
          DM-7071

          Fix Django admin interface

          1 Angelo Fausti Done
           
          DM-7005

          List the code changes from job to job and link the package names with the corresponding git url

          20 Angelo Fausti Done
           
          DM-7006

          Update squash to use bokeh 0.12.1

          1 Angelo Fausti Done
           
          DM-6917

          Write User Guide for new validate_drp metric/measurement API

          4.2 Jonathan Sick Done

            Activity

            frossie Frossie Economou created issue -
            frossie Frossie Economou made changes -
            Field Original Value New Value
            Cycle Fall 2016 [ 10803 ]
            Description
            This epic covers work to deliver the following improvements to the SQuaSH prototype stood up in X16:

            - drilldown 1 level (time series->histogram)

            - multiple testdata options (requires jenkins, backend, dashboard extension)

            - processccd + validate_drp pseudo-workflow

            - pseudo-provenance (track manifest.txt - real LSST provenance system will be swapped in for extensive functionality when available)
            This epic covers work to deliver the following improvements to the SQuaSH prototype stood up in X16:

            - drilldown 1 level (time series->histogram)

            - multiple testdata options (requires jenkins, backend, dashboard extension)

            - processccd + validate_drp pseudo-workflow

            - pseudo-provenance (track manifest.txt - real LSST provenance system will be swapped in for extensive functionality when available)
            frossie Frossie Economou made changes -
            Story Points 92
            frossie Frossie Economou made changes -
            WBS 1.02C.10.02
            jhoblitt Joshua Hoblitt made changes -
            Epic Child DM-6328 [ 24955 ]
            jhoblitt Joshua Hoblitt made changes -
            Epic Child DM-6330 [ 24957 ]
            jhoblitt Joshua Hoblitt made changes -
            Epic Child DM-6103 [ 24450 ]
            jhoblitt Joshua Hoblitt made changes -
            Epic Child DM-6104 [ 24451 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-5945 [ 24159 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-6374 [ 25044 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-6629 [ 25468 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-6917 [ 25804 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-6990 [ 25979 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-6992 [ 25981 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7005 [ 25996 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7006 [ 25997 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7041 [ 26048 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7042 [ 26049 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7043 [ 26050 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7071 [ 26085 ]
            jhoblitt Joshua Hoblitt made changes -
            Epic Child DM-7082 [ 26100 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7098 [ 26148 ]
            jhoblitt Joshua Hoblitt made changes -
            Epic Child DM-7216 [ 26297 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7334 [ 26419 ]
            jbecla Jacek Becla made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7427 [ 26524 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7438 [ 26535 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7441 [ 26538 ]
            frossie Frossie Economou made changes -
            Assignee Frossie Economou [ frossie ] Angelo Fausti [ afausti ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7517 [ 26655 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7581 [ 26761 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-6374 [ 25044 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7041 [ 26048 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7042 [ 26049 ]
            frossie Frossie Economou made changes -
            Link This issue is cloned by DM-7605 [ DM-7605 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7612 [ 26798 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7682 [ 26886 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-7749 [ 26971 ]
            afausti Angelo Fausti made changes -
            Comment [
            - finished method to return the list of packages that changed in a given job with respect to the previous one (check diffs in the git commit sha of each package) the measurements api returns the required info for those packages (see above)
            - implemented bokeh data table to present the results (any suggestion here?)
            - still learning how to update the bokeh data table connected with the hover tool, that is possible using a custom JS callback as explained here http://bokeh.pydata.org/en/latest/docs/user_guide/interaction/callbacks.html#customjs-for-hover
            ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8071 [ 27511 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8195 [ 27706 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-8285 [ 27807 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8368 [ 27977 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8398 [ 28009 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8414 [ 28025 ]
            frossie Frossie Economou made changes -
            Epic Child DM-6104 [ 24451 ]
            frossie Frossie Economou made changes -
            Epic Child DM-6103 [ 24450 ]
            frossie Frossie Economou made changes -
            Epic Child DM-8071 [ 27511 ]
            frossie Frossie Economou made changes -
            Epic Child DM-7043 [ 26050 ]
            frossie Frossie Economou made changes -
            Epic Child DM-6328 [ 24955 ]
            frossie Frossie Economou made changes -
            Epic Child DM-8285 [ 27807 ]
            frossie Frossie Economou made changes -
            Epic Child DM-7438 [ 26535 ]
            frossie Frossie Economou made changes -
            Epic Child DM-7334 [ 26419 ]
            frossie Frossie Economou made changes -
            Epic Child DM-6330 [ 24957 ]
            frossie Frossie Economou made changes -
            Epic Child DM-6990 [ 25979 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8501 [ 28140 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-8557 [ 28207 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7328 [ 26413 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-7737 [ 26959 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-8558 [ 28208 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-8561 [ 28214 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-8573 [ 28226 ]
            jsick Jonathan Sick made changes -
            Epic Child DM-8711 [ 28501 ]
            afausti Angelo Fausti made changes -
            Epic Child DM-8711 [ 28501 ]
            afausti Angelo Fausti made changes -
            Reviewers Frossie Economou [ frossie ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            frossie Frossie Economou made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            frossie Frossie Economou made changes -
            Attachment Safari.png [ 28922 ]
            frossie Frossie Economou made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            afausti Angelo Fausti made changes -
            Component/s Developer Infrastructure [ 10712 ]

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              frossie Frossie Economou
              Reviewers:
              Frossie Economou
              Watchers:
              Angelo Fausti, Frossie Economou
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Summary Panel