Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16942

Create a dashboard for validate_drp results on the HSC reprocessing in Chronograf

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None

      Description

      Need to include metadata from the execution environment in the InfluxDB data model. In particular data from the HSC re-processing at NCSA is not being inserted in InfluxDB after DM-16660.

        Attachments

          Issue Links

            Activity

            Hide
            yusra Yusra AlSayyad added a comment -

            Just noting that this ticket (incorporating the results from the RC2 datasets) is blocking SQuaSH's full usefulness to DRP.  

            Show
            yusra Yusra AlSayyad added a comment - Just noting that this ticket (incorporating the results from the RC2 datasets) is blocking SQuaSH's full usefulness to DRP.  
            Hide
            afausti Angelo Fausti added a comment -

            Hi Yusra AlSayyad thanks for pointing this out. I'll be working on this right after DM-18103.

            Show
            afausti Angelo Fausti added a comment - Hi Yusra AlSayyad thanks for pointing this out. I'll be working on this right after DM-18103 .
            Hide
            afausti Angelo Fausti added a comment -

            Hi Yusra AlSayyad, Hsin-Fang Chiang and Simon Krughoff you can see a preview of the DRP dashboard for the HSC reprocessing at https://chronograf-demo.lsst.codes

            Once you logged in into Chronograf, click on the "DRP metrics from the HSC reprocessing".

            Make sure you select a time window that is long enough to display the data. For this dashboard, I would go to "Date Picker" and select "Past Year".

            Now that the data model includes metadata from the execution environment at LDF, we can display things like the RUN ID and VERSION on the table. I've attached a screenshot to this ticket as well.

            Note that the queries still point to the squash-demo database but soon I'll change that to squash-prod and the plots will update as new data is sent to SQuaSH.

            Simon Krughoff if we add metadata in valide_drp to identify the tracts Hsin-Fang Chiang will be able to send results for all the tracts that are being processed and with Chronograf it will be easy to monitor multiple tracts on the same plot or add another widget to select a specific tract.

            Show
            afausti Angelo Fausti added a comment - Hi Yusra AlSayyad , Hsin-Fang Chiang and Simon Krughoff you can see a preview of the DRP dashboard for the HSC reprocessing at https://chronograf-demo.lsst.codes Once you logged in into Chronograf, click on the "DRP metrics from the HSC reprocessing". Make sure you select a time window that is long enough to display the data. For this dashboard, I would go to "Date Picker" and select "Past Year". Now that the data model includes metadata from the execution environment at LDF, we can display things like the RUN ID and VERSION on the table. I've attached a screenshot to this ticket as well. Note that the queries still point to the squash-demo database but soon I'll change that to squash-prod and the plots will update as new data is sent to SQuaSH. Simon Krughoff if we add metadata in valide_drp to identify the tracts Hsin-Fang Chiang will be able to send results for all the tracts that are being processed and with Chronograf it will be easy to monitor multiple tracts on the same plot or add another widget to select a specific tract.
            Hide
            afausti Angelo Fausti added a comment -

            DM-18529 will add the necessary metadata to monitor multiple tracts and then we can ask Hsin-Fang Chiang to send results for all tracts to SQuaSH.

            Show
            afausti Angelo Fausti added a comment - DM-18529 will add the necessary metadata to monitor multiple tracts and then we can ask Hsin-Fang Chiang to send results for all tracts to SQuaSH.

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti, Simon Krughoff, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.