Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16942

Create a dashboard for validate_drp results on the HSC reprocessing in Chronograf

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • squash
    • None

    Description

      Need to include metadata from the execution environment in the InfluxDB data model. In particular data from the HSC re-processing at NCSA is not being inserted in InfluxDB after DM-16660.

      Attachments

        Issue Links

          Activity

            Just noting that this ticket (incorporating the results from the RC2 datasets) is blocking SQuaSH's full usefulness to DRP.  

            yusra Yusra AlSayyad added a comment - Just noting that this ticket (incorporating the results from the RC2 datasets) is blocking SQuaSH's full usefulness to DRP.  

            Hi yusra thanks for pointing this out. I'll be working on this right after DM-18103.

            afausti Angelo Fausti added a comment - Hi yusra thanks for pointing this out. I'll be working on this right after DM-18103 .

            Hi yusra, hchiang2 and krughoff you can see a preview of the DRP dashboard for the HSC reprocessing at https://chronograf-demo.lsst.codes

            Once you logged in into Chronograf, click on the "DRP metrics from the HSC reprocessing".

            Make sure you select a time window that is long enough to display the data. For this dashboard, I would go to "Date Picker" and select "Past Year".

            Now that the data model includes metadata from the execution environment at LDF, we can display things like the RUN ID and VERSION on the table. I've attached a screenshot to this ticket as well.

            Note that the queries still point to the squash-demo database but soon I'll change that to squash-prod and the plots will update as new data is sent to SQuaSH.

            krughoff if we add metadata in valide_drp to identify the tracts hchiang2 will be able to send results for all the tracts that are being processed and with Chronograf it will be easy to monitor multiple tracts on the same plot or add another widget to select a specific tract.

            afausti Angelo Fausti added a comment - Hi yusra , hchiang2 and krughoff you can see a preview of the DRP dashboard for the HSC reprocessing at https://chronograf-demo.lsst.codes Once you logged in into Chronograf, click on the "DRP metrics from the HSC reprocessing". Make sure you select a time window that is long enough to display the data. For this dashboard, I would go to "Date Picker" and select "Past Year". Now that the data model includes metadata from the execution environment at LDF, we can display things like the RUN ID and VERSION on the table. I've attached a screenshot to this ticket as well. Note that the queries still point to the squash-demo database but soon I'll change that to squash-prod and the plots will update as new data is sent to SQuaSH. krughoff if we add metadata in valide_drp to identify the tracts hchiang2 will be able to send results for all the tracts that are being processed and with Chronograf it will be easy to monitor multiple tracts on the same plot or add another widget to select a specific tract.

            DM-18529 will add the necessary metadata to monitor multiple tracts and then we can ask hchiang2 to send results for all tracts to SQuaSH.

            afausti Angelo Fausti added a comment - DM-18529 will add the necessary metadata to monitor multiple tracts and then we can ask hchiang2 to send results for all tracts to SQuaSH.

            People

              afausti Angelo Fausti
              afausti Angelo Fausti
              Angelo Fausti, Simon Krughoff (Inactive), Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.