Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31519

Improve handling of NaN metric values by dispatch_verify

    XMLWordPrintable

    Details

      Description

      NaN is now disallowed in JSON, which causes problems if trying to dispatch metrics that contain NaNs to SQuaSH. dispatchverify.py (in lsst.verify) needs to be updated to identify NaN metrics and convert them to a JSON-compliant value before dispatching the job. See this Slack thread and subsequent conversation in that channel for context.

        Attachments

          Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. DM-31131 lsst.verify outputs non-standard NaN in JSON

          • Done
          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            jcarlin Jeffrey Carlin created issue -
            jcarlin Jeffrey Carlin made changes -
            Field Original Value New Value
            Remote Link This issue links to "Page (Confluence)" [ 30300 ]
            sullivan Ian Sullivan made changes -
            Labels verify_metrics ap-analysis verify_metrics
            sullivan Ian Sullivan made changes -
            Epic Link DM-30501 [ 510159 ]
            Team Alert Production [ 10300 ]
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            This is now fixed via DM-31131. We decided to preserve the NaN values and represent them as null in the JSON job document. In the SQuaSH database they are stored as SQLAlchemy NULL, and when writing to InfluxDB the recommended approach is to drop the NaN values.

            Show
            afausti Angelo Fausti added a comment - This is now fixed via DM-31131 . We decided to preserve the NaN values and represent them as null in the JSON job document. In the SQuaSH database they are stored as SQLAlchemy NULL, and when writing to InfluxDB the recommended approach is to drop the NaN values.
            sullivan Ian Sullivan made changes -
            Link This issue is duplicated by DM-31131 [ DM-31131 ]
            sullivan Ian Sullivan made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Invalid [ 11005 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jcarlin Jeffrey Carlin
              Watchers:
              Angelo Fausti, Ian Sullivan, Jeffrey Carlin, Leanne Guy
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.