[DM-31519] Improve handling of NaN metric values by dispatch_verify - Jira

XML

Word

Printable

Details

Type: Story
Status: Invalid
Resolution: Done
Fix Version/s: None
Component/s: verify_metrics
Labels:
- ap-analysis
- verify_metrics

Epic Link:
f21b-ap-emergent
Team:
Alert Production
Urgent?:
No

Description

NaN is now disallowed in JSON, which causes problems if trying to dispatch metrics that contain NaNs to SQuaSH. dispatchverify.py (in lsst.verify) needs to be updated to identify NaN metrics and convert them to a JSON-compliant value before dispatching the job. See this Slack thread and subsequent conversation in that channel for context.

Attachments

Issue Links

is duplicated by

DM-31131 lsst.verify outputs non-standard NaN in JSON

Done

mentioned in: Page Loading...

Activity

Hide

Permalink

Angelo Fausti added a comment - 23/Sep/21 6:12 PM

This is now fixed via ~~DM-31131~~. We decided to preserve the NaN values and represent them as null in the JSON job document. In the SQuaSH database they are stored as SQLAlchemy NULL, and when writing to InfluxDB the recommended approach is to drop the NaN values.

Show

Angelo Fausti added a comment - 23/Sep/21 6:12 PM This is now fixed via DM-31131 . We decided to preserve the NaN values and represent them as null in the JSON job document. In the SQuaSH database they are stored as SQLAlchemy NULL, and when writing to InfluxDB the recommended approach is to drop the NaN values.

People

Assignee:: Unassigned

Reporter:: Jeffrey Carlin

Watchers:: Angelo Fausti, Ian Sullivan, Jeffrey Carlin, Leanne Guy

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 24/Aug/21 11:55 PM

Updated:: 28/Oct/21 6:14 PM

Resolved:: 28/Oct/21 6:14 PM

Jenkins

No builds found.

Data Management