Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16319

ap_verify source count metrics do not exist

    XMLWordPrintable

    Details

      Description

      ap.association and ap.verify.measurements record a number of source count metrics using the lsst.verify framework. However, these metrics are not actually defined in verify_metrics. Add proper definitions for these metrics so that they appear in SQuaSH.

        Attachments

          Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. DM-16333 push ap_verify results to squash

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-16399 Backfill ap_verify CI jobs into SQuaSH demo instance

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            This issue has caused problems in DM-16399, because SQuaSH fails if measurements are not associated with a known metric. Angelo Fausti has a workaround, but it would be best to fix the problem quickly.

            Show
            krzys Krzysztof Findeisen added a comment - This issue has caused problems in DM-16399 , because SQuaSH fails if measurements are not associated with a known metric. Angelo Fausti has a workaround, but it would be best to fix the problem quickly.
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            Krzysztof Findeisen for the record, I found that the following metrics need to be added the verify_metrics.

            • association.numNewDiaObjects
            • association.fracUpdatedDiaObjects
            • association.numUnassociatedDiaObjects
            • ip_diffim.numSciSources
            • ip_diffim.fracDiaSourcesToSciSources
            • association.totalUnassociatedDiaObjects

            They are being ignored. Instead of raising an exception, the SQuaSH API gives a warning now.

            [pid: 17|app: 0|req: 1/1] 10.56.0.1 () {42 vars in 540 bytes} [Mon Nov  5 18:43:52 2018] POST /register => generated 57 bytes in 23 msecs (HTTP/1.1 400) 2 headers in 80 bytes (1 switches on core 0)
            		 
            [pid: 18|app: 0|req: 1/2] 10.56.0.1 () {42 vars in 532 bytes} [Mon Nov  5 18:43:55 2018] POST /auth => generated 188 bytes in 99 msecs (HTTP/1.1 200) 2 headers in 72 bytes (1 switches on core 0)
            		 
            ./app/api_v1/job.py:304: UserWarning: Metric `association.numNewDiaObjects` not found, it looks like the metrics definition is out of date.
            		 
              "date.".format(metric_name))
            		 
            ./app/api_v1/job.py:304: UserWarning: Metric `association.fracUpdatedDiaObjects` not found, it looks like the metrics definition is out of date.
            		 
              "date.".format(metric_name))
            		 
            ./app/api_v1/job.py:304: UserWarning: Metric `association.numUnassociatedDiaObjects` not found, it looks like the metrics definition is out of date.
            		 
              "date.".format(metric_name))
            		 
            ./app/api_v1/job.py:304: UserWarning: Metric `ip_diffim.numSciSources` not found, it looks like the metrics definition is out of date.
            		 
              "date.".format(metric_name))
            		 
            ./app/api_v1/job.py:304: UserWarning: Metric `ip_diffim.fracDiaSourcesToSciSources` not found, it looks like the metrics definition is out of date.
            		 
              "date.".format(metric_name))
            		 
            ./app/api_v1/job.py:304: UserWarning: Metric `association.totalUnassociatedDiaObjects` not found, it looks like the metrics definition is out of date.
            		 
              "date.".format(metric_name))
            		 
            [pid: 20|app: 0|req: 1/3] 10.56.0.1 () {44 vars in 732 bytes} [Mon Nov  5 18:44:33 2018] POST /job => generated 156 bytes in 412038 msecs (HTTP/1.1 202) 2 headers in 78 bytes (1 switches on core 0)

            Show
            afausti Angelo Fausti added a comment - Krzysztof Findeisen for the record, I found that the following metrics need to be added the verify_metrics . association.numNewDiaObjects association.fracUpdatedDiaObjects association.numUnassociatedDiaObjects ip_diffim.numSciSources ip_diffim.fracDiaSourcesToSciSources association.totalUnassociatedDiaObjects They are being ignored. Instead of raising an exception, the SQuaSH API gives a warning now. [pid: 17|app: 0|req: 1/1] 10.56.0.1 () {42 vars in 540 bytes} [Mon Nov 5 18:43:52 2018] POST /register => generated 57 bytes in 23 msecs (HTTP/1.1 400) 2 headers in 80 bytes (1 switches on core 0) [pid: 18|app: 0|req: 1/2] 10.56.0.1 () {42 vars in 532 bytes} [Mon Nov 5 18:43:55 2018] POST /auth => generated 188 bytes in 99 msecs (HTTP/1.1 200) 2 headers in 72 bytes (1 switches on core 0) ./app/api_v1/job.py:304: UserWarning: Metric `association.numNewDiaObjects` not found, it looks like the metrics definition is out of date. "date.".format(metric_name)) ./app/api_v1/job.py:304: UserWarning: Metric `association.fracUpdatedDiaObjects` not found, it looks like the metrics definition is out of date. "date.".format(metric_name)) ./app/api_v1/job.py:304: UserWarning: Metric `association.numUnassociatedDiaObjects` not found, it looks like the metrics definition is out of date. "date.".format(metric_name)) ./app/api_v1/job.py:304: UserWarning: Metric `ip_diffim.numSciSources` not found, it looks like the metrics definition is out of date. "date.".format(metric_name)) ./app/api_v1/job.py:304: UserWarning: Metric `ip_diffim.fracDiaSourcesToSciSources` not found, it looks like the metrics definition is out of date. "date.".format(metric_name)) ./app/api_v1/job.py:304: UserWarning: Metric `association.totalUnassociatedDiaObjects` not found, it looks like the metrics definition is out of date. "date.".format(metric_name)) [pid: 20|app: 0|req: 1/3] 10.56.0.1 () {44 vars in 732 bytes} [Mon Nov 5 18:44:33 2018] POST /job => generated 156 bytes in 412038 msecs (HTTP/1.1 202) 2 headers in 78 bytes (1 switches on core 0)
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment - - edited

            Hi Angelo Fausti, as an interested party, would you be willing to review this ticket? Thanks!

            The pull requests are having trouble showing up: verify_metrics#14 and ap_verify#51.

            Show
            krzys Krzysztof Findeisen added a comment - - edited Hi Angelo Fausti , as an interested party, would you be willing to review this ticket? Thanks! The pull requests are having trouble showing up: verify_metrics#14 and ap_verify#51 .
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            Looks good. I've tested this branch with SQuaSH and the new metrics were created correctly.

            Show
            afausti Angelo Fausti added a comment - Looks good. I've tested this branch with SQuaSH and the new metrics were created correctly.
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            Thanks for the quick review!

            Show
            krzys Krzysztof Findeisen added a comment - Thanks for the quick review!

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Angelo Fausti
              Watchers:
              Angelo Fausti, Eric Bellm, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.