Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22479

SQuaSH being spammed with timestamps

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: verify
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      AP S20-1 (December)
    • Team:
      Alert Production

      Description

      DM-22093 added a "Task finished" timestamp to the metadata for all verify timing metrics. This inadvertantly caused the timestamps to be treated as tags in SQuaSH (see attached image), and will eventually bog down the server.

      Change the timestamps from metadata to extras, which are not uploaded to SQuaSH. This is a breaking change, and will make accessing the data a bit clunkier:

      measurement.notes["end"]
      measurement.extras["end"].quantity  # yes, still a `str`
      

        Attachments

          Issue Links

            Activity

            Hide
            krzys Krzysztof Findeisen added a comment -

            Hi Chris Morrison, can you review this fix? I think it will break some of your test code, sorry about that.

            Show
            krzys Krzysztof Findeisen added a comment - Hi Chris Morrison , can you review this fix? I think it will break some of your test code, sorry about that.
            Hide
            cmorrison Chris Morrison added a comment -

            Approved the pull request. I haven't exclusively started to use this output yet so no test code on my end will be broken yet.

            Show
            cmorrison Chris Morrison added a comment - Approved the pull request. I haven't exclusively started to use this output yet so no test code on my end will be broken yet.
            Hide
            krzys Krzysztof Findeisen added a comment -

            I've fixed the code so that it won't upload more timestamps. Angelo Fausti, would it be worthwhile/possible to erase the existing *Time.end metadata from the SQuaSH DB? Unfortunately it's attached to about a dozen metrics.

            Show
            krzys Krzysztof Findeisen added a comment - I've fixed the code so that it won't upload more timestamps. Angelo Fausti , would it be worthwhile/possible to erase the existing *Time.end metadata from the SQuaSH DB? Unfortunately it's attached to about a dozen metrics.
            Hide
            afausti Angelo Fausti added a comment - - edited

            Yes, I think that's possible. I'll have to rewrite the measurement without that tag following a procedure similar to the one described here Workaround for dropping individual field keys

            Show
            afausti Angelo Fausti added a comment - - edited Yes, I think that's possible. I'll have to rewrite the measurement without that tag following a procedure similar to the one described here Workaround for dropping individual field keys

              People

              • Assignee:
                krzys Krzysztof Findeisen
                Reporter:
                krzys Krzysztof Findeisen
                Reviewers:
                Chris Morrison
                Watchers:
                Angelo Fausti, Chris Morrison, Krzysztof Findeisen
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel