Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33820

Emit periodic log messages during Faro data loading

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: faro
    • Labels:
      None
    • Story Points:
      2
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Some of the batch system monitoring tools expect tasks to periodically emit log messages, to ensure that the task is still running and not in some stuck state. Panda e.g. expects these messages at at least once every two hours, but it's advisable for it to be more frequent. Faro's match preparation tasks can potentially spend a long time in data loading, and should have some log messages added during this process.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            Yes, your previous patch (if you still have it) could seemingly be backported easily even if this better version is merged to main. I have not backported my logging to butler get because they changed the node configuration.

            Show
            tjenness Tim Jenness added a comment - Yes, your previous patch (if you still have it) could seemingly be backported easily even if this better version is merged to main. I have not backported my logging to butler get because they changed the node configuration.
            Hide
            kannawad Arun Kannawadi added a comment -

            I can pull up my earlier patch from my reflogs before too late. As long it is acceptable to have the backport branch have the same behavior with different implementation, we should do that.

            Show
            kannawad Arun Kannawadi added a comment - I can pull up my earlier patch from my reflogs before too late. As long it is acceptable to have the backport branch have the same behavior with different implementation, we should do that.
            Hide
            tjenness Tim Jenness added a comment -

            I don't see any problem with having the branch use a slightly different implementation.

            Show
            tjenness Tim Jenness added a comment - I don't see any problem with having the branch use a slightly different implementation.
            Hide
            tjenness Tim Jenness added a comment - - edited

            Arun Kannawadi we've discussed the backporting of this and decided that we do not need to do the backport because step 3 (the step where this would matter) is now almost complete and there is not going to be another 23.0.1 release candidate.

            Show
            tjenness Tim Jenness added a comment - - edited Arun Kannawadi we've discussed the backporting of this and decided that we do not need to do the backport because step 3 (the step where this would matter) is now almost complete and there is not going to be another 23.0.1 release candidate.
            Hide
            kannawad Arun Kannawadi added a comment -

            Alright, I had raised a PR for backporting to not lose the discarded commits altogether. I closed the DM-33820-v23 PR and merged the main ticket branch to the main branch.

            Show
            kannawad Arun Kannawadi added a comment - Alright, I had raised a PR for backporting to not lose the discarded commits altogether. I closed the DM-33820 -v23 PR and merged the main ticket branch to the main branch.

              People

              Assignee:
              kannawad Arun Kannawadi
              Reporter:
              ctslater Colin Slater
              Reviewers:
              Tim Jenness
              Watchers:
              Arun Kannawadi, Colin Slater, Huan Lin, Shuwei Ye, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.