Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33820

Emit periodic log messages during Faro data loading

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • faro
    • None
    • 2
    • Data Release Production
    • No

    Description

      Some of the batch system monitoring tools expect tasks to periodically emit log messages, to ensure that the task is still running and not in some stuck state. Panda e.g. expects these messages at at least once every two hours, but it's advisable for it to be more frequent. Faro's match preparation tasks can potentially spend a long time in data loading, and should have some log messages added during this process.

      Attachments

        Issue Links

          Activity

            tjenness Tim Jenness added a comment -

            Yes, your previous patch (if you still have it) could seemingly be backported easily even if this better version is merged to main. I have not backported my logging to butler get because they changed the node configuration.

            tjenness Tim Jenness added a comment - Yes, your previous patch (if you still have it) could seemingly be backported easily even if this better version is merged to main. I have not backported my logging to butler get because they changed the node configuration.

            I can pull up my earlier patch from my reflogs before too late. As long it is acceptable to have the backport branch have the same behavior with different implementation, we should do that.

            kannawad Arun Kannawadi added a comment - I can pull up my earlier patch from my reflogs before too late. As long it is acceptable to have the backport branch have the same behavior with different implementation, we should do that.
            tjenness Tim Jenness added a comment -

            I don't see any problem with having the branch use a slightly different implementation.

            tjenness Tim Jenness added a comment - I don't see any problem with having the branch use a slightly different implementation.
            tjenness Tim Jenness added a comment - - edited

            kannawad we've discussed the backporting of this and decided that we do not need to do the backport because step 3 (the step where this would matter) is now almost complete and there is not going to be another 23.0.1 release candidate.

            tjenness Tim Jenness added a comment - - edited kannawad we've discussed the backporting of this and decided that we do not need to do the backport because step 3 (the step where this would matter) is now almost complete and there is not going to be another 23.0.1 release candidate.

            Alright, I had raised a PR for backporting to not lose the discarded commits altogether. I closed the DM-33820-v23 PR and merged the main ticket branch to the main branch.

            kannawad Arun Kannawadi added a comment - Alright, I had raised a PR for backporting to not lose the discarded commits altogether. I closed the DM-33820 -v23 PR and merged the main ticket branch to the main branch.

            People

              kannawad Arun Kannawadi
              ctslater Colin Slater
              Tim Jenness
              Arun Kannawadi, Colin Slater, Huan Lin, Shuwei Ye, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.