Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33921

Replace in-place periodic loggers with PeriodicLogging class

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      2
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Replace repetitive periodic logging in various tasks with PeriodicLogger objects to log heartbeat messages.

        Attachments

          Issue Links

          is triggered by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33919 Add heartbeat logging support to utils

          • Done
          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33925 Remove loggingInterval fields from tasks

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33950 Periodic logger must issue messages from caller code

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            kannawad Arun Kannawadi added a comment -

            Tim Jenness Is there a procedure to follow to remove pexConfig fields from Tasks (in this case, `loggingInterval`). I doubt any one would have explicitly used the field though.

            Show
            kannawad Arun Kannawadi added a comment - Tim Jenness  Is there a procedure to follow to remove pexConfig fields from Tasks (in this case, `loggingInterval`). I doubt any one would have explicitly used the field though.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            https://developer.lsst.io/stack/deprecating-interfaces.html#config-deprecation

            Show
            tjenness Tim Jenness added a comment - https://developer.lsst.io/stack/deprecating-interfaces.html#config-deprecation
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Looks great. Thanks for getting on this so quickly. It definitely cleans things up.

            Show
            tjenness Tim Jenness added a comment - Looks great. Thanks for getting on this so quickly. It definitely cleans things up.
            Hide
            Permalink
            kannawad Arun Kannawadi added a comment -

            Successful Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36040/pipeline

            Show
            kannawad Arun Kannawadi added a comment - Successful Jenkins run:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36040/pipeline
            Hide
            Permalink
            kannawad Arun Kannawadi added a comment -

            On another run, I turned on the logging level to INFO and reduced the interval to 0, just to trigger the logs in Jenkins.

            https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36052/pipeline

            The run was successful and produced log messages such as 

             

            INFO 2022-03-06T02:58:35.739+00:00 lsst.deblend.multibandDeblend (deblend:{skymap: 'discrete/ci_hsc', tract: 0, patch: 69})(logging.py:386) - Deblended 154 parent sources out of 2994

             

            INFO 2022-03-06T02:58:36.414+00:00 lsst.imageDifference.forcedMeasurement (imageDifference:{instrument: 'HSC', skymap: 'discrete/ci_hsc', detector: 17, visit: 903336, ...})(logging.py:386) - Forced measurement complete for 10 parents (and their children) out of 271

             

            Show
            kannawad Arun Kannawadi added a comment - On another run, I turned on the logging level to INFO and reduced the interval to 0, just to trigger the logs in Jenkins. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36052/pipeline The run was successful and produced log messages such as    INFO 2022-03-06T02:58:35.739+00:00 lsst.deblend.multibandDeblend (deblend:{skymap: 'discrete/ci_hsc', tract: 0, patch: 69})(logging.py:386) - Deblended 154 parent sources out of 2994   INFO 2022-03-06T02:58:36.414+00:00 lsst.imageDifference.forcedMeasurement (imageDifference:{instrument: 'HSC', skymap: 'discrete/ci_hsc', detector: 17, visit: 903336, ...})(logging.py:386) - Forced measurement complete for 10 parents (and their children) out of 271  
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Thanks. That reminded me that I'm supposed to issue the log message from the caller line.

            Show
            tjenness Tim Jenness added a comment - Thanks. That reminded me that I'm supposed to issue the log message from the caller line.
            Hide
            Permalink
            kannawad Arun Kannawadi added a comment -

            Oh yes, I missed noting that.

            Show
            kannawad Arun Kannawadi added a comment - Oh yes, I missed noting that.
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Preops because this whole periodic logging business was spurred by panda on the IDF 

            Show
            yusra Yusra AlSayyad added a comment - Preops because this whole periodic logging business was spurred by panda on the IDF 

              People

              Assignee:
              kannawad Arun Kannawadi
              Reporter:
              kannawad Arun Kannawadi
              Reviewers:
              Tim Jenness
              Watchers:
              Arun Kannawadi, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.