Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31120

Change butler logger to python logging

    XMLWordPrintable

    Details

    • Story Points:
      4
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      Currently the butler sends all log output to lsst.log. Now that there is a way to forward all log messages from lsst.log to python (DM-30996) change the way that butler (and derivatives such as ctrl_mpexec) handles logging. This will make everything consistent regardless of whether lsst.log is available and will simplify the implementation of DM-30977.

        Attachments

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-31180 ap_verify loses logs from pipeline

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31530 Log outputs are being swallowed when running pipe_drivers scripts

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-30977 Write pipeline task log output to butler

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-30996 Implement log4cxx appender to Python logging

          • Done

            Activity

            Ascending order - Click to sort in descending order
            5 older comments
            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            I think we are doing it right already by changing `log4cxx` log level/threshold in sync with Python. Logging macros that we define for C++ side do check effective logging level before formatting. And appender is not even called if message level is below logger threshold.

            Show
            salnikov Andy Salnikov added a comment - I think we are doing it right already by changing `log4cxx` log level/threshold in sync with Python. Logging macros that we define for C++ side do check effective logging level before formatting. And appender is not even called if message level is below logger threshold.
            Hide
            Permalink
            ktl Kian-Tat Lim added a comment -

            Oh, somehow I thought I saw that the C++ side was defaulting to DEBUG and relying on the Python side for filtering. But if that's not the case, then OK.

            Show
            ktl Kian-Tat Lim added a comment - Oh, somehow I thought I saw that the C++ side was defaulting to DEBUG and relying on the Python side for filtering. But if that's not the case, then OK.
            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            K-T is right as usual, and I missed that part. Python logging and lsst.log are initialized with different level, if --logging-level is not used then lsst.log will use DEBUG.
            This call https://github.com/lsst/daf_butler/blob/master/python/lsst/daf/butler/cli/cliLog.py#L111 should use "INFO" to match Python basicConfig().

            Show
            salnikov Andy Salnikov added a comment - K-T is right as usual, and I missed that part. Python logging and lsst.log are initialized with different level, if --logging-level is not used then lsst.log will use DEBUG. This call https://github.com/lsst/daf_butler/blob/master/python/lsst/daf/butler/cli/cliLog.py#L111 should use "INFO" to match Python basicConfig().
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Doesn't this line override things though?

            https://github.com/lsst/daf_butler/blob/master/python/lsst/daf/butler/cli/cliLog.py#L136

            I don't see lsst.log accepting debug messages from python lsst.log usage (in the sense that isEnabledFor is not letting them through).

            Show
            tjenness Tim Jenness added a comment - Doesn't this line override things though? https://github.com/lsst/daf_butler/blob/master/python/lsst/daf/butler/cli/cliLog.py#L136 I don't see lsst.log accepting debug messages from python lsst.log usage (in the sense that isEnabledFor is not letting them through).
            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            You are right, I missed that explicit call to _setLogLevel. Sorry for my confusion, indeed things look fine as they are now.

            Show
            salnikov Andy Salnikov added a comment - You are right, I missed that explicit call to _setLogLevel. Sorry for my confusion, indeed things look fine as they are now.

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Andy Salnikov
              Watchers:
              Andy Salnikov, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.