Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-797

Change default logging settings for RSP notebooks

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Until recently all pipelines software logged using lsst.log and Jupyter diverted the log messages to a black hole because it didn't know what to do with them (system stderr is not the same as python stderr – see DM-15361).

      With recent changes to how Tasks work we are now using python logging everywhere except in C++ code. This means that Jupyter does have some say in the output of log messages. Some testing on IDF RSP shows that we have configured Jupyter such that:

      • The default log level is Python's default of WARNING
      • The default log message format is to just show the message with no context (which does not seem to be the Jupyter default).

      I propose that we make two changes to the default notebook configuration:

      Firstly change the default log message format to something like "%(name)s %(levelname)s: %(message)s" – this is the default used by pipetask and makes it a bit clearer about where the message is coming from and its severity.

      Secondly I would like to change the default log level to INFO. pipetask and command line task (and lsst.log) default to INFO and this would be more consistent with expectations. If people do not want this can we please have documentation explaining how people can change the default log level for all their notebooks so that they do not have to change it every time.

      Two caveats:

      • C++ log messages will still disappear unless somewhere we run lsstLog.configure_pylog_MDC("INFO") somewhere at start up.
      • The science pipeline convention for loggers is to not include a lsst. prefix in the logger name. This means that it's not possible for use to set the default log level for all LSST loggers to one thing and leave the matplotlib and conda loggers (for example) at WARNING. We should ponder whether we want to change that policy.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            Okay. Following Simon Krughoff's comment I'll adopt this RFC.

            Show
            tjenness Tim Jenness added a comment - Okay. Following Simon Krughoff 's comment I'll adopt this RFC.
            Hide
            ktl Kian-Tat Lim added a comment -

            Instead of

            <font color="red">
            

            you should at least use

            <span style="color: red">
            

            See https://www.w3schools.com/tags/tag_font.asp

            Show
            ktl Kian-Tat Lim added a comment - Instead of < font color = "red" > you should at least use < span style = "color: red" > See https://www.w3schools.com/tags/tag_font.asp
            Hide
            tjenness Tim Jenness added a comment -

            I'm moving discussion of the logging handler to DM-31684

            Show
            tjenness Tim Jenness added a comment - I'm moving discussion of the logging handler to DM-31684
            Hide
            tjenness Tim Jenness added a comment -

            Adam Thornton is it now the case that logging is set up for all RSP users as described on this RFC?

            Show
            tjenness Tim Jenness added a comment - Adam Thornton is it now the case that logging is set up for all RSP users as described on this RFC?
            Hide
            athornton Adam Thornton added a comment -

            All users who have spawned a lab newer than w_2021_39, yes.

            Show
            athornton Adam Thornton added a comment - All users who have spawned a lab newer than w_2021_39, yes.

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Adam Thornton, Alex Drlica-Wagner, Frossie Economou, Gregory Dubois-Felsmann, Jeffrey Carlin, Kian-Tat Lim, Melissa Graham, Merlin Fisher-Levine, Simon Krughoff, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.