Details
Description
Messages sent to lsst.log are not visible from Jupyter, because Jupyter only looks at Python's stderr and stdout, which are not the same as system stderr and stdout.
Following some discussion on Slack, the best approach appears to be writing an lsst.log and/or log4cxx handler that writes to either Python's stderr or stdout or some Jupyter cell output hook; the latter in particular could enable things like using rich-text output to highlight messages at different levels.
Attachments
Activity
To start with, copying a StackExchange answer, I've used a thread and a pipe to take logging output (os file descriptor 1) and feed it into Python's sys.stderr, which Jupyter in turn places in the cell. No fancy level-dependent coloring, but it appears to be basically functional.
Pinging this again, as Chris Morrison and I want to be able to see Butler warnings and errors in jupyter. Any progress?
Maybe it is easier to configure lsst.log to append to the same file descriptor as Python sys.stderr?
Can you try this piece of code in Jupiter:
import lsst.log
|
import sys
|
|
|
_LOG_PROP = """
|
log4j.rootLogger = DEBUG, JUPYTER
|
log4j.appender.JUPYTER=org.apache.log4j.FileAppender
|
log4j.appender.JUPYTER.File=/dev/fd/{}
|
log4j.appender.JUPYTER.layout=PatternLayout
|
log4j.appender.JUPYTER.layout.ConversionPattern=%c %p: %m%n
|
"""
|
lsst.log.configure_prop(_LOG_PROP.format(sys.stderr.fileno()))
|
and see if there is any output from lsst.log in a notebook? (change DEBUG to INFO if it is too chatty)
Chris Morrison: can you please try the above in one of your astrometry notebooks and see if you get some log output?
I have a couple of attempts at this that seemed to work but that I have not yet been able to test extensively. If someone could pick up one of these solutions (including Andy's above) and incorporate it into a package, that would be great.
https://gist.github.com/ktlim/4e9deae888b150bb6e17efb32c1e46c3 logs to cells
https://gist.github.com/ktlim/8b30bfbe8645583d3427718f96054974 logs to a Jupyter widget
Fritz Mueller, Frossie Economou — I think this is likely a SQuaRE issue rather than a DAX one, but since the team is set to DAX maybe you can check if you agree with me?
For what it's worth, this has been identified as a priority by Pipelines folks using the Notebook Aspect. I know everybody's snowed under, but please bear that in mind when scheduling.