Details
-
Type:
RFC
-
Status: Implemented
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
The Task framework is an implementation of a fairly generic pipeline building block that has the possibility to be useful outside of the LSST pipeline software environment. One impediment to this is the dependency on lsst.log which requires a C++ compiler, depends on both utils and base packages and also uses the Log4CXX package. We propose that we switch the Task class (and pipe_base in general) to use the python logging package. This implies that we also change all python Task subclasses over to python logging.
To aid with the transition we will make an initial modification of Task to use a python logging subclass that implements compatibility methods (such as .getName() for .name and .trace() for .log(TRACE...)) which will start issuing deprecation messages. These can be fixed as related code is changed. Once we are confident that lsst.log is not in use in Tasks any more we can switch Task to using a standard python logger.
This builds on the decisions made in RFC-29 where it was agreed that pure-python packages can use python logging and are not required to use lsst.log.
Attachments
Issue Links
- is triggering
-
- Done
- relates to
-
- Implemented
-
- Implemented
-
- Implemented
Activity
Have you got an example of a case where the logger itself is passed into C++? From what we've seen it's the name of the logger (getLogger does a look up of the name). We have built lsst_distrib using this shim python logger and everything "worked" (no tests failed at least). When using pipetask we configure both lsst.log and logging so that there is consistent formatting between python and C++ loggers. Gen2 Command line task only configures a python2 logger.
We should try a ci_hsc build as well to be more sure that isn't a problem.
We should also make sure the logic used by pipetask is easy to use in other contexts where users want both kinds of logging, and we've got examples of common extensions (e.g. making both log to a file, or changing the level of a specific logger name without knowing whether it's a C++ or Python logger. This has the potential to make logging setup more difficult than it already is, but that's a problem we can address with some convenience code and documentation.
Other than that, I'm in favor.
Given that gen2 still exists, we will need to copy the logger configuration code from gen3 to command line task.
Adopting this with a plan of:
- Change Task log class to be a subclass of logging that provides compatibility with lsst.log but issues deprecation warnings.
- As time goes by change task code that uses those methods.
- Update gen2 command line task such that it configures python logging as well as lsst.log (the code for doing this can be copied from daf_butler)
- Eventually switch to standard python logging.
What about Tasks that have a C++ component that uses lsst.log?