Details
-
Type:
RFC
-
Status: Implemented
-
Resolution: Done
-
Component/s: DM
-
Labels:None
-
Location:This issue
Description
The Data Access team has concerns about how we're doing logging. They would like to use Python's built-in logging library for most standalone, pure-Python utilities. In addition, we're currently using pex_logging within C++ and Python code throughout much of the LSST Stack. This custom code should be replaced. lsst::log has been developed as a thin layer on top of log4cxx that will be more flexible and maintainable as it is based on a standard external package. We need to transition the Stack to this interface.
Among the Data Access team's concerns that point in favor of using Python logging:
- Desire to minimize dependencies on LSST Stack and log4cxx
- Simplicity of usage and default configuration
- Use with third-party packages (that typically use Python logging)
- Familiarity
I propose to decree the following policy, although I'm not going to do much of the actual coding work:
- C++ code throughout the Stack and Qserv uses lsst::log. (If anything is missing or poorly performing in that package, it should be fixed.)
- All Python code developed by LSST DM uses Python logging.
- Python apps (like CmdLineTask) that use packages that contain C++ code explicitly direct Python logging to use lsst.log in code (not via a configuration file). (I'm still worried that this will require configuration of individual logging components in multiple places depending on whether they're C++ or Python.)
- A standard default Python logging message format and a standard default lsst::log message format that are compatible with each other will be defined in a separate RFC. This includes things like always logging timestamps in UTC. (Note for pedants: I think these would be TAI if and only if we take the step of shifting all production machines' system times to TAI. Even then, I would expect that non-production machines like developer's laptops will never produce TAI.)
- All current code must be ported from pex_logging to the above by the end of Winter 2016 (Summer 2015?).
- The Python logging configuration file that is currently installed and used by default by Qserv will be removed.
Attachments
Issue Links
- is triggering
-
DM-8270 Port daf_ingest to log package
- Done
-
DM-8365 deprecate pex_logging
- Done
-
DM-8992 Migrate logging in ctrl_provenance to log
- Won't Fix
- relates to
-
RFC-782 Remove Task dependency on lsst.log
- Implemented
-
RFC-789 Make lsst.log python interface more compatible with python logging
- Implemented
-
DM-6520 Prepare an RFC about logging migration
- Done
-
RFC-203 Use lsst::log in pipeline tasks
- Implemented
John Swinbank Thanks. I had an old checkout of ci_hsc.
daf_ingest is covered by
DM-8270.ctrl_provenance has noticket.