Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-782

Remove Task dependency on lsst.log

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      The Task framework is an implementation of a fairly generic pipeline building block that has the possibility to be useful outside of the LSST pipeline software environment. One impediment to this is the dependency on lsst.log which requires a C++ compiler, depends on both utils and base packages and also uses the Log4CXX package. We propose that we switch the Task class (and pipe_base in general) to use the python logging package. This implies that we also change all python Task subclasses over to python logging.

      To aid with the transition we will make an initial modification of Task to use a python logging subclass that implements compatibility methods (such as .getName() for .name and .trace() for .log(TRACE...)) which will start issuing deprecation messages. These can be fixed as related code is changed. Once we are confident that lsst.log is not in use in Tasks any more we can switch Task to using a standard python logger.

      This builds on the decisions made in RFC-29 where it was agreed that pure-python packages can use python logging and are not required to use lsst.log.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            What about Tasks that have a C++ component that uses lsst.log?

            Show
            Parejkoj John Parejko added a comment - What about Tasks that have a C++ component that uses lsst.log?
            Hide
            tjenness Tim Jenness added a comment - - edited

            Have you got an example of a case where the logger itself is passed into C++? From what we've seen it's the name of the logger (getLogger does a look up of the name). We have built lsst_distrib using this shim python logger and everything "worked" (no tests failed at least). When using pipetask we configure both lsst.log and logging so that there is consistent formatting between python and C++ loggers. Gen2 Command line task only configures a python2 logger.

            Show
            tjenness Tim Jenness added a comment - - edited Have you got an example of a case where the logger itself is passed into C++? From what we've seen it's the name of the logger (getLogger does a look up of the name). We have built lsst_distrib using this shim python logger and everything "worked" (no tests failed at least). When using pipetask we configure both lsst.log and logging so that there is consistent formatting between python and C++ loggers. Gen2 Command line task only configures a python2 logger.
            Hide
            jbosch Jim Bosch added a comment -

            We should try a ci_hsc build as well to be more sure that isn't a problem.

            We should also make sure the logic used by pipetask is easy to use in other contexts where users want both kinds of logging, and we've got examples of common extensions (e.g. making both log to a file, or changing the level of a specific logger name without knowing whether it's a C++ or Python logger.  This has the potential to make logging setup more difficult than it already is, but that's a problem we can address with some convenience code and documentation.

            Other than that, I'm in favor.

            Show
            jbosch Jim Bosch added a comment - We should try a ci_hsc build as well to be more sure that isn't a problem. We should also make sure the logic used by pipetask is easy to use in other contexts where users want both kinds of logging, and we've got examples of common extensions (e.g. making both log to a file, or changing the level of a specific logger name without knowing whether it's a C++ or Python logger.  This has the potential to make logging setup more difficult than it already is, but that's a problem we can address with some convenience code and documentation. Other than that, I'm in favor.
            Hide
            tjenness Tim Jenness added a comment -

            Given that gen2 still exists, we will need to copy the logger configuration code from gen3 to command line task.

            Show
            tjenness Tim Jenness added a comment - Given that gen2 still exists, we will need to copy the logger configuration code from gen3 to command line task.
            Hide
            tjenness Tim Jenness added a comment - - edited

            Adopting this with a plan of:

            • Change Task log class to be a subclass of logging that provides compatibility with lsst.log but issues deprecation warnings.
            • As time goes by change task code that uses those methods.
            • Update gen2 command line task such that it configures python logging as well as lsst.log (the code for doing this can be copied from daf_butler)
            • Eventually switch to standard python logging.
            Show
            tjenness Tim Jenness added a comment - - edited Adopting this with a plan of: Change Task log class to be a subclass of logging that provides compatibility with lsst.log but issues deprecation warnings. As time goes by change task code that uses those methods. Update gen2 command line task such that it configures python logging as well as lsst.log (the code for doing this can be copied from daf_butler) Eventually switch to standard python logging.

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Colin Slater, Gregory Dubois-Felsmann, Jim Bosch, John Parejko, Kian-Tat Lim, Leanne Guy, Michelle Butler, Tatiana Goldina, Tim Jenness, Wil O'Mullane, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  CI Builds

                  No builds found.