# How to handle logging for a repeatedly-run subtask?

XMLWordPrintable

#### Details

• Type: Story
• Status: To Do
• Resolution: Unresolved
• Fix Version/s: None
• Component/s:
• Labels:
None
• Team:
Data Access and Database

#### Description

How should one handle limiting the logging of a subtask when it is run many times?

This arose in DM-7611: a task can be run either once (in which case it generates a limited amount of useful INFO logging), or be run as a subtask, in which case it is run many (>100) times. In the latter case, the INFO logging is not as useful, and the calling task squashes the INFO logging of the subtask before calling its run method, but that could be dangerous if the user actually wants to get all of the INFO messages.

Here is the discussion in the PR:
https://github.com/lsst/ip_diffim/pull/53#discussion_r108770375

What is the correct way to handle this?

#### Activity

Hide
Tim Jenness added a comment -

Does anyone have any comment as to what the solution for this ticket might be? Maybe Yusra AlSayyad, Nate Lust, or Andy Salnikov have opinions as to how to deal with this or whether we should ignore it?

Show
Tim Jenness added a comment - Does anyone have any comment as to what the solution for this ticket might be? Maybe Yusra AlSayyad , Nate Lust , or Andy Salnikov have opinions as to how to deal with this or whether we should ignore it?
Hide
Andy Salnikov added a comment -

I think setting logging level in the code is a bad practice indeed. Someone has to decide on logging strategy and I'm not sure if we can provide any decision for that. I believe we already have all tools that are needed to implement a reasonable strategy if someone else decides what it is.

One random idea for a strategy - the task could have a flag which defines the logging level for the messages, you could select INFO (or default) if you run it once or DEBUG if you run it many times. If a user want to see messages from latter case then it's easy to change logging level from command line for that particular task (or whole thing).

Show
Andy Salnikov added a comment - I think setting logging level in the code is a bad practice indeed. Someone has to decide on logging strategy and I'm not sure if we can provide any decision for that. I believe we already have all tools that are needed to implement a reasonable strategy if someone else decides what it is. One random idea for a strategy - the task could have a flag which defines the logging level for the messages , you could select INFO (or default) if you run it once or DEBUG if you run it many times. If a user want to see messages from latter case then it's easy to change logging level from command line for that particular task (or whole thing).
Hide
Kian-Tat Lim added a comment -

I very much agree with Andy that setting the logging threshold in the code is a bad practice.

I can empathize with the strong psychological pull to having some default logging from a Task, but rationally it seems to me that if a message should be DEBUG in the run-many-times context, it should still be DEBUG in the run-once context.

Show
Kian-Tat Lim added a comment - I very much agree with Andy that setting the logging threshold in the code is a bad practice. I can empathize with the strong psychological pull to having some default logging from a Task, but rationally it seems to me that if a message should be DEBUG in the run-many-times context, it should still be DEBUG in the run-once context.
Hide
Nate Lust added a comment -

What about through config? Seems like this is something that is natural for pex_config.

• Config base class for tasks defines a field that controls the logging for that task
• There is a setting that controls the behavior for all subtasks that fires before the task config log level so you could override just one task
• This allows Pipelines to control longing per label, in the case there is more than one instance of a task running
• Allows individual control for a task if it is both a top level task and a sub task
• log levels would then be stored with configuration (in config files, pipelines, persisted configs etc) and would be repeatable each time a task is run.
Show

#### People

Assignee:
Unassigned
Reporter:
David Reiss
Watchers:
Andy Salnikov, David Reiss, Fritz Mueller, Hsin-Fang Chiang, John Parejko, Kian-Tat Lim, Nate Lust, Simon Krughoff, Tim Jenness