Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: summit_extras
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
The new logBrowser finds duplicat logs with some collections (presumably ones chained in a certain way).
This should be a relatively easy fix by chaning the butler query, I just need to check with middleware and patch.
Attachments
Issue Links
- is triggering
-
- Done
Activity
Oh, I'm not saying it's a general problem, I just didn't want to have every log start with
VERBOSE lsst.daf.butler.datastores.fileDatastore: Number of datasets found in datastore for chunk 0 = 0/1 (running total: 0/1)
|
VERBOSE lsst.daf.butler.datastores.fileDatastore: Number of datasets found in datastore for chunk 0 = 0/1 (running total: 0/1)
|
VERBOSE lsst.daf.butler.datastores.fileDatastore: Number of datasets found in datastore for chunk 0 = 0/1 (running total: 0/1)
|
VERBOSE lsst.daf.butler.datastores.fileDatastore: Number of datasets found in datastore for chunk 0 = 0/1 (running total: 0/1)
|
VERBOSE lsst.daf.butler.datastores.fileDatastore: Number of datasets found in datastore for chunk 0 = 0/1 (running total: 0/1)
|
by default. And having an option to supress any given loggers seemed like a nice feature.
(that is the start of a single log from characterizeImage btw)
(and FWIW, I just checked processStar and that also has 5 - I don't know if any of this matters, but just thought I'd let you know in case that's telling you something interesting about the butler.)
(I think that log message looks like it has a off-by-one error in it. Also, it probably should not be a log message at all if there is only one group of files to look for).
The reason it's verbose is that we had some places in batch where we are asking for thousands of datasets to be checked and if we don't print anything out we get no feedback into the log system that anything is happening (it could take tens of minutes).