Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_base
-
Labels:
-
Story Points:1
-
Team:Architecture
-
Urgent?:No
Description
In DM-33820 it seems that we are getting a situation on the IDF where hundreds of input datasets are being read by ButlerQuantumContext.get() and it taking longer than two hours to read them all from the object store.
Granted that this probably means the algorithm should be rewritten to use deferred gets, but it does suggest that we should consider a heartbeat verbose log message where every ten minutes or so we report "Read N out of M requested datasets". Presumably there is a risk that put() also needs this.
I've added the logging for .get(). I'm inclined to not add more logging until we come across a situation where things are taking a long time and a log message every 10 minutes or so would help people.