Details
-
Type:
Bug
-
Status: Won't Fix
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_butler
-
Labels:None
-
Epic Link:
-
Sprint:AP S20-3 (February)
-
Team:Alert Production
-
Urgent?:No
Description
During development of DM-23174 and DM-22599, Krzysztof Findeisen found that it was sometimes expensive to create a new repository (4-6 seconds), or even to create a Butler to a new collection in an existing repository (~0.8 seconds). We could not reproduce this behavior on any other machine, including lsst-dev. See discussions on daf_butler#229 and pipe_base#114.
We initially suspected this was due to a slow disk on Krzysztof Findeisen's computer, but the behavior also appears with in-memory (both datastore and registry) repositories. Sometimes multiple repository creations are all slow, and sometimes only the first one is; this aspect cannot be reliably reproduced.
Tim Jenness suggested https://pypi.org/project/pytest-profiling/ as a starting point for more detailed investigations.
Attachments
Activity
I was never able to reproduce since on my machine it was always super fast.
We are currently in a window of unreproducibility, because I don't have access to the machine in question (and probably won't for several months).
However, since nobody had any idea what was causing the problem, I'd be surprised if incidental changes happened to fix it. I'll understand if you want to close it, but I think it makes sense to leave it open.
Krzysztof Findeisen
added a comment - We are currently in a window of unreproducibility, because I don't have access to the machine in question (and probably won't for several months).
However, since nobody had any idea what was causing the problem, I'd be surprised if incidental changes happened to fix it. I'll understand if you want to close it, but I think it makes sense to leave it open. Fine with me; let's leave it open for now, and check back at the start of next cycle!
The window of reproducibility is effectively closed – I still have the machine, but trying to recreate a development environment just to test this issue doesn't seem worth it.
Krzysztof Findeisen
added a comment - The window of reproducibility is effectively closed – I still have the machine, but trying to recreate a development environment just to test this issue doesn't seem worth it. 
Hey folks — is this still a current issue? I'm worried that since it was only ever seen on one machine, and since the Butler codebase is now moving fast, we may have passed the “window of reproducibility” here, in which case we should close this down.