Thanks for the example code. The million isinstance checks are a problem. The Config class is optimized for c["x", "y", "z"] form since that is all done internally whereas c["x"]["y"]["z"] form requires that an entirely new Config object is returned at each level of the hierarchy. Your code seems to think it has a dict so treats it like a dict – it would be faster to force the Config to a dict and then use that if you aren't actually using any Config features after the YAML parse phase. The other problem is that Config supports c["x", "4", "y"] form where the integer can index into a list so it needs to do an instance check for a sequence at every level in the hierarchy. The ["x"]["y"] form can be sped up a little by tweaking how the intermediate Config is created (so far about a third faster). It would also help I think if we had a FrozenConfig that would be read only since FrozenConfig could then cache some lookups. I need to try and profile things a little but for the record for the Config intermediates I go from 40 sec to 25 sec with a better Config constructor and then down to 15 sec if I don't do the sequence check (but we have to do that). It's 11 seconds using the tuple form directly.

I've made some changes to speed things up a little.

For reference the for the dict-like accessor with these changes my timing improved from 33 seconds to 22 seconds. I can get it down to 5 seconds but only if I break compatibility. The problem is that when you ask for a subset of a Config that is itself a dict, you are returned a new Config that is completely detached from the parent such that if you modify a value in the new Config it won't change anything in the original config – this means that at every level in the hierarchy a whole new deepcopy is done and those really add up. This is fundamentally different from how a normal dict hierarchy works and I'm not sure how we ended up in this situation and whether it was like this in daf_persistence originally. It's clear that if we had a FrozenConfig then this could be made much faster.

I'll send what I have out for review though since I don't really want to go down a Frozen Config rabbit hole on this ticket.
I think though that your best bet is to call `toDict()` on the config and then access it.

I reran the tests in DM-32241 using w44 + this ticket branch.  As you expected it doesn't speed up bps by too much:

• single quantum clustering:  117 -> 109
• transform: 9341 -> 8869
• prepare: 826 -> 757

I'm not quite sure what to make of the comment about bps using Config like a dict.  That's true now, but I don't know for sure it won't use other features in Config in the future (I didn't want to limit it just because.)  bps does use the config[".x.y.z"] notation in some places so that would have to be changed in the code.  I don't know if any users are using that style notation in their config values (presumably not, but we didn't do anything to prevent it).  I've made a comment in and linked this ticket in DM-31245, which is the generic rewrite BpsConfig ticket.  For bps, waiting to see how the value caching works in real use (available last weekly) before doing more work explicitly just for speed.

A couple comments about comments and one code question in the review. I did a pipelines_check submission with bps just to double check and it ran fine.   Changes approved for merging.