# Error reading in Quantum Graph with measurement config

XMLWordPrintable

#### Details

• Type: Bug
• Status: To Do
• Resolution: Unresolved
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
Data Release Production
• Urgent?:
No

#### Description

When submitting the attached BPS config file, an Exception is thrown when trying to read the generated quantum graph. The quantum graph is generated with a config file that configures a measurement plugin that is not in lsst_distrib. The creation and running of the Quantum Graph works from the command line but fails when trying to submit jobs via BPS.

#### Attachments

1. bpsConfig.yaml
0.4 kB

#### Activity

Hide
Nate Lust added a comment - - edited

Arun Kannawadi can you point me to where the test quantum graph is on disk?
 - nevermind I overlooked a comment I see it now.

Show
Nate Lust added a comment - - edited Arun Kannawadi can you point me to where the test quantum graph is on disk?  - nevermind I overlooked a comment I see it now.
Hide

The path earlier (fixed now) was incorrect and you can fin the quantum graph at /project/kannawad/gaapCLI.qgraph. It can also be generated using Michelle's comment.

Show
Arun Kannawadi added a comment - The path earlier (fixed now) was incorrect and you can fin the quantum graph at /project/kannawad/gaapCLI.qgraph. It can also be generated using Michelle's comment.
Hide
Nate Lust added a comment -

So here is the issue, and it is not entirely unexpected. The quantum graph contains a pickled version of the configs. With packages in the stack these are already setup and unpickling can pull all the appropriate bits in. However, external packages that register something have a side effect on the config when they are created, but if they are not imported in the new environment, do not get a change to fire their "import time" behavior prior to the config being persisted.

Normally this is mitigated by having imports at the top of persisted config files, but in the case of the quantum graph, the object has only pickles that don't go through the normal load process.

I have a few ideas to mitigate this issue when I get a few moments, I will look at implementing them.

Show
Nate Lust added a comment - So here is the issue, and it is not entirely unexpected. The quantum graph contains a pickled version of the configs. With packages in the stack these are already setup and unpickling can pull all the appropriate bits in. However, external packages that register something have a side effect on the config when they are created, but if they are not imported in the new environment, do not get a change to fire their "import time" behavior prior to the config being persisted. Normally this is mitigated by having imports at the top of persisted config files, but in the case of the quantum graph, the object has only pickles that don't go through the normal load process. I have a few ideas to mitigate this issue when I get a few moments, I will look at implementing them.
Hide
Michelle Gower added a comment -

I was browsing tickets with component ctrl_bps and ran across this one.  Is this still a problem?

Show
Michelle Gower added a comment - I was browsing tickets with component ctrl_bps and ran across this one.  Is this still a problem?
Hide

It's not a problem any more in the sense that meas_extensions_gaap is now part of lsst_distrib. However, I don't know if there has been changes to ctrl_bps that will avoid this problem for any new plugin that may be developed. I could check with a dummy plugin if we want to mark it as Invalid or not.

Show
Arun Kannawadi added a comment - It's not a problem any more in the sense that meas_extensions_gaap is now part of lsst_distrib. However, I don't know if there has been changes to ctrl_bps that will avoid this problem for any new plugin that may be developed. I could check with a dummy plugin if we want to mark it as Invalid or not.

#### People

Assignee:
Nate Lust
Reporter:
Watchers:
Arun Kannawadi, Michelle Gower, Nate Lust