Details
-
Type:
Story
-
Status: Won't Fix
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: meas_extensions_scarlet
-
Labels:
-
Team:Data Release Production
-
Urgent?:No
Description
As noted in DM-31067, a thus far unidentified/acknowledged error has been observed in the DC2 reprocessing runs. The traceback for a gen2 run log is:
/datasets/DC2/repoRun2.2i/rerun/w_2021_32/DM-31351/logs/04_multiBandDriver/mt3829-05.o54859: |
|
|
Traceback (most recent call last):
|
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/meas_extensions_scarlet/21.0.0-14-g574b836+2eb1d5918e/python/lsst/meas/extensions/scarlet/scarletDeblendTask.py", line 755, in deblend |
blend, skipped, spectrumInit = deblend(mExposure, foot, self.config) |
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/meas_extensions_scarlet/21.0.0-14-g574b836+2eb1d5918e/python/lsst/meas/extensions/scarlet/scarletDeblendTask.py", line 300, in deblend |
blend = Blend(sources, observation) |
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/scarlet/lsst-dev-g965bb5fbbf+f31336177f/lib/python/scarlet/blend.py", line 79, in __init__ |
super().__init__(self.sources) |
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/scarlet/lsst-dev-g965bb5fbbf+f31336177f/lib/python/scarlet/component.py", line 237, in __init__ |
super().__init__(frame, children=components, bbox=box) |
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/scarlet/lsst-dev-g965bb5fbbf\ |
+f31336177f/lib/python/scarlet/component.py", line 38, in __init__ |
super().__init__(*parameters, children=children) |
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/scarlet/lsst-dev-g965bb5fbbf+f31336177f/lib/python/scarlet/model.py", line 47, in __init__ |
self.check_parameters() |
File "/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/scarlet/lsst-dev-g965bb5fbbf+f31336177f/lib/python/scarlet/model.py", line 122, in check_parameters |
raise ArithmeticError(msg) |
ArithmeticError: Model Blend, Parameter 'spectrum' is not finite: |
[3.3891191e+04 3.0141157e-01 9.9999997e-21 nan 9.9999997e-21 |
9.9999997e-21] |
And a similar error is seen in the Gen3 run (DM-31044) in this log file for patch 35:
/scratch/brendal4/bps-gen3-dc2/submit/2.2i/runs/test-med-1/w_2021_28/DM-31044/20210716T012652Z/jobs/deblend/1362_deblend_3829_35_.3117668.err |
The earliest occurrence appears to date back to w_2021_24. It doesn't appear in the w_2021_16 and prior logs (but log persistence has been inconsistent...). This error has not been seen in the HSC RC2 processing, so it does seem this is an edge case, but is still worth investigating.
This is no longer an issue now that scarlet_lite is set to replace scarlet.