Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_analysis
-
Labels:None
-
Story Points:4
-
Epic Link:
-
Sprint:DRP F20-3 (Aug)
-
Team:Data Release Production
-
Urgent?:No
Description
coaddAnalysis.py gave the following error for some data:
Traceback (most recent call last):
|
File "/home/hchiang2/PDR2/pa_stack/pipe_analysis/bin/coaddAnalysis.py", line 3, in <module>
|
CoaddAnalysisTask.parseAndRun()
|
File "/software/lsstsw/stack_20200220/stack/miniconda3-4.7.12-984c9f7/Linux64/pipe_base/19.0.0-9-g0ae078d+2/python/lsst/pipe/base/cmdLineTask.py", line 610, in parseAndRun
|
resultList = taskRunner.run(parsedCmd)
|
File "/software/lsstsw/stack_20200220/stack/miniconda3-4.7.12-984c9f7/Linux64/pipe_base/19.0.0-9-g0ae078d+2/python/lsst/pipe/base/cmdLineTask.py", line 221, in run
|
resultList = list(mapFunc(self, targetList))
|
File "/software/lsstsw/stack_20200220/stack/miniconda3-4.7.12-984c9f7/Linux64/pipe_base/19.0.0-9-g0ae078d+2/python/lsst/pipe/base/cmdLineTask.py", line 385, in __call__
|
result = self.runTask(task, dataRef, kwargs)
|
File "/software/lsstsw/stack_20200220/stack/miniconda3-4.7.12-984c9f7/Linux64/pipe_base/19.0.0-9-g0ae078d+2/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
|
return task.runDataRef(dataRef, **kwargs)
|
File "/home/hchiang2/PDR2/pa_stack/pipe_analysis/python/lsst/pipe/analysis/coaddAnalysis.py", line 356, in runDataRef
|
repoInfo.dataId, forcedStr="unforced " + self.catLabel, **plotKwargs)
|
File "/home/hchiang2/PDR2/pa_stack/pipe_analysis/python/lsst/pipe/analysis/coaddAnalysis.py", line 881, in plotPsfFluxSnHists
|
"Flux>{:.1f}".format(highFlux), "psf_used"])
|
File "/home/hchiang2/PDR2/pa_stack/pipe_analysis/python/lsst/pipe/analysis/analysis.py", line 605, in plotHistogram
|
axes2.set_ylim(0, 1.05)
|
UnboundLocalError: local variable 'axes2' referenced before assignment
|
I was using w_2020_08 + pipe_analysis 09a7675
Attachments
Issue Links
- relates to
-
- Done
Activity
The code seems to be:
584 for name, data in self.data.items(): |
585 if not data.mag.any(): |
586 continue |
587 color = "tab:" + data.color |
588 ic += 1 |
589 good = np.isfinite(data.quantity) |
590 if magThreshold and stats is not None: |
591 good &= data.mag < magThreshold |
592 nValid = np.abs(data.quantity[good]) <= self.qMax # need to have datapoints lying within range |
593 if good.sum() == 0 or nValid.sum() == 0: |
594 continue |
595 num, fluxBins, _ = axes.hist(data.quantity[good], bins=numBins, range=(self.qMin, self.qMax), |
596 density=density, log=logPlot, color=color, alpha=alpha, |
597 label=name, histtype="stepfilled") |
598 if cumulative: |
599 axes2 = axes.twinx() # instantiate a second axes that shares the same x-axis |
600 axes2.hist(data.quantity[good], bins=fluxBins, density=True, log=False, color=data.color, |
601 label=name + "_cum", histtype="step", cumulative=cumulative) |
602 # yaxis limit for non-normalized histograms |
603 numMax = max(numMax, num.max()*1.1) if not density else numMax |
604 if cumulative: |
605 axes2.set_ylim(0, 1.05) |
606 axes2.tick_params(axis="y", which="both", direction="in") |
So I think you get this error if that for loop never loops over anything or the continue statement is hit before axes2 is defined.
Indeed...hopefully I can get a fix in tomorrow. Thanks for the report Hsin-Fang Chiang (& diagnosis Tim Jenness)!
I hate to dump another one on you, but I know you have run into this bug in your recent work, so it would be great if you can confirm if these fixes work for you. I have confirmed that the example how-to-reproduce above is now resolved, i.e. running:
coaddAnalysis.py /datasets/hsc/repo/ --rerun DM-23243/MULTIBAND/DEEP:private/lauren/DM-23829 --id tract=08766 filter=HSC-I2 |
now runs to completion without error.
It turned out that the above case revealed a deeper issue with the S/N threshold scaling for low visit numbers (was rounding down to zero!) which is now accommodated.
PR is here.
Just FYI, I just added an additional fix for the scaling and plot labels in the colorAnalysis.py script. I've tested it on limited (in coverage and number of input visits) datasets and all is working well and ready for review.
These changes look fine to me. Thanks for fixing this so quickly.
And thank you for the speedy review! Merged and done.
One example input data to reproduce:
coaddAnalysis.py /datasets/hsc/repo/ --calib /datasets/hsc/calib/20200115/ --rerun DM-23243/MULTIBAND/DEEP:private/whatever --doraise --config doWriteParquetTables=True --id tract=08766 filter=HSC-I2