# pipe_analysis needs updates for the wcs dataset name changes

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1
• Team:
Data Release Production

## Description

Running the visit-level scripts using the current master of pipe_analysis (6b5727d) with the w_2018_10 stack gave errors in getting the wcs dataset.

 Traceback (most recent call last):  File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/14.0-6-ge2c9487+51/python/lsst/pipe/base/cmdLineTask.py", line 408, in __call__  result = task.run(dataRef, **kwargs)  File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/visitAnalysis.py", line 215, in run  raise RuntimeError("No datasets found for datasetType = {:s}".format(repoInfo.dataset)) RuntimeError: No datasets found for datasetType = wcs 

It needs updates for the dataset name changes of DM-11138.

## Activity

Hide
Lauren MacArthur added a comment -

Hsin-Fang, I've pushed a branch that I think should fix things to work with the new and the old wcs dataset naming conventions.  Could you double-check if this works on both your w_2018_10 and w_2018_08 RC2 runs?  I'm not entirely sure if the comparison script adaptation will work, so if you could check the comparison between those two runs as well, that would be great.

Show
Lauren MacArthur added a comment - Hsin-Fang, I've pushed a branch that I think should fix things to work with the new and the old wcs dataset naming conventions.  Could you double-check if this works on both your w_2018_10 and w_2018_08 RC2 runs?  I'm not entirely sure if the comparison script adaptation will work, so if you could check the comparison between those two runs as well, that would be great.
Hide
Hsin-Fang Chiang added a comment -

Lauren MacArthur I ran into the following with compareVisitAnalysis:

 Traceback (most recent call last):  File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/14.0-6-ge2c9487+51/python/lsst/pipe/base/cmdLineTask.py", line 408, in __call__  result = task.run(dataRef, **kwargs)  File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/visitAnalysis.py", line 640, in run  if self.config.doApplyUberCal2 and "wcs" in repoInfo2.dataset2: AttributeError: 'Struct' object has no attribute 'dataset2' 

Looks like codes around line 640 need updates too

Show
Hsin-Fang Chiang added a comment - Lauren MacArthur  I ran into the following with compareVisitAnalysis: Traceback (most recent call last): File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/14.0-6-ge2c9487+51/python/lsst/pipe/base/cmdLineTask.py" , line 408 , in __call__ result = task.run(dataRef, **kwargs) File "/home/hchiang2/stack/pipe_analysis/python/lsst/pipe/analysis/visitAnalysis.py" , line 640 , in run if self.config.doApplyUberCal2 and "wcs" in repoInfo2.dataset2: AttributeError: 'Struct' object has no attribute 'dataset2' Looks like codes around line 640 need updates too
Hide
Lauren MacArthur added a comment -

Ok, I pushed a fix.  Let me know if it works now.

Show
Lauren MacArthur added a comment - Ok, I pushed a fix.  Let me know if it works now.
Hide
Hsin-Fang Chiang added a comment -

The new fix works with compareVisitAnalysis with w_2018_10, and all 5 scripts for the RC2 w_2018_10 have been finished (DM-13761).

I'll try to run a few examples with w_2018_08 and report back.

Show
Hsin-Fang Chiang added a comment - The new fix works with compareVisitAnalysis with  w_2018_10 , and all 5 scripts for the RC2 w_2018_10 have been finished ( DM-13761 ). I'll try to run a few examples with w_2018_08 and report back.
Hide
Hsin-Fang Chiang added a comment -

I ran visitAnalysis.py using the ticket branch with one visit in the w_2018_08 output repo, and it ran fine, with the expected message visitAnalysis INFO: Old meas_mosaic dataset naming: wcs (new name is jointcal_wcs) in the log.

compareVisitAnalysis in DM-13761 was the w_2018_10 repo against the w_2018_08 repo, and it finished fine too. So I think the branch is good to merge.

Not in this ticket, but may we add some unit tests to pipe_analysis?

Show
Hsin-Fang Chiang added a comment - I ran visitAnalysis.py using the ticket branch with one visit in the w_2018_08 output repo, and it ran fine, with the expected message visitAnalysis INFO: Old meas_mosaic dataset naming: wcs (new name is jointcal_wcs) in the log. compareVisitAnalysis in DM-13761 was the w_2018_10 repo against the w_2018_08 repo, and it finished fine too. So I think the branch is good to merge. Not in this ticket, but may we add some unit tests to pipe_analysis?
Hide
Hsin-Fang Chiang added a comment -

The ticket branch has been merged to master per Lauren MacArthur's request

Show
Hsin-Fang Chiang added a comment - The ticket branch has been merged to master per Lauren MacArthur 's request

## People

• Assignee:
Lauren MacArthur
Reporter:
Hsin-Fang Chiang
Reviewers:
Hsin-Fang Chiang
Watchers:
Hsin-Fang Chiang, Lauren MacArthur