# Complete and test use of jointcal results in validate_drp

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
4
• Epic Link:
• Sprint:
Alert Production F17 - 9, Alert Production F17 - 10, Alert Production F17 - 11, AP S18-1
• Team:
Alert Production

#### Description

DM-10729 added incomplete support for using meas_mosaic (and soon, jointcal) results in to calibrate the catalogs used by validate_drp. This feature has only been tested in a one-off sense, because we currently don't have any CI processing of a dataset large enough to run meas_mosaic/jointcal. Once that's addressed, we should finish making it possible to utilize jointcal results in the main driver scripts used by SQuaSH and enable these tests in CI.

If anyone knows of a ticket for adding larger datasets to CI, please add it as a blocker.

#### Activity

Hide
John Parejko added a comment -

I'm taking this on for September, to facilitate the meas_mosaic/jointcal comparison.

Show
John Parejko added a comment - I'm taking this on for September, to facilitate the meas_mosaic/jointcal comparison.
Hide
John Parejko added a comment -

As to the question about larger datasets: I don't see why validation_data_hsc isn't big enough? It's got 4 full focal plane exposures in I band, which should be enough to get a reasonably interesting fit in jointcal.

Show
John Parejko added a comment - As to the question about larger datasets: I don't see why validation_data_hsc isn't big enough? It's got 4 full focal plane exposures in I band, which should be enough to get a reasonably interesting fit in jointcal.
Hide
John Parejko added a comment -

Note that WIDE tract: 9372 filter: HSC-R has only 27 visits, so is a reasonably small test case. I have a script for it here:

 /scratch/parejkoj/compare/scripts/validate-SSP_WIDE_9372_HSC-R.sl 

Show
John Parejko added a comment - Note that WIDE tract: 9372 filter: HSC-R has only 27 visits, so is a reasonably small test case. I have a script for it here: /scratch/parejkoj/compare/scripts/validate-SSP_WIDE_9372_HSC-R.sl
Hide
John Parejko added a comment -

Simon Krughoff: can you please review this PR? It's short.

Jim Bosch: can you please confirm that the meas_mosaic output in /project/parejkoj/DM-11783/validate-meas_mosaic is "reasonable" (for whatever definition of reasonable you like)? You can compare it with the processCcd output for the same tracts in /project/parejkoj/DM-11783/validate-singleFrame.

Show
John Parejko added a comment - Simon Krughoff : can you please review this PR? It's short. Jim Bosch : can you please confirm that the meas_mosaic output in /project/parejkoj/ DM-11783 /validate-meas_mosaic is "reasonable" (for whatever definition of reasonable you like)? You can compare it with the processCcd output for the same tracts in /project/parejkoj/ DM-11783 /validate-singleFrame .
Hide
Simon Krughoff added a comment -

Seems fine. There are a few comments. The major one is how the skipTEx parameter is passed around.

Sorry for the delay. The SciPlat workshop took all my attention last week.

Show
Simon Krughoff added a comment - Seems fine. There are a few comments. The major one is how the skipTEx parameter is passed around. Sorry for the delay. The SciPlat workshop took all my attention last week.
Hide
Jim Bosch added a comment -

I've just spot-checked a few of the plots, but everything I've looked at seems fine. As I think we've discussed, some of the model-fitting in the check_astrometry and check_photometry plots isn't very robust and probably can't be trusted, but most of the plots seem quite usable. It was interesting to see that sometimes running meas_mosaic produces essentially the same AM1 (astrometric scatter) with a much smaller AF1 (astrometric outlier fraction), but I don't think that's problematic.

Show
Jim Bosch added a comment - I've just spot-checked a few of the plots, but everything I've looked at seems fine. As I think we've discussed, some of the model-fitting in the check_astrometry and check_photometry plots isn't very robust and probably can't be trusted, but most of the plots seem quite usable. It was interesting to see that sometimes running meas_mosaic produces essentially the same AM1 (astrometric scatter) with a much smaller AF1 (astrometric outlier fraction), but I don't think that's problematic.
Hide
John Parejko added a comment -

Thanks for the reviews. I've filed DM-12975 about merging the new astrometry KPMs into verify_metric, and fixed the things you commented on, Simon Krughoff.

Merged and done.

Show
John Parejko added a comment - Thanks for the reviews. I've filed DM-12975 about merging the new astrometry KPMs into verify_metric, and fixed the things you commented on, Simon Krughoff . Merged and done.

#### People

Assignee:
John Parejko
Reporter:
Jim Bosch
Reviewers:
Jim Bosch, Simon Krughoff
Watchers:
Jim Bosch, John Parejko, Michael Wood-Vasey, Simon Krughoff
Votes:
0 Vote for this issue
Watchers:
4 Start watching this issue

#### Dates

Created:
Updated:
Resolved:

#### Jenkins

No builds found.