# Rerun and create a repository for CFHT astrometry test.

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1
• Team:
SQuaRE

#### Description

Understand, re-rerun, and recreate clean version of Dominique Boutigny 's CFHT astrometry test for the astrometry RMS for two sample CFHT observations. This test is on the NCSA machines in
/lsst8/boutigny/valid_cfht

Create a repository for this test with an eye toward it becoming integrated in a validation suite for the stack.

#### Activity

Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

Yes, this is leading toward a key piece of the long term validation infrastructure. This Story is part of the Epic DM-3864: "Integration Dataset for metrics and regression tests - Part I":

This present DM-4706 was covering specifically adding Dominique Boutigny 's astrometric performance check.

You're very right to call out the connection of this epic and ci_hsc. I've just added DM-4817 as a Story under Epic DM-3864 to call this out explicitly. My intention is to develop a few more tests here and then look through the ci_hsc in the next month.

I'm honestly not familiar with the state of how much of ci_hsc is runnable in the current DM stack (e.g., building obs_subaru with lsstsw a4d9de0 and lsst_build 015c01a current fails on the psycopg2 dependency).

The idea of "validation" vs. "testing" is 2-fold. One: it's a test of the integration rather than of specific modules. But two, and the distinction from continuous integration, is that the validation effort will be runnable at a variety of scales:
1. Quick 1-minute turn around for continuous integration.
2. Perhaps nightly 30-minute scale processing to alert if there's been any more subtle regression in performance.
3. Weekly hours-long processing scale to calculate more detailed performance metrics that are part of the goals of the DM stack.

The bin/check_astrometry.py in lsst_dm_stack_demo is exactly the same code that's being further developed in validate_drp. The duplication here is exactly intentional. lsst_dm_stack_demo has been a very loosely "validation" that the stack kind of actually works after an install. There's no strong current intent to develop lsst_dm_stack_demo that much further as part of Epic DM-3864. We can decide later whether lsst_dm_stack_demo should explicitly import from validate_drp, but my preference would be to actually not create any additional dependencies for lsst_dm_stack_demo until we have a clearer idea of the purpose of the lsst_dm_stack_demo.

Show
Michael Wood-Vasey [X] (Inactive) added a comment - Yes, this is leading toward a key piece of the long term validation infrastructure. This Story is part of the Epic DM-3864 : "Integration Dataset for metrics and regression tests - Part I": This present DM-4706 was covering specifically adding Dominique Boutigny 's astrometric performance check. You're very right to call out the connection of this epic and ci_hsc. I've just added DM-4817 as a Story under Epic DM-3864 to call this out explicitly. My intention is to develop a few more tests here and then look through the ci_hsc in the next month. I'm honestly not familiar with the state of how much of ci_hsc is runnable in the current DM stack (e.g., building obs_subaru with lsstsw a4d9de0 and lsst_build 015c01a current fails on the psycopg2 dependency). The idea of "validation" vs. "testing" is 2-fold. One: it's a test of the integration rather than of specific modules. But two, and the distinction from continuous integration, is that the validation effort will be runnable at a variety of scales: 1. Quick 1-minute turn around for continuous integration. 2. Perhaps nightly 30-minute scale processing to alert if there's been any more subtle regression in performance. 3. Weekly hours-long processing scale to calculate more detailed performance metrics that are part of the goals of the DM stack. The bin/check_astrometry.py in lsst_dm_stack_demo is exactly the same code that's being further developed in validate_drp. The duplication here is exactly intentional. lsst_dm_stack_demo has been a very loosely "validation" that the stack kind of actually works after an install. There's no strong current intent to develop lsst_dm_stack_demo that much further as part of Epic DM-3864 . We can decide later whether lsst_dm_stack_demo should explicitly import from validate_drp, but my preference would be to actually not create any additional dependencies for lsst_dm_stack_demo until we have a clearer idea of the purpose of the lsst_dm_stack_demo.
Hide
John Swinbank added a comment -

Thanks for that; sounds useful & reasonable.

To clarify the status of obs_subaru: we're aware that it doesn't build with lsstsw at present. The fix is all ready to go (DM-3518), but unfortunately we are blocked from merging by DM-4733. As soon as that's resolved we should be up and running and will press ahead with getting ci_hsc integrated (that's DM-4307). This is essential for the HSC port, so it's top priority from our point of view.

From a quick look at the ci_hsc and validate_drp packages (I claim expertise on neither), it seems like ci_hsc has a more thought-through infrastructure for adding and running tests (based on SCons, which seems like it would be just what you want for this kind of thing), but the individual tests are pretty simplistic compared to the astrometry check in validate_drp. It sounds like it would be a great idea to investigate whether they can adopt a common approach before they diverge further. In other words, on DM-4817!

Show
John Swinbank added a comment - Thanks for that; sounds useful & reasonable. To clarify the status of obs_subaru : we're aware that it doesn't build with lsstsw at present. The fix is all ready to go ( DM-3518 ), but unfortunately we are blocked from merging by DM-4733 . As soon as that's resolved we should be up and running and will press ahead with getting ci_hsc integrated (that's DM-4307 ). This is essential for the HSC port, so it's top priority from our point of view. From a quick look at the ci_hsc and validate_drp packages (I claim expertise on neither), it seems like ci_hsc has a more thought-through infrastructure for adding and running tests (based on SCons, which seems like it would be just what you want for this kind of thing), but the individual tests are pretty simplistic compared to the astrometry check in validate_drp . It sounds like it would be a great idea to investigate whether they can adopt a common approach before they diverge further. In other words, on DM-4817 !
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

David Nidever [X] Could you kindly take a moment to review this issue?

Show
Michael Wood-Vasey [X] (Inactive) added a comment - David Nidever [X] Could you kindly take a moment to review this issue?
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

David Nidever [X] Could you kindly take a look at this ticket?

If you are unable to review it, please let me know and I will ask someone else.

Show
Michael Wood-Vasey [X] (Inactive) added a comment - David Nidever [X] Could you kindly take a look at this ticket? If you are unable to review it, please let me know and I will ask someone else.
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

This issue ended up being completely covered by DM-4707, DM-4708, DM-4709, DM-4814, DM-4827. I am closing this issue as Done.

By the time the prototype was reviewed (DM-4709), and the generalizations to SDSS (DM-4708), and DECam (DM-4707, DM-4814), and then further development to standardize structure of the code (DM-4827), this present issue, DM-4706, was covered and reviewed.

Show
Michael Wood-Vasey [X] (Inactive) added a comment - This issue ended up being completely covered by DM-4707 , DM-4708 , DM-4709 , DM-4814 , DM-4827 . I am closing this issue as Done. By the time the prototype was reviewed ( DM-4709 ), and the generalizations to SDSS ( DM-4708 ), and DECam ( DM-4707 , DM-4814 ), and then further development to standardize structure of the code ( DM-4827 ), this present issue, DM-4706 , was covered and reviewed.

#### People

Assignee:
Michael Wood-Vasey [X] (Inactive)
Reporter:
Michael Wood-Vasey [X] (Inactive)
Watchers:
David Nidever [X] (Inactive), Frossie Economou, John Swinbank, Michael Wood-Vasey [X] (Inactive)
0 Vote for this issue
Watchers:
4 Start watching this issue

#### Dates

Due:
Created:
Updated:
Resolved:

#### Jenkins

No builds found.